PXP-459 Futurize/modernize (make it python 2/3 compatible)

[yonran]

PXP-459

Use the futurize script and future backport library to make this library compatible with both python2 and python3. Then I manually switched from future.utils and builtins to six.moves to remove the future dependency and to avoid changing python2 behavior (urlparse).

Generally, these are the steps to using futurize:

• Install the future~=0.18.2 library (which comes with the futurize script)
• Run futurize --stage1 *.py remoteobjects examples tests --write --nobackups on all the files. futurize --stage1 consists of “safe” fixes: “The goal for this stage is to create most of the diff for the entire porting process, but without introducing any bugs.”, but “The output of stage 1 will probably not (yet) run on Python 3.”
• Run futurize --stage2 --unicode-literals *.py remoteobjects examples tests --write --nobackups on all the files. futurize --stage2. Stage2 makes the code import __future__ and future backports so that hopefully it behaves the same on python2 and python3.
• Separating text from bytes
  • Any binary packets must use binary literals b'…' and functions like bytes
  • Any python reflection must use native literals '' and functions like future.util.native_str
  • Ensure that text strings are handled consistently and optionally convert them to unicode u'…' or future’s builtins.str
• Post-conversion: run tests and check What else you need to know for tricky cases that require manual tweaking

Here are notes specific to remoteobjects

• I did not enable from __future__ import unicode_literals since most strings are python attributes (keys of __dict__, __get__, __setattr__, __name__, etc) which must be native_str in python2 and gives weird errors if you use unicode (e.g. error on import remoteobjects.fields: “AttributeError: 'module' object has no attribute 'fields'” because one of the functions’ __name__ was unicode instead of native_str)
• In python2, RemoteObject Fields parsed from JSON still return str or unicode depending on what simplejson returns
• In python3, RemoteObject Fields parsed from JSON are unicode
• In python3, I think RemoteObject.get url and Field setter can take bytes or unicode since the underlying urllib.client.urlparse, simplejson.dumps take either bytes or unicode.
• In python3, we UTF-8 decode before JSON loads; this replaces more invalid JSON with the replacement character than before (see new test in test_http.py)
• I switched from future to six because six worked better in a number of ways:
  • I don’t understand the metaclasses, but future.utils.with_metaclass threw exceptions whereas six.with_metaclass worked.
  • future changes the behavior of urlparse, urljoin. In particular, it provides backports that always return unicode and then throw an exception when you mix str and unicode args (parse.py). On the other hand, six just renames the functions from python2 urllib that allow mixing unicode with str.
  • importing future.standard_library raises ImportError python 3.12, since that library is abandoned.
• I also ran modernize to see what it would do differently. The 3 main differences were:
  • modernize added from __future__ import absolute_import to every file which is unnecessary for the most part
  • modernize omitted some list() wrappers in for x in d.keys():, which is a bug in modernize so I filed a PR Add back list() when iterating over d.items() in a for statement PyCQA/modernize#262
  • modernize used six.moves.urllib.urlparse instead of future’s backfilledurllib.urlparse. I went with modernize’s solution.

Testing

virtualenv -p python2.7 venv2
virtualenv -p python3 venv3
. venv2/bin/activate && pip install -r requirements.txt -r requirements-test.txt && python -m unittest
. venv3/bin/activate && pip install -r requirements.txt -r requirements-test.txt && python -m unittest

[stanimoto]

Looks great!

[yonran]

@stanimoto since your review at f7b9c1d, I added a few more minor changes to be91cde. Then I performed force pushes with equal trees from be91cde to 3116611 to 532fba7 in order to clean up history (combine the futurize steps, rearrange commits which could be applied to python2, remove references to future which is entirely replaced with six).

I don’t have permission to rebase and merge to this repo (“Only those with write access to this repository can merge pull requests.”). Would you mind doing so?

[stanimoto]

I've updated the access to this repo. Could you try it now?