import-checker: open all source files as utf-8 Before, we opened in text mode and used the default encoding to interpret the bytes within. This caused problems interpreting some byte sequences in some files. This commit changes things to always open files as UTF-8, which makes the error go away. test-check-module-imports.t now passes on Python 3.5 and 3.6 with this change. Differential Revision: https://phab.mercurial-scm.org/D7225

localrepo: use str for lookup in vars() vars() returns a dict of str. So always use a native str for the key lookup. Differential Revision: https://phab.mercurial-scm.org/D7227

automation: install python3-venv Debian package Debian's python install has a crippled venv by default, as it is lacking ensurepip. When you try to run `python3 -m venv` it tells you to install `python3-venv`. So this commit does that in our automation environment so we can fully test installing Mercurial using venv+pip with the system Python. Differential Revision: https://phab.mercurial-scm.org/D7229

tests: look for ensurepip before using venv Debian appears to cripple the venv module by default by removing the associated ensurepip functionality. (The module isn't present at all.) This caused test-install.t to fail when using the Debian python3 unless the python3-venv package was installed. This commit introduces a new hghave requirement for detecting ensurepip and makes the Python 3 install variant conditional on its presence. This should make test-install.t pass when using an incomplete Debian Python. Differential Revision: https://phab.mercurial-scm.org/D7228

automation: avoid '~' in the temp directory on Windows If a long-ish username is used, the environment variable ends up with a '~' to be 8.3 path compatible. That in turn causes a handful of tests (mostly ssh related) to add quotes around $TESTMP. I have no AWS experience, so I have no idea if this is the proper way to do it. But I've hit this problem locally, and redirecting the directory is a workaround. I don't recall if the directory is created on demand by the test harness, but presumably if this is configured before the machine boots, Windows will do it for us. Differential Revision: https://phab.mercurial-scm.org/D7130

tests: use venv on Python 3 This test was failing in some Python 3 environments because `$PYTHON -m virtualenv` was somehow resulting in Python 2 being used. Why, I'm not sure. Python 3 includes virtualenv in the standard library as the `venv` module. This commit changes test-install.t to use `$PYTHON -m venv` on Python 3 and `$PYTHON -m virtualenv` on Python 2 (if available). I chose to make some test output duplicated because we can't have nested conditionals and there is no easy way to express ORing of hghave checks. Differential Revision: https://phab.mercurial-scm.org/D7224

tests: remove HGALLOWPYTHON3 reference This variable was removed from setup.py in c3e10f705a6c. Differential Revision: https://phab.mercurial-scm.org/D7223

run-tests: use byte strings for inserted output We were inserting str on Python 3 which resulted in mixed str/bytes types on the list. This would later blow up when trying to write str to the .err file opened in bytes mode. Differential Revision: https://phab.mercurial-scm.org/D7222

contrib: require Python 3.7 for byteify-strings.py bb509f39d387 made an error, it's actually 3.7 that introduced token.COMMENT. Differential Revision: https://phab.mercurial-scm.org/D7220

hghave: fix bytes/string issue on Python 3