py3: open chistedit file in binary mode using vfs We write bytes to the file, so it should be open in binary mode. Opening it via the vfs takes care of that for us. Now you'll get yet a different traceback if you try to confirm you histedit plan. Differential Revision: https://phab.mercurial-scm.org/D7185

py3: avoid another b''.format() in chistedit Now you'll get a different traceback if you try to confirm you histedit plan. Differential Revision: https://phab.mercurial-scm.org/D7184

py3: render message about conflicts in chistedit code Now you can also reorder commits that (potentially) conflict. Just don't try to confirm the changes yet (because then it crashes). Differential Revision: https://phab.mercurial-scm.org/D7183

py3: handle keypresses in chistedit Now you can navigate and change the action for a commit. You can also reorder commits, as long as that doesn't result in a conflict (then it crashes). Differential Revision: https://phab.mercurial-scm.org/D7182

py3: make chistedit render Now you can see the list of commits, but it crashes when you press a key. Differential Revision: https://phab.mercurial-scm.org/D7181

py3: fix exception display encoding in contrib/simplemerge.py Differential Revision: https://phab.mercurial-scm.org/D7191

py3: fix exception message check in test-linerange.py's testOutOfRange Differential Revision: https://phab.mercurial-scm.org/D7192

py3: fix exception message encoding in scmutil.py's simplekeyvaluefile.read Differential Revision: https://phab.mercurial-scm.org/D7193

py3: fix crecord.py's editpatchwitheditor exception message encoding Differential Revision: https://phab.mercurial-scm.org/D7194

py3: fix exception message encoding in infinitepush Differential Revision: https://phab.mercurial-scm.org/D7195

py3: fix fsmonitor's _handleunavailable exception message encoding Differential Revision: https://phab.mercurial-scm.org/D7196

packaging: update built-in Fedora support to Fedora 31 This is now quite easy ...

packaging: refactor "fedora29" target to a single more generic "fedora" target Fedora moves fast in version numbers, and often with Mercurial packaging being backwards compatible. Also, most people use the system package. There is thus much work and tech debt and little value in providing explicit built-in support for several versions. Thus, only aim for providing built-in support for latest Fedora version, and make it easy to update.

packaging: make dockerrpm fedora target more generic Fedora moves fast in version numbers, and often with Mercurial packaging being backwards compatible. Thus, only aim for providing built-in support for latest Fedora version, and make it easy to update. With this refactoring, 'dockerrpm fedora31' also works. 'dockerrpm fedora' will use the 'fedora:latest' Docker image.

packaging: use "python3" for fedora29 ... and as buildrpm default Change the buidrpm default. The CentOS targets explicitly use "python", and changing the default will only influence Fedora 29. A Python 3 package needs python3 dependencies, so pythonexe (and pythonver) is used for specifying dependencies. Other OS versions will keep using "python" as before ... or potentially change to explicit "python2". Fedora 29 packages can thus also still be built for Python 2 - just not in the docker image that is updated for Python 3.

packaging: use "--python python" for centos7 to avoid explicit "python2" This is a partial backout of 92a51a45d44c. We will need to be able to control whether package dependencies are python2 or python3. Generally (at least in recent Fedora), the package prefix match the name of the python executable ... but CentOS 7 doesn't use the python2 prefix in package name or alias for python-docutils yet, so just keep centos7 in the unversioned "python" world. Change the new (unused) buildrpm "--python3" option (introduced in a6dcac6454c1) to "--python python3" to get a more generic method for explicit control over whether we use python, python2 or python3.

packaging: fix docker-centos5 - use pythonexe and set to "python" as before Fix 92a51a45d44c .

packaging: move dockerrpm output directory creation to dockerrpm Avoid having to compute the directory in two places in different environments.

packaging: drop "support" for unsupported Fedora versions Fedora 31 has just been released, and Fedora 29 will be EOL in a month. Don't spend any time thinking about dead stuff.

mail: black wants to add this blank line I can't figure out how this got overlooked on previous runs, but here we are. It looks like the culprit change is already public?

hghave: verify we have a black that is new enough for our format We require what is currently the absolute latest black, so let's be paranoid.

contrib: fix up example fix configuration for our move to released black

phabricator: use True primitive instead of b'true' for phabupdate actions Something I'd missed in the creatediff port. This didn't matter before with the old PHP form style wireformat, but breaks with the new arcanist format. Differential Revision: https://phab.mercurial-scm.org/D7152

setup: allow py3 install without env vars 5.2 is the first release of Mercurial where py3 support is expected to be widely used, therefore we should allow installing it without hoop-jumping. Differential Revision: https://phab.mercurial-scm.org/D7151

formatting: drop `grey`, our custom black version Now that the official black has all we want, we can drop this.

formatting: using black to check for formatting

formatting: run black version 19.10b0 on the codebase The latest version of black is out and contains the change we needed. So we can start using it now. note: `test-check-format.t` will complains about this changes because it still use `grey` and need to be migrated to `black`. See next changesets for this.

packaging: fix buildrpm whitespace

packaging: drop outdated buildrpm "tested on" comment Packaging usually works on other versions too, and it is not efficient to maintain the list in repo. It is already out of sync with the Makefile targets.

packaging: also include hgweb.wsgi in rpms

packaging: introduce Python3 support as buildrpm --python3 Just overrule the HGPYTHON3 warning.

packaging: be explicit about Python version in rpm spec Fedora 31 has Python3 at /usr/bin/python ... but expect everybody to not just find python in $PATH but be explicit about whether they want python2 or python3. mercurial.spec just used 'python' and would fail when it unknowingly used Python 3 and ended up with Mercurial setup.py reporting "Python 3.7 detected." and talking about the HGPYTHON3 environment variable. For now, just be explicit about using system python2 as python executable when building rpms.

packaging: make python snippets in rpm building python3 compatible Fedora 31 has Python3 at /usr/bin/python, and buildrpm would fail on snippets that use python2 syntax. Instead of forcing python2, just accept for the future while staying backwards compatible.

py3: fix patchbomb to accept non-ASCII header value for email preview Since mail.headencode() is disabled by -n/--test, non-ASCII header value has to be allowed. Spotted by Denis Laxalde.

tests: check patchbomb with a non-ascii commit message This fails on Python 3 but gets fixed in the next changeset.

formatter: fix handling of None value in templater mapping For historical reasons, None in mapping dict means there's no such keyword, and falls back to b"". That's fine in log templates where mapping item is generally a callable returning a value (which may be None,) but the formatter directly puts an "evaluated" value in the mapping. So the None value has to be lifted to wrappedvalue(None) to avoid confusion in the template engine.

config: add support for defaultvalue of list of printable elements

config: fix -Tjson to not crash due to unsupported defaultvalue types Maybe it isn't great to ignore unsupported types at all, but otherwise "hg config -Tjson" would crash.

tests: handle Message-Id email header possible wrapping The "Message-Id" header will get wrapped with a new line when exceeding 75 characters on Python 3 (see changeset 7d4f2e4899c5 introducing usage of email.header.Header.encode and respective doc). This will occur in an unpredictable manner depending on the hostname's length. To make the test output consistent across Python versions and hostname configuration, we add a filter to unwrap this header value.

py3: leverage pycompat.long

packaging: remove version info from Breaks+Replaces in Debian package The versioned Breaks: and Replaces: cause problem when trying to install our package over the one in Debian. $ sudo apt install ./packages/debian-buster/mercurial_5.2~rc0+15-buster-a2ff3aff81d2_amd64.deb Reading package lists... Done Building dependency tree Reading state information... Done Note, selecting 'mercurial' instead of './packages/debian-buster/mercurial_5.2~rc0+15-buster-a2ff3aff81d2_amd64.deb' Some packages could not be installed. This may mean that you have requested an impossible situation or if you are using the unstable distribution that some required packages have not yet been created or been moved out of Incoming. The following information may help to resolve the situation: The following packages have unmet dependencies: mercurial : Breaks: mercurial-common (< 5.2~rc0+15-buster-a2ff3aff81d2) but 5.2~rc0-1 is to be installed E: Unable to correct problems, you have held broken packages. Removing version information resolves the situation.

py3: fix generated non-ascii message in test-notify.t The resulting "Subject:" header contains an encoded word in Python 3 so we have to add distinct outputs for Python 3 but underlying values are the same: >>> from email.header import decode_header >>> decode_header('=?utf-8?b?w6AuLi4=?=') [(b'\xc3\xa0...', 'utf-8')]

py3: decode encoding literal before passing to .decode() bytes.decode(<encoding>) wants an str as "encoding" parameter, it raises a TypeError if given a bytestring. encoding.encoding and encoding.fallbackencoding are bytes values.

py3: decode payload of notify email This fixes one UnicodeEncodeError in test-notify.t:422 when testing the notify hook with non-ascii content (there are more later). We only decode on Python 3, since it's not safe for sure on Python 2.

py3: decode email headers with mail.headdecode() in notify extension

py3: use stdlib's parseaddr() to get sender header in notify extension In Python 3, email headers are unicode string so using stringutil.email() will not work as it compares with bytestring. So let's use email.utils.parseaddr() from the stdlib which has a consistent behavior across Python versions. The same is done in patchbomb extension already.

py3: use a BytesParser in notify extension This is the first step to make the "long line" case in test-notify.t pass by fixing a UnicodeDecodeError on Python 3. We alias a parsebytes() in mail module, similarly as we already have a parse() function for Python 2 and Python 3 compatibility.

py3: fix headencode() with display=False We previously called str() on a email.header.Header object. On Python 2, this returns a bytestring and the __str__ method is actually an alias to .encode() method. On Python 3, __str__ does not perform encoding (and returns a unicode string). To keep a consistent behavior across Python versions, we explicitly use .encode() and we wrap the result with encoding.strtolocal() to get a bytestring in all cases. As a side effect of forcing bytes conversion, we need to decode back in _addressencode(). This is to make test-notify.t pass on Python 3. Also note that headers are now encoded in some patchbomb tests; this is because the charset is not always "us-ascii" ("iso-8859-1" otherwise) on Python 3.

mail: catch LookupError in headdecode() We already catch this exception in _encode() (called by headencode()). It gets raised when running test-notify.t with Python 3.

py3: account for extra line break in email headers in test-notify.t Long headers appears to be wrapped with new lines. In test-notify.t, we have a "filter.py" that replaces "\n" by " ", so we get an extra space in a Message-Id with a long value.

py3: use as_bytes() method of EmailMessage In Python 3, as_bytes() corresponds to as_string() in Python 2.

py3: use %d instead of %s when formatting an int into a bytestring The latter wasn't noticed before since no tests exercise --confirm at all, let alone on an existing DREV. The former is only hit during an intermittent network issue during amending at the end of a phabsend, so doesn't seem testable. Differential Revision: https://phab.mercurial-scm.org/D7153

packaging: ship only a single binary Debian package We merge the mercurial and mercurial-common binary packages into a single mercurial package. This is essentially to ease installation (and upgrade) using a simple "dpkg -i" command. This also simplifies debian/rules by removing arch (in)dependent cleanups during installation. We have the mercurial binary Breaks: and Replaces: mercurial-common so that the latter will be removed upon upgrade. Also note the change from "override_dh_install" to "override_dh_auto_install" in debian/rules: this is because we do not want "make install" to be run automatically as we need the --install-layout=deb of "setup.py install" (otherwise, files would end up in $DESTDIR/usr/local).

packaging: avoid running bare "make install" in debian/rules We change the "override_dh_install" target to "override_dh_auto_install" in debian/rules (see dh_auto_install(1) for details). This is because we do not want "make install" to be run automatically as we need the --install-layout=deb of "setup.py install". Otherwise, files end up in $DESTDIR/usr/local. At the moment, this is not a problem since files installed in debian/tmp (the default location) are not packed into binary packages (because there are debian/mercurial and debian/mercurial-common directories). This is cleaner to avoid running make more than needed.

packaging: distinguish clean and build steps from install in Debian

packaging: also move Debian .buildinfo file in output directory

packaging: upgrade packages distributed with Windows installers We like to use the latest versions of things. I added pywin32-ctypes to the explicit list of packages so running on !Windows will pull in the dependency.

automation: install Python 2.7.17, 3.7.5, and PyPy 7.2.0 These were all recently released and we should use them in automation.

contrib: install Python 2.7.17 and 3.7.5 in Windows environment These are the latest Python versions and we should use them.

packaging: use /usr/bin/python3 shebang for scripts in Debian "hg" script is already correct because it is handled by setup.py but "hg-ssh" will be rewritten by dh_python into "/usr/bin/python" which is not wanted as we target Python 3. By passing --shebang=/usr/bin/python3 to dh_python3, we force shebangs to be set with this value.