hgweb: fix a few `str` type conditional for py3 Differential Revision: https://phab.mercurial-scm.org/D7534

repair: fix an `isinstance(nodelist, str)` check for py3 All of the callers appear to pass a list, so this doesn't fix anything in core hg. But maybe out of tree extensions use this shortcut. Differential Revision: https://phab.mercurial-scm.org/D7533

log: map None rev to wdirrev when filtering revisions with --line-range When 'hg log -f --line-range <file>,<range>' is invoked with <range> containing uncommitted changes, the command crashes on Python 3 as follows: [...] File "/usr/lib/python3/dist-packages/mercurial/commands.py", line 4725, in log revs, differ = logcmdutil.getlinerangerevs(repo, revs, opts) File "/usr/lib/python3/dist-packages/mercurial/logcmdutil.py", line 933, in getlinerangerevs if rev not in userrevs: File "/usr/lib/python3/dist-packages/mercurial/smartset.py", line 969, in __contains__ if l < x: TypeError: '<' not supported between instances of 'int' and 'NoneType' The None value is because requested line range has uncommitted changes, so 'rev' is the working directory revision. This only occurs in Python 3 as Python 2 allows comparing None with int. As suggested by Yuya Nishihara, mapping None to node.wdirrev resolves the issue and also make the '--line-range' option properly work with -r 'wdir()'. We add extra tests for non-regression and to illustrate handling of 'wdir()'.

tests: check that 'log --line-range' follows uncommitted changes The reason we start walking revisions from the working directory (None revision) in logcmdutil.getlinerangerevs() is because we can follow uncommitted changes. Adding a test to illustrate this based on an uncommitted rename as there was none before. This helps understand the fix in next changeset.

test: don't put $BINDIR in $PATH for test-merge-tools.t We call $BINDIR/hg explicitly anyway, so don't need it in $PATH. This fixes failures when running the test --with-hg=/usr/bin/hg, where we pick up /usr/bin/false as merge tool when we expected not to find it.

webutil: add missing argument to join() Differential Revision: https://phab.mercurial-scm.org/D7516

singlehead: making config item a bool again with the use of `configsuboptions`, the main config item has become a string (unless it's just the default value). This makes it in particular hard to override in a cascade of HGRC files, as we do in Heptapod to re-allow multiple heads on specific repositories while the default behaviour is to forbid them. The added test case reflects that use-case

util: convert an exception to bytes when passing to Abort() I happened to notice this searching for how to convert an exception to bytes in the previous patch. I'm pretty sure I've got a bunch of other instances that use `pycompat.bytestr()` suppressed locally. Differential Revision: https://phab.mercurial-scm.org/D7467

patch: fix a str + bytes issue in an exception handler Flagged by pytype. Differential Revision: https://phab.mercurial-scm.org/D7466

py3: wrap a __func__ in sysbytes() before logging as bytes Differential Revision: https://phab.mercurial-scm.org/D7461

py3: make doc strings containing deprecated '\.' escape sequence raw strings Differential Revision: https://phab.mercurial-scm.org/D7462

shelve: add the missing `create` parameter to the bundlerepo constructor Caught by pytype. Differential Revision: https://phab.mercurial-scm.org/D7458

shelve: fix a missing variable in the exception handler for delete Caught by pytype. I haven't paid much attention to the progress of this extension, but I *think* this was the intent. Differential Revision: https://phab.mercurial-scm.org/D7457

py3: use pycompat.bytestr() instead of pycompat.sysstr() pycompat.sysstr() doesn’t work because it doesn’t accept arguments of type `type` and returns a unicode object on Python3, while the format string wants a bytes-like object.

zeroconf: fix traceback under py3 hg serve under py3 caused struct.error: char format requires a bytes object of length 1 <pulkit25> ah, I think that should be `pycompat.bytechr` instead of chr

cffi: fix build on Python 3 CFFI expects the arguments to be of type str, which means that the string literals should not have the `b` prefix.

pure: use string for another exception in the pure version of base85 That message does not seems tested, but I am assuming that the same reasoning as for the previous changeset applies.

pure: use string for exception in the pure version of base85 Without this change, running the test with python3 and --pure gives the following error:: --- /home/marmoute/src/mercurial-dev/tests/test-import-git.t +++ /home/marmoute/src/mercurial-dev/tests/test-import-git.t.err @@ -518,7 +518,7 @@ > > EOF applying patch from stdin - abort: could not decode "binary2" binary patch: bad base85 character at position 6 + abort: could not decode "binary2" binary patch: b'bad base85 character at position 6' [255] $ hg revert -aq To make the cext implementation, we use a "native" string for the exception. This fix the test failure.

py3: avoid iterating over a literal bytes in highlight In Python 3, iterating over a bytes literal yields integers. Since we use the value in `text.replace()`, this fails on Python 3 with the following trackback: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/mercurial/hgweb/hgwebdir_mod.py", line 378, in run_wsgi for r in self._runwsgi(req, res): File "/usr/lib/python3/dist-packages/mercurial/hgweb/hgweb_mod.py", line 326, in run_wsgi for r in self._runwsgi(req, res, repo): File "/usr/lib/python3/dist-packages/mercurial/hgweb/hgweb_mod.py", line 449, in _runwsgi return getattr(webcommands, cmd)(rctx) File "/usr/lib/python3/dist-packages/mercurial/hgweb/webcommands.py", line 211, in file return _filerevision(web, webutil.filectx(web.repo, web.req)) File "/usr/lib/python3/dist-packages/hgext/highlight/__init__.py", line 72, in filerevision_highlight pygmentize(web, b'fileline', fctx, web.tmpl) File "/usr/lib/python3/dist-packages/hgext/highlight/__init__.py", line 58, in pygmentize field, fctx, style, tmpl, guessfilenameonly=filenameonly File "/usr/lib/python3/dist-packages/hgext/highlight/highlight.py", line 62, in pygmentize text = text.replace(c, b'') TypeError: a bytes-like object is required, not 'int'

relnotes: copy "next" to "5.2" and clear "next" This is the same thing as we did for 5.1 in cba59b338976 (relnotes: copy "next" to "5.1" and clear "next", 2019-08-01). Differential Revision: https://phab.mercurial-scm.org/D7231

py3: avoid `b'%s' % type(...)` in a ProgrammingError Differential Revision: https://phab.mercurial-scm.org/D7363

py3: fix sorting of obsolete markers in obsutil (issue6217) This is similar to 01e8eefd9434 and others. We move the sortedmarkers() function from exchange module to obsutil.

py3: fix handling of ctrl keys in crecord (issue6213) The "keypressed" value in handlekeypressed() is a key name obtained by curses's getkey(); this can be a multibyte string for special keys like CTRL keys. Calling curses.unctrl() with such a value fails on Python 3 with a TypeError as described in issue6213. (On Python 2, this does not crash, but I'm not sure the result is correct, though it does no matter here.) So instead of calling unctrl(), we compare "keypressed" with the expected "^L" obtained by curses.ascii.ctrl("L").

py3: keep "keypressed" a native str in crecord This will help in the next changeset by avoiding a decode step. Also, the actual bytes conversion seems superfluous since values coming from curses's getkey() will be a native string. As a consequence, we open the "testcommands" file (used in test-interactive-curses.t) in text mode.

py3: compare response of crecord's confirmationwindow with str confirmationwindow() returns a native string, as a result of calling chr() on getch(). On Python 3, response.lower().startswith(b"y") leads to a TypeError. This fixes a crash when typing "r" in the curses interface of interactive commit.

py3: compare http server's command with a native string The "command" attribute is an str, so comparing with a bytes would not work on Python 3. This might solve issues in test-lfs-serve-access.t that happens sometimes (especially in CI): --- /hgwork/src/tests/test-lfs-serve-access.t +++ /hgwork/src/tests/test-lfs-serve-access.t.err @@ -163,11 +163,13 @@ $ cat $TESTTMP/access.log $TESTTMP/errors.log $LOCALIP - - [$LOGDATE$] "POST /missing/objects/batch HTTP/1.1" 404 - (glob) + $LOCALIP - - [05/Nov/2019 16:32:34] "{"objects": [{"oid": "f03217a32529a28a42d03b1244fe09b6e0f9fd06d7b966d4d50567be2abe6c0e", "size": 20}], "operation": "download"}" HTTPStatus.BAD_REQUEST - $LOCALIP - - [$LOGDATE$] "GET /subdir/mount/point?cmd=capabilities HTTP/1.1" 200 - (glob) $LOCALIP - - [$LOGDATE$] "GET /subdir/mount/point?cmd=batch HTTP/1.1" 200 - x-hgarg-1:cmds=heads+%3Bknown+nodes%3D x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$ partial-pull (glob) $LOCALIP - - [$LOGDATE$] "GET /subdir/mount/point?cmd=getbundle HTTP/1.1" 200 - x-hgarg-1:bookmarks=1&bundlecaps=HG20%2Cbundle2%3DHG20%250Abookmarks%250Achangegroup%253D01%252C02%252C03%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Aphases%253Dheads%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps%250Arev-branch-cache%250Astream%253Dv2&cg=1&common=0000000000000000000000000000000000000000&heads=525251863cad618e55d483555f3d00a2ca99597e&listkeys=bookmarks&phases=1 x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$ partial-pull (glob) $LOCALIP - - [$LOGDATE$] "POST /subdir/mount/point/.git/info/lfs/objects/batch HTTP/1.1" 200 - (glob) $LOCALIP - - [$LOGDATE$] "GET /subdir/mount/point/.hg/lfs/objects/f03217a32529a28a42d03b1244fe09b6e0f9fd06d7b966d4d50567be2abe6c0e HTTP/1.1" 200 - (glob) + $LOCALIP - - [05/Nov/2019 16:32:34] code 400, message Bad request version ('"download"}') Blobs that already exist in the usercache are linked into the repo store, even though the client doesn't send the blob. @@ -195,6 +197,7 @@ server2/.hg/store/lfs/objects/f0/3217a32529a28a42d03b1244fe09b6e0f9fd06d7b966d4d50567be2abe6c0e $ "$PYTHON" $RUNTESTDIR/killdaemons.py $DAEMON_PIDS $ cat $TESTTMP/errors.log + $LOCALIP - - [05/Nov/2019 16:32:34] code 400, message Bad request version ('"download"}') $ cat >> $TESTTMP/lfsstoreerror.py <<EOF > import errno (from https://ci.hg.gregoryszorc.com/job-info/hg-committed-ca3dca416f8d5863ca6f5a4a6a6bb835dcd5feeb-debian10-cpython-3.7-0)

py3: don't use bytes with vars() or __dict__ Inspired by D7227. These were all the remaining instances I could find. Differential Revision: https://phab.mercurial-scm.org/D7230

Added signature for changeset ca3dca416f8d

Added tag 5.2 for changeset ca3dca416f8d

py3: add inline comment about encoding issue of str(Abort())

py3: do not reimplement Abort.__str__() on Python 2 It isn't necessary on Python 2, and the default implementation should be better than our BaseException_str() clone.

tests: write out file using bytes I/O The encoding of sys.stdout varies between Python versions. So using a one-liner to write a file from a Unicode string is not deterministic. This commit writes out the file using bytes I/O to ensure we have exactly the bytes we want in the file. This change fixes a test failure in Python 3.5/3.6. Differential Revision: https://phab.mercurial-scm.org/D7226

import-checker: open all source files as utf-8 Before, we opened in text mode and used the default encoding to interpret the bytes within. This caused problems interpreting some byte sequences in some files. This commit changes things to always open files as UTF-8, which makes the error go away. test-check-module-imports.t now passes on Python 3.5 and 3.6 with this change. Differential Revision: https://phab.mercurial-scm.org/D7225

localrepo: use str for lookup in vars() vars() returns a dict of str. So always use a native str for the key lookup. Differential Revision: https://phab.mercurial-scm.org/D7227

automation: install python3-venv Debian package Debian's python install has a crippled venv by default, as it is lacking ensurepip. When you try to run `python3 -m venv` it tells you to install `python3-venv`. So this commit does that in our automation environment so we can fully test installing Mercurial using venv+pip with the system Python. Differential Revision: https://phab.mercurial-scm.org/D7229

tests: look for ensurepip before using venv Debian appears to cripple the venv module by default by removing the associated ensurepip functionality. (The module isn't present at all.) This caused test-install.t to fail when using the Debian python3 unless the python3-venv package was installed. This commit introduces a new hghave requirement for detecting ensurepip and makes the Python 3 install variant conditional on its presence. This should make test-install.t pass when using an incomplete Debian Python. Differential Revision: https://phab.mercurial-scm.org/D7228

automation: avoid '~' in the temp directory on Windows If a long-ish username is used, the environment variable ends up with a '~' to be 8.3 path compatible. That in turn causes a handful of tests (mostly ssh related) to add quotes around $TESTMP. I have no AWS experience, so I have no idea if this is the proper way to do it. But I've hit this problem locally, and redirecting the directory is a workaround. I don't recall if the directory is created on demand by the test harness, but presumably if this is configured before the machine boots, Windows will do it for us. Differential Revision: https://phab.mercurial-scm.org/D7130

tests: use venv on Python 3 This test was failing in some Python 3 environments because `$PYTHON -m virtualenv` was somehow resulting in Python 2 being used. Why, I'm not sure. Python 3 includes virtualenv in the standard library as the `venv` module. This commit changes test-install.t to use `$PYTHON -m venv` on Python 3 and `$PYTHON -m virtualenv` on Python 2 (if available). I chose to make some test output duplicated because we can't have nested conditionals and there is no easy way to express ORing of hghave checks. Differential Revision: https://phab.mercurial-scm.org/D7224

tests: remove HGALLOWPYTHON3 reference This variable was removed from setup.py in c3e10f705a6c. Differential Revision: https://phab.mercurial-scm.org/D7223

run-tests: use byte strings for inserted output We were inserting str on Python 3 which resulted in mixed str/bytes types on the list. This would later blow up when trying to write str to the .err file opened in bytes mode. Differential Revision: https://phab.mercurial-scm.org/D7222

contrib: require Python 3.7 for byteify-strings.py bb509f39d387 made an error, it's actually 3.7 that introduced token.COMMENT. Differential Revision: https://phab.mercurial-scm.org/D7220

hghave: fix bytes/string issue on Python 3

packaging: add support for CentOS 8 The resulting executable has not been tested in detail yet. I ran 'hg version' and 'hg clone', which worked fine (except for extensions acting up due to Python 3). Differential Revision: https://phab.mercurial-scm.org/D7216

packaging: allow choosing python version depending on centos version Differential Revision: https://phab.mercurial-scm.org/D7217

fsmonitor: use stringutil.forcebytestr() instead of str() on an exception Similar to 5fa8ac91190e / D7206, should get test-install.t passing on py3. Differential Revision: https://phab.mercurial-scm.org/D7218

py3: add a __str__ method to Abort This improves the rendering of some exceptions by avoiding raw bytestrings, especially when using --traceback option.

py3: add Python 3 exception output to test-lfs-serve-access.t Similar to a973a75e92bf or 3e9c6cef949b.

automation: install black This should unblock us from running the code formatting test in our automated environment. Differential Revision: https://phab.mercurial-scm.org/D7197

py3: use %d to format an int Avoids a TypeError under py3. Fortunately this is very much an edge case since it requires the user to have deliberately created a local tag of the form 'D\d+' that isn't truthful. Differential Revision: https://phab.mercurial-scm.org/D7215

fsmonitor: normalize exception types to bytes Unavailable.msg should now always be bytes. We also rename Unavailable.__str__ to __bytes__ as it always returns bytes. We make __str__ a simple wrapper that decodes that result to str. There's probably some excessive strutil.forcebytestr() in fsmonitor/__init__.py now. But at least the exceptions around type coercion should now be gone. Differential Revision: https://phab.mercurial-scm.org/D7214

fsmonitor: normalize clock value to bytes We normalize the value returned by watchman because we perform a number of compares with this value in code. So the easiest path forward is to normalize to bytes so we don't have to update many call sites. With this commit, the fsmonitor extension appears to be working with Python 3! Although there are still some failures in edge cases... Differential Revision: https://phab.mercurial-scm.org/D7213

fsmonitor: use next() instead of .next() This is needed for Python 3 compatibility. Differential Revision: https://phab.mercurial-scm.org/D7212

fsmonitor: normalize Watchman paths to bytes Otherwise it will be a str on Python 3 and operations below which operate in the bytes domain will fail. Differential Revision: https://phab.mercurial-scm.org/D7211

fsmonitor: handle unicode keys in tuples In Python 3, keys in the bset tuple are typically str, not bytes. PyBytes_AsString() would return NULL. But we weren't checking the return value and this would lead to a segfault. This commit makes the code type and Python version aware. The Python version specific code is to allow us to utilize a modern API for converting str -> char* without having to allocate an extra PyObject. FWIW I wanted to assume that keys were always str. However, there appear to be some bytes keys in some cases. I haven't debugged this further. Differential Revision: https://phab.mercurial-scm.org/D7210

fsmonitor: make _hashignore compatible with Python 3 The Hasher wants a bytes but we were feeding it a str. Let's use our repr() implementation to return bytes. In addition, the hexdigest() would return a str, which would be compared against a bytes and would always fail. Normalize to bytes so the compare works. Differential Revision: https://phab.mercurial-scm.org/D7209

fsmonitor: normalize hostname to bytes Without this, we get a str/bytes mismatching when using % formatting a few lines below. Differential Revision: https://phab.mercurial-scm.org/D7208

fsmonitor: access repo.root There is no repo._root. It looks like fsmonitor has been busted since this access was introduced in ab1900323b1 in July 2019! Differential Revision: https://phab.mercurial-scm.org/D7207

fsmonitor: coerce watchman exception to bytes Without this, we get errors due to passing str to a function which expects bytes. Differential Revision: https://phab.mercurial-scm.org/D7206

fsmonitor: fix str/bytes mismatch when accessing watchman version There were 2 bugs here. First, keys in the tuple are always str. Second, we needed to normalize the value to bytes to prevent a str/bytes mismatch on Python 3. With this commit, `hg debuginstall` with fsmonitor enabled now works on Python 3. Differential Revision: https://phab.mercurial-scm.org/D7205

fsmonitor: reapply b1f62cd39b5c The recent revendoring of pywatchman undid this changeset. Let's reapply it. This commit was generated by running `hg graft -f b1f62cd39b5c`. It applied cleanly. Differential Revision: https://phab.mercurial-scm.org/D7204

fsmonitor: reapply dd35abc409ee The recent revendoring of pywatchman undid this bug fix. Let's reapply it. Differential Revision: https://phab.mercurial-scm.org/D7203

fsmonitor: remove pywatchman from exclusion rule The recently vendored pywatchman code base is now formatted with black. We can now remove pywatchman from our black exclusion rule. Differential Revision: https://phab.mercurial-scm.org/D7202

fsmonitor: refresh pywatchman with upstream This commit vendors pywatchman commit 259dc66dc9591f9b7ce76d0275bb1065f390c9b1 from upstream without modifications. The previously vendored pywatchman from changeset 16f4b341288d was from Git commit c77452. This commit effectively undoes the following Mercurial changesets: * dd35abc409ee fsmonitor: correct an error message * b1f62cd39b5c fsmonitor: layer on another hack in bser.c for os.stat() compat (issue5811) * c31ce080eb75 py3: convert arguments, cwd and env to native strings when spawning subprocess * 876494fd967d cleanup: delete lots of unused local variables * 57264906a996 watchman: add the possibility to set the exact watchman binary location The newly-vendored code has support for specifying the binary location, so 57264906a996 does not need applied. But we do need to modify our code to specify a proper argument name. 876494fd967d is not important, so it will be ignored. c31ce080eb75 globally changed the code base to always pass str to subprocess. But pywatchman's code is Python 3 clean, so we don't need to do this. This leaves dd35abc409ee and b1f62cd39b5c, which will be re-applied in subsequent commits. Differential Revision: https://phab.mercurial-scm.org/D7201

py3: encode strings before setting rev summary in gnuarch converter

py3: use raw string to query EmailMessage in gnuarch converter

py3: use mail.parsebytes() in gnuarch catlog parser We drop 'catlogparser' attribute now unused.

tests: handle Message-Id line wrapping in test-notify-changegroup.t This fixes this test on Python 3 with a long hostname. See changeset 4128ffba4431 for details.

py3: define and use json.loads polyfill Python 3.5's json.loads() requires a str. Only Python 3.6+ supports passing a bytes or bytearray. This commit implements a json.loads() polyfill on Python 3.5 so that we can use bytes. The added function to detect encodings comes verbatim from Python 3.7.

contrib: require Python 3.6 for byteify-strings.py This script makes use of `token.COMMENT`, which apparently isn't present until Python 3.6. So make the script and its test conditional on Python 3.6.

demandimportpy3: only use lazy extension loader on Python 3.6+ There was an inline comment denoting a bug in the lazy extension loader on Python 3.5 which prevents it from working there. But the code was not conditional on the Python version. The result of this was a myriad of failures on Python 3.5 due to getattr() and friends not working on lazy extension modules. By making extension modules non-lazy on Python 3.5, we reduce the number of test failures from 48 to 22 on that Python version.

ui: flush before prompting for input with readline I was using `hg absorb` with Python 3 and noticed that the prompt was appearing without any output about what would be done. After I answered the prompt, the output was printed to stdout. This appears to be a buffering difference between Python 2 and Python 3. To work around it, this commit adds an explicit flush() before calling the raw input function when readline is used.

histedit: restore hex nodeids to be 12 digits long I accidentally switched from 12 digits to 40 digits while making the code py3-compatible. Thanks to Yuya for noticing. Differential Revision: https://phab.mercurial-scm.org/D7200

tests: fix typo "includfe" Differential Revision: https://phab.mercurial-scm.org/D7180

py3: fix fsmonitor's _watchmantofsencoding exception message encoding Differential Revision: https://phab.mercurial-scm.org/D7190

py3: use native strings as keys into **opts in chistedit Now you should be able to successfully confirm your histedit plan (at least in the case I tried). Even continuing after conflicts and finishing the histedit worked. Differential Revision: https://phab.mercurial-scm.org/D7186

py3: open chistedit file in binary mode using vfs We write bytes to the file, so it should be open in binary mode. Opening it via the vfs takes care of that for us. Now you'll get yet a different traceback if you try to confirm you histedit plan. Differential Revision: https://phab.mercurial-scm.org/D7185

py3: avoid another b''.format() in chistedit Now you'll get a different traceback if you try to confirm you histedit plan. Differential Revision: https://phab.mercurial-scm.org/D7184

py3: render message about conflicts in chistedit code Now you can also reorder commits that (potentially) conflict. Just don't try to confirm the changes yet (because then it crashes). Differential Revision: https://phab.mercurial-scm.org/D7183

py3: handle keypresses in chistedit Now you can navigate and change the action for a commit. You can also reorder commits, as long as that doesn't result in a conflict (then it crashes). Differential Revision: https://phab.mercurial-scm.org/D7182

py3: make chistedit render Now you can see the list of commits, but it crashes when you press a key. Differential Revision: https://phab.mercurial-scm.org/D7181

py3: fix exception display encoding in contrib/simplemerge.py Differential Revision: https://phab.mercurial-scm.org/D7191

py3: fix exception message check in test-linerange.py's testOutOfRange Differential Revision: https://phab.mercurial-scm.org/D7192

py3: fix exception message encoding in scmutil.py's simplekeyvaluefile.read Differential Revision: https://phab.mercurial-scm.org/D7193

py3: fix crecord.py's editpatchwitheditor exception message encoding Differential Revision: https://phab.mercurial-scm.org/D7194

py3: fix exception message encoding in infinitepush Differential Revision: https://phab.mercurial-scm.org/D7195

py3: fix fsmonitor's _handleunavailable exception message encoding Differential Revision: https://phab.mercurial-scm.org/D7196

packaging: update built-in Fedora support to Fedora 31 This is now quite easy ...

packaging: refactor "fedora29" target to a single more generic "fedora" target Fedora moves fast in version numbers, and often with Mercurial packaging being backwards compatible. Also, most people use the system package. There is thus much work and tech debt and little value in providing explicit built-in support for several versions. Thus, only aim for providing built-in support for latest Fedora version, and make it easy to update.

packaging: make dockerrpm fedora target more generic Fedora moves fast in version numbers, and often with Mercurial packaging being backwards compatible. Thus, only aim for providing built-in support for latest Fedora version, and make it easy to update. With this refactoring, 'dockerrpm fedora31' also works. 'dockerrpm fedora' will use the 'fedora:latest' Docker image.

packaging: use "python3" for fedora29 ... and as buildrpm default Change the buidrpm default. The CentOS targets explicitly use "python", and changing the default will only influence Fedora 29. A Python 3 package needs python3 dependencies, so pythonexe (and pythonver) is used for specifying dependencies. Other OS versions will keep using "python" as before ... or potentially change to explicit "python2". Fedora 29 packages can thus also still be built for Python 2 - just not in the docker image that is updated for Python 3.

packaging: use "--python python" for centos7 to avoid explicit "python2" This is a partial backout of 92a51a45d44c. We will need to be able to control whether package dependencies are python2 or python3. Generally (at least in recent Fedora), the package prefix match the name of the python executable ... but CentOS 7 doesn't use the python2 prefix in package name or alias for python-docutils yet, so just keep centos7 in the unversioned "python" world. Change the new (unused) buildrpm "--python3" option (introduced in a6dcac6454c1) to "--python python3" to get a more generic method for explicit control over whether we use python, python2 or python3.

packaging: fix docker-centos5 - use pythonexe and set to "python" as before Fix 92a51a45d44c .

packaging: move dockerrpm output directory creation to dockerrpm Avoid having to compute the directory in two places in different environments.

packaging: drop "support" for unsupported Fedora versions Fedora 31 has just been released, and Fedora 29 will be EOL in a month. Don't spend any time thinking about dead stuff.

mail: black wants to add this blank line I can't figure out how this got overlooked on previous runs, but here we are. It looks like the culprit change is already public?

hghave: verify we have a black that is new enough for our format We require what is currently the absolute latest black, so let's be paranoid.

contrib: fix up example fix configuration for our move to released black

phabricator: use True primitive instead of b'true' for phabupdate actions Something I'd missed in the creatediff port. This didn't matter before with the old PHP form style wireformat, but breaks with the new arcanist format. Differential Revision: https://phab.mercurial-scm.org/D7152

setup: allow py3 install without env vars 5.2 is the first release of Mercurial where py3 support is expected to be widely used, therefore we should allow installing it without hoop-jumping. Differential Revision: https://phab.mercurial-scm.org/D7151

formatting: drop `grey`, our custom black version Now that the official black has all we want, we can drop this.

formatting: using black to check for formatting

formatting: run black version 19.10b0 on the codebase The latest version of black is out and contains the change we needed. So we can start using it now. note: `test-check-format.t` will complains about this changes because it still use `grey` and need to be migrated to `black`. See next changesets for this.

packaging: fix buildrpm whitespace

packaging: drop outdated buildrpm "tested on" comment Packaging usually works on other versions too, and it is not efficient to maintain the list in repo. It is already out of sync with the Makefile targets.

packaging: also include hgweb.wsgi in rpms

packaging: introduce Python3 support as buildrpm --python3 Just overrule the HGPYTHON3 warning.

packaging: be explicit about Python version in rpm spec Fedora 31 has Python3 at /usr/bin/python ... but expect everybody to not just find python in $PATH but be explicit about whether they want python2 or python3. mercurial.spec just used 'python' and would fail when it unknowingly used Python 3 and ended up with Mercurial setup.py reporting "Python 3.7 detected." and talking about the HGPYTHON3 environment variable. For now, just be explicit about using system python2 as python executable when building rpms.

packaging: make python snippets in rpm building python3 compatible Fedora 31 has Python3 at /usr/bin/python, and buildrpm would fail on snippets that use python2 syntax. Instead of forcing python2, just accept for the future while staying backwards compatible.

py3: fix patchbomb to accept non-ASCII header value for email preview Since mail.headencode() is disabled by -n/--test, non-ASCII header value has to be allowed. Spotted by Denis Laxalde.

tests: check patchbomb with a non-ascii commit message This fails on Python 3 but gets fixed in the next changeset.

formatter: fix handling of None value in templater mapping For historical reasons, None in mapping dict means there's no such keyword, and falls back to b"". That's fine in log templates where mapping item is generally a callable returning a value (which may be None,) but the formatter directly puts an "evaluated" value in the mapping. So the None value has to be lifted to wrappedvalue(None) to avoid confusion in the template engine.

config: add support for defaultvalue of list of printable elements

config: fix -Tjson to not crash due to unsupported defaultvalue types Maybe it isn't great to ignore unsupported types at all, but otherwise "hg config -Tjson" would crash.

tests: handle Message-Id email header possible wrapping The "Message-Id" header will get wrapped with a new line when exceeding 75 characters on Python 3 (see changeset 7d4f2e4899c5 introducing usage of email.header.Header.encode and respective doc). This will occur in an unpredictable manner depending on the hostname's length. To make the test output consistent across Python versions and hostname configuration, we add a filter to unwrap this header value.

py3: leverage pycompat.long

packaging: remove version info from Breaks+Replaces in Debian package The versioned Breaks: and Replaces: cause problem when trying to install our package over the one in Debian. $ sudo apt install ./packages/debian-buster/mercurial_5.2~rc0+15-buster-a2ff3aff81d2_amd64.deb Reading package lists... Done Building dependency tree Reading state information... Done Note, selecting 'mercurial' instead of './packages/debian-buster/mercurial_5.2~rc0+15-buster-a2ff3aff81d2_amd64.deb' Some packages could not be installed. This may mean that you have requested an impossible situation or if you are using the unstable distribution that some required packages have not yet been created or been moved out of Incoming. The following information may help to resolve the situation: The following packages have unmet dependencies: mercurial : Breaks: mercurial-common (< 5.2~rc0+15-buster-a2ff3aff81d2) but 5.2~rc0-1 is to be installed E: Unable to correct problems, you have held broken packages. Removing version information resolves the situation.

py3: fix generated non-ascii message in test-notify.t The resulting "Subject:" header contains an encoded word in Python 3 so we have to add distinct outputs for Python 3 but underlying values are the same: >>> from email.header import decode_header >>> decode_header('=?utf-8?b?w6AuLi4=?=') [(b'\xc3\xa0...', 'utf-8')]

py3: decode encoding literal before passing to .decode() bytes.decode(<encoding>) wants an str as "encoding" parameter, it raises a TypeError if given a bytestring. encoding.encoding and encoding.fallbackencoding are bytes values.

py3: decode payload of notify email This fixes one UnicodeEncodeError in test-notify.t:422 when testing the notify hook with non-ascii content (there are more later). We only decode on Python 3, since it's not safe for sure on Python 2.

py3: decode email headers with mail.headdecode() in notify extension

py3: use stdlib's parseaddr() to get sender header in notify extension In Python 3, email headers are unicode string so using stringutil.email() will not work as it compares with bytestring. So let's use email.utils.parseaddr() from the stdlib which has a consistent behavior across Python versions. The same is done in patchbomb extension already.

py3: use a BytesParser in notify extension This is the first step to make the "long line" case in test-notify.t pass by fixing a UnicodeDecodeError on Python 3. We alias a parsebytes() in mail module, similarly as we already have a parse() function for Python 2 and Python 3 compatibility.

py3: fix headencode() with display=False We previously called str() on a email.header.Header object. On Python 2, this returns a bytestring and the __str__ method is actually an alias to .encode() method. On Python 3, __str__ does not perform encoding (and returns a unicode string). To keep a consistent behavior across Python versions, we explicitly use .encode() and we wrap the result with encoding.strtolocal() to get a bytestring in all cases. As a side effect of forcing bytes conversion, we need to decode back in _addressencode(). This is to make test-notify.t pass on Python 3. Also note that headers are now encoded in some patchbomb tests; this is because the charset is not always "us-ascii" ("iso-8859-1" otherwise) on Python 3.

mail: catch LookupError in headdecode() We already catch this exception in _encode() (called by headencode()). It gets raised when running test-notify.t with Python 3.

py3: account for extra line break in email headers in test-notify.t Long headers appears to be wrapped with new lines. In test-notify.t, we have a "filter.py" that replaces "\n" by " ", so we get an extra space in a Message-Id with a long value.

py3: use as_bytes() method of EmailMessage In Python 3, as_bytes() corresponds to as_string() in Python 2.

py3: use %d instead of %s when formatting an int into a bytestring The latter wasn't noticed before since no tests exercise --confirm at all, let alone on an existing DREV. The former is only hit during an intermittent network issue during amending at the end of a phabsend, so doesn't seem testable. Differential Revision: https://phab.mercurial-scm.org/D7153

packaging: ship only a single binary Debian package We merge the mercurial and mercurial-common binary packages into a single mercurial package. This is essentially to ease installation (and upgrade) using a simple "dpkg -i" command. This also simplifies debian/rules by removing arch (in)dependent cleanups during installation. We have the mercurial binary Breaks: and Replaces: mercurial-common so that the latter will be removed upon upgrade. Also note the change from "override_dh_install" to "override_dh_auto_install" in debian/rules: this is because we do not want "make install" to be run automatically as we need the --install-layout=deb of "setup.py install" (otherwise, files would end up in $DESTDIR/usr/local).