destutil: provide hint on rebase+merge for how to specify destination/rev Without a destination specified, rebase and merge attempt to identify a good candidate; this fails if there's too many heads, the heads have a bookmark, or other reasons. Complicating the issue, users may have specified -t, thinking it means target, and are confused when the error message says that they need to specify an explicit rev or explicit destination - they feel like they already have. Differential Revision: https://phab.mercurial-scm.org/D7024

hghave: document format for version feature checks as <name><vers>, no dots I had a bit of a difficult time when attempting to fix a use of `py-38` when trying to figure out what the correct way of specifying the feature was. By having the strings 'py-3*' and 'py3.8' in the hghave.checkvers docstring, I would have had a much easier time of it instead of having to trace exactly what was happening here. Differential Revision: https://phab.mercurial-scm.org/D7023

check-commit: allow foo_bar naming in functions nameswithallthewordssmashedtogetherarehardtoread. especiallyifenglishisnotyourprimarylanguage. Let's align with the rest of the programming universe and allow_the_use_of_underscores_in_names. We took a hand poll at the 5.2 sprint regarding this change and all but 1 person supported it. The person who didn't expressed concerns around excessive API breakage if we mass renamed things. But we're not planning to mass rename things for the sake of renaming, so all should be well. Differential Revision: https://phab.mercurial-scm.org/D2010

infinitepush: mark extension as likely to be deleted .. bc:: The infinitepush extension is believed to be unused, and will be deleted at the end of 2020 unless users contact mercurial-devel@mercurial-scm.org. Differential Revision: https://phab.mercurial-scm.org/D7022

tests: use range() in generate-churning-module.py This is a test-only script. Performance on Python 2 for creating a full list instead of a generator doesn't matter. With this change, test-check-pyflakes.t passes on Python 3! Differential Revision: https://phab.mercurial-scm.org/D7019

tests: allow warning about file The code is protected by a block that makes it Python 2 only, so use of file is acceptable. Differential Revision: https://phab.mercurial-scm.org/D7018

lsprof: remove __main__ functionality I'm pretty sure nobody uses this. I noticed it because Python 3 linting is complaining about execfile. Differential Revision: https://phab.mercurial-scm.org/D7017

tests: use proper Python 3.8 feature Fix typo introduced in 830eacef67f8. Differential Revision: https://phab.mercurial-scm.org/D7016

py3: finish porting iteritems() to pycompat and remove source transformer This commit finishes porting .iteritems() to pycompat.iteritems() for the mercurial package. The translation of .iteritems() to .items() was the last conversion performed by the source transformer. With the porting to pycompat complete, we no longer have a need for the source transformer. So the source transformer has been removed. Good riddance! The code base is now compatible with Python 2 and Python 3. For the record, as the person who introduced the source transformer, it brings me joy to delete it. It accomplished its goal to facilitate a port to Python 3 without overly burdening people on some painful low-level differences between Python 2 and 3. It is unfortunate we still have to wallpaper over many differences with the pycompat shim. But it is what it is. Differential Revision: https://phab.mercurial-scm.org/D7015

py3: define and use pycompat.iteritems() for hgext/ .iteritems() -> .items() is the last source transform being performed. But it is also the most widely used. This commit adds a pycompat.iteritems symbol and imports it in place of .iteritems() for usage in hgext/. I chose to stop at just hgext/ because the patch will be large and it is an easy boundary to stop at since we can disable source transformation on a per-package basis. There are places where the type does implement items() and we could call items() directly. However, this would require critical thought and I thought it would be easier to just blindly change the code. We know which call sites need to be audited in the future because they have "pycompat.iteritems." With this change, we no longer perform source transformation on hgext! Differential Revision: https://phab.mercurial-scm.org/D7014

py3: define and use pycompat.itervalues() .itervalues() only exists on Python 2. Python 3's equivalent is .values(). But we don't want to blindly use .values() everywhere because on Python 2, it will create a list, which will have performance implications. This commit introduces pycompat.itervalues() which will call the appropriate method on the passed object. We update all callers of obj.itervalues() to pycompat.itervalues(obj) instead. With this commit, the only source tranforming remaining is for iteritems(). Victory is near... Differential Revision: https://phab.mercurial-scm.org/D7013

py3: stop normalizing 2nd argument of *attr() to unicode Now that we don't byteify strings, we can stop normalizing the 2nd string argument to getattr() and remove explicit overrides we were using in the code base. We no longer use some helper functions in the source transformer, so we remove those as well. Differential Revision: https://phab.mercurial-scm.org/D7012

log: populate keywords if specified in custom -Tjson(...) or -Tcbor(...) To make things simple, early return for ui.quiet is disabled if the formatter is templated and provides some datahint().

formatter: map -Tjson(...) and -Tcbor(...) to templater Even though custom JSON output could be generated by using --config template.json="{dict(...)|json}" ..., doing that is tedious because of the trailing comma handling. This patch introduces special syntax for JSON/CBOR formats. -Tjson(...) is translated to template as if function-style template definition were supported: [templates] json(...) = "{dict(...)|json}" json(...):docheader = "[\n " json(...):docfooter = "\n]\n" json(...):separator = ",\n "

formatter: parse name of built-in formatter templates in standard way This slightly makes it easier to add "-Tjson(...)" handling, which should be enabled only if the template specifier doesn't look like a literal template. In other words, it should be handled after "if '{' in tmpl". This makes "log -Tpickle" and "log -Tdebug" abort, which I think is better than just printing "picklepicklepickle...".

formatter: pass in template spec to templateformatter as argument Prepare for the next patch, which will unify handling of the formatter names and the template names.

templater: add public parseexpr() function to parse "-Tjson(...)" Extracted _addparseerrorhint() to show nicer hint on error.

import: read X-Mercurial-Node email header to determine nodeid This would be useful to import and obsolete patches sent using 'hg email --plain', using evolve's --obsolete option of 'hg import'. If email body contains Mercurial patch header ('# HG changeset patch' block), nodeid parsed from X-Mercurial-Node header will still be overridden by respective value found in body.

patchbomb: use procutil.shellquote() instead of shlex to escape email address This basically backs out 2cc453284d5c, and inserts procutil.shellquote() instead. I don't care about Windows compatibility here, but shlex.quote() can't handle byte strings on Python 3.

formatting: introduce a `test-check-format-black.t` that enforce formatting This should prevent use to drift away from the expect format.

formatting: run black on all file again Apparently, since the blackgnarok, we divergence from the expected formatting. Formatted using:: grey.py -S $(hg files 'set:**.py - mercurial/thirdparty/** - "contrib/python-zstandard/**" - hgext/fsmonitor/pywatchman/** - contrib/grey.py') # skip-blame mass-reformatting only # no-check-commit reformats foo_bar functions

formatting: make black --quiet in the example `hg fix` config I do like cake, but I cannot have so many of them.

formatting: remove the data-ogre from the config example Without he final `-`, black ignore the stdin and just looks at file provided as command line argument. Since `hg fix` feeds the file content through stdin and does not pass file argument, this meant black happily exited successful (all files passed as argument were formatted) without any output. Fix picked this "no output" as the new file content, deleting all previous content. I appreciate the fact this effectively removed all buggy code in any files processing that way, but this also ate all my data. The example config is now fixed in that regards.

py3: stop normalizing .encode()/.decode() arguments to unicode Now that we don't byte transform string literals, we no longer need this transform. While we're here, we also drop some superfluous u'' prefix in existing callers. Differential Revision: https://phab.mercurial-scm.org/D7011

py3: manually import pycompat.delattr where it is needed And with this change, we no longer need the auto-inserted import statement in the source transformer, so it has been removed! Differential Revision: https://phab.mercurial-scm.org/D7010

py3: manually import getattr where it is needed The march continues. Differential Revision: https://phab.mercurial-scm.org/D7009

py3: stop injecting pycompat.hasattr into modules I only found a single user of this pattern, probably because we use util.hasattr everywhere. Differential Revision: https://phab.mercurial-scm.org/D7008

py3: manually import pycompat.setattr where it is needed Continuing to eliminate the implicit import of symbols in the Python 3 source transformer so we can eliminate it. Differential Revision: https://phab.mercurial-scm.org/D7007

py3: stop implicitly importing unicode We should be pycompat.unicode everywhere. It turns out we were doing this everywhere except for one place in templatefilters! Differential Revision: https://phab.mercurial-scm.org/D7006

py3: manually import pycompat.open into files that need it We want to eliminate the source transformer. Currently it inserts a `from mercurial.pycompat import ...` at the top of files to alias some builtins. This commit replaces the implicit import of `open` with an explicit import on files that need it and changes the source transformer to no longer import `open`. As part of this, we needed to store an explicit local for `open` in the Python 2 code path in `pycompat` so the import works. (Builtins that are automatically in scope cannot be imported.) Differential Revision: https://phab.mercurial-scm.org/D7005

import-checker: allow symbol imports from mercurial.pycompat Currently, the source transformer inserts `from mercurial.pycompat import delattr, getattr, hasattr, setattr, open, unicode` to the top of every file. As part of getting rid of the source transformer, we'll need to have source code call these wrappers directly. Rather than rewrite all call sites to call pycompat.*, I think it makes sense to import needed symbols via explicit imports. That requires loosening the import checker to allow this. Differential Revision: https://phab.mercurial-scm.org/D7004

contrib: add a fork of black (as "grey") that includes my changes This is black with https://github.com/psf/black/pull/826 applied as of today. The current git hash of black master is d9e71a75ccfefa3d9156a64c03313a0d4ad981e5, and the hash of my commit is dc1add6e94e212eff37bb3619e1422fb3c6d8dc8. In order to use this, you need to install `black` (from github master) and `typed-ast` using pip, preferably into python3, and then you can run `grey.py` with that Python and you'll have my patched version of black, which is how we've been formatting the codebase. Once my PR is merged, I'll follow up by removing this fork and updating instructions in the example config. # no-check-commit bad style Differential Revision: https://phab.mercurial-scm.org/D7002

rust-cpython: change license of ref_sharing.rs to MIT Since we plan to upstream this feature, it's better to continue further refactoring under the same license as rust-cpython crate. According to the file history, copyright holders are: - Raphaël Gomès <rgomes@octobus.net> - Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> - Yuya Nishihara <yuya@tcha.org>

contrib: fix check-code to be able to detect missing _() with bytestrings Differential Revision: https://phab.mercurial-scm.org/D6997

cleanup: mark some ui.(status|note|warn|write) calls as not needing i18n These used to be marked with no-op parens, but black removes those now and this is more explicit. # skip-blame: fallout from mass reformatting Differential Revision: https://phab.mercurial-scm.org/D6996

ui: define (write|status|warn|note)noi18n aliases We currently use `write(('...'))` to suppress check-code warnings about not using translated strings. However, when we run black, it will strip the `((...))`. In order to placate black, we'll need to use a different mechanism to pass untranslatable strings. This commit introduces a `writenoi18n` alias (and friends) to `write` for that purpose. Differential Revision: https://phab.mercurial-scm.org/D6994

mangler: stop rewriting string constants to be bytes literals We've rewritten everything, so we no longer require this step. Differential Revision: https://phab.mercurial-scm.org/D6973

formatting: byteify all mercurial/ and hgext/ string literals Done with python3.7 contrib/byteify-strings.py -i $(hg files 'set:mercurial/**.py - mercurial/thirdparty/** + hgext/**.py - hgext/fsmonitor/pywatchman/** - mercurial/__init__.py') black -l 80 -t py33 -S $(hg files 'set:**.py - mercurial/thirdparty/** - "contrib/python-zstandard/**" - hgext/fsmonitor/pywatchman/**') # skip-blame mass-reformatting only Differential Revision: https://phab.mercurial-scm.org/D6972

formatting: blacken the codebase This is using my patch to black (https://github.com/psf/black/pull/826) so we don't un-wrap collection literals. Done with: hg files 'set:**.py - mercurial/thirdparty/** - "contrib/python-zstandard/**"' | xargs black -S # skip-blame mass-reformatting only # no-check-commit reformats foo_bar functions Differential Revision: https://phab.mercurial-scm.org/D6971

style: run a patched black on a subset of mercurial This applied black to the 20 smallest files in mercurial/: ls -S1 mercurial/*.py | tail -n20 | xargs black --skip-string-normalization Note that a few files failed to format, presumably due to a bug in my patch. The intent is to be able to compare results to D5064 with https://github.com/python/black/pull/826 applied to black. I skipped string normalization on this patch for clarity - in reality I think we'd want one pass without string normalization, followed by another to normalize strings (which is basically replacing ' with " globally.) # skip-blame mass-reformatting only Differential Revision: https://phab.mercurial-scm.org/D6342

tests: conditionalize output for Python 3 It appears that the random.randint() behavior is different between Python 2 and Python 3. So make the test conditional on that. This makes the test pass on Python 3.7 (and presumably other Python 3 versions). Differential Revision: https://phab.mercurial-scm.org/D6964

hg: remove HGUNICODEPEDANTRY and RTUNICODEPEDANTRY This was added in 73e4a02e6d23 and 89822d7a9d5f as a preliminary way to better support Python 3. With the Python 3 port nearly done and better procedures in place, we shouldn't need it. Differential Revision: https://phab.mercurial-scm.org/D6991

convert: use pycompat.fsencode() This avoids a `TypeError: Can't mix strings and bytes in path components`. Differential Revision: https://phab.mercurial-scm.org/D6965

tests: make tarball output conditional on Python version Python 3.8 changed the default tar file archive format: https://docs.python.org/3.8/whatsnew/3.8.html#tarfile. This commit teaches our tests about the new behavior. Differential Revision: https://phab.mercurial-scm.org/D6986

tests: conditionalize test-run-tests.t for Python 3.8 It looks like Python 3.8 changed the pretty XML formatting slightly to strip some whitespace. Let's teach our tests about that. Differential Revision: https://phab.mercurial-scm.org/D6988

hgweb: use importlib.reload() if available reload() was nuked in Python 3. We need to use importlib.reload() instead. But pyflakes isn't smart enough to detect our conditional usage, so we allow this error. Differential Revision: https://phab.mercurial-scm.org/D6992

tests: allow xrange warning from perf.py The use of xrange in this file is acceptable. We need to make the output optional - and not conditional on the Python version - because `pyflakes` could be executed by any Python version and pyflakes behaves differently depending on the Python version. Differential Revision: https://phab.mercurial-scm.org/D6990

debugsidedata: fix verbose mode on python3 We need to be gently with python3 for it to display this binary data. Differential Revision: https://phab.mercurial-scm.org/D6995