formatter: map -Tjson(...) and -Tcbor(...) to templater Even though custom JSON output could be generated by using --config template.json="{dict(...)|json}" ..., doing that is tedious because of the trailing comma handling. This patch introduces special syntax for JSON/CBOR formats. -Tjson(...) is translated to template as if function-style template definition were supported: [templates] json(...) = "{dict(...)|json}" json(...):docheader = "[\n " json(...):docfooter = "\n]\n" json(...):separator = ",\n "

formatter: parse name of built-in formatter templates in standard way This slightly makes it easier to add "-Tjson(...)" handling, which should be enabled only if the template specifier doesn't look like a literal template. In other words, it should be handled after "if '{' in tmpl". This makes "log -Tpickle" and "log -Tdebug" abort, which I think is better than just printing "picklepicklepickle...".

formatter: pass in template spec to templateformatter as argument Prepare for the next patch, which will unify handling of the formatter names and the template names.

templater: add public parseexpr() function to parse "-Tjson(...)" Extracted _addparseerrorhint() to show nicer hint on error.

import: read X-Mercurial-Node email header to determine nodeid This would be useful to import and obsolete patches sent using 'hg email --plain', using evolve's --obsolete option of 'hg import'. If email body contains Mercurial patch header ('# HG changeset patch' block), nodeid parsed from X-Mercurial-Node header will still be overridden by respective value found in body.

patchbomb: use procutil.shellquote() instead of shlex to escape email address This basically backs out 2cc453284d5c, and inserts procutil.shellquote() instead. I don't care about Windows compatibility here, but shlex.quote() can't handle byte strings on Python 3.

formatting: introduce a `test-check-format-black.t` that enforce formatting This should prevent use to drift away from the expect format.

formatting: run black on all file again Apparently, since the blackgnarok, we divergence from the expected formatting. Formatted using:: grey.py -S $(hg files 'set:**.py - mercurial/thirdparty/** - "contrib/python-zstandard/**" - hgext/fsmonitor/pywatchman/** - contrib/grey.py') # skip-blame mass-reformatting only # no-check-commit reformats foo_bar functions

formatting: make black --quiet in the example `hg fix` config I do like cake, but I cannot have so many of them.

formatting: remove the data-ogre from the config example Without he final `-`, black ignore the stdin and just looks at file provided as command line argument. Since `hg fix` feeds the file content through stdin and does not pass file argument, this meant black happily exited successful (all files passed as argument were formatted) without any output. Fix picked this "no output" as the new file content, deleting all previous content. I appreciate the fact this effectively removed all buggy code in any files processing that way, but this also ate all my data. The example config is now fixed in that regards.

py3: stop normalizing .encode()/.decode() arguments to unicode Now that we don't byte transform string literals, we no longer need this transform. While we're here, we also drop some superfluous u'' prefix in existing callers. Differential Revision: https://phab.mercurial-scm.org/D7011

py3: manually import pycompat.delattr where it is needed And with this change, we no longer need the auto-inserted import statement in the source transformer, so it has been removed! Differential Revision: https://phab.mercurial-scm.org/D7010

py3: manually import getattr where it is needed The march continues. Differential Revision: https://phab.mercurial-scm.org/D7009

py3: stop injecting pycompat.hasattr into modules I only found a single user of this pattern, probably because we use util.hasattr everywhere. Differential Revision: https://phab.mercurial-scm.org/D7008

py3: manually import pycompat.setattr where it is needed Continuing to eliminate the implicit import of symbols in the Python 3 source transformer so we can eliminate it. Differential Revision: https://phab.mercurial-scm.org/D7007

py3: stop implicitly importing unicode We should be pycompat.unicode everywhere. It turns out we were doing this everywhere except for one place in templatefilters! Differential Revision: https://phab.mercurial-scm.org/D7006

py3: manually import pycompat.open into files that need it We want to eliminate the source transformer. Currently it inserts a `from mercurial.pycompat import ...` at the top of files to alias some builtins. This commit replaces the implicit import of `open` with an explicit import on files that need it and changes the source transformer to no longer import `open`. As part of this, we needed to store an explicit local for `open` in the Python 2 code path in `pycompat` so the import works. (Builtins that are automatically in scope cannot be imported.) Differential Revision: https://phab.mercurial-scm.org/D7005

import-checker: allow symbol imports from mercurial.pycompat Currently, the source transformer inserts `from mercurial.pycompat import delattr, getattr, hasattr, setattr, open, unicode` to the top of every file. As part of getting rid of the source transformer, we'll need to have source code call these wrappers directly. Rather than rewrite all call sites to call pycompat.*, I think it makes sense to import needed symbols via explicit imports. That requires loosening the import checker to allow this. Differential Revision: https://phab.mercurial-scm.org/D7004

contrib: add a fork of black (as "grey") that includes my changes This is black with https://github.com/psf/black/pull/826 applied as of today. The current git hash of black master is d9e71a75ccfefa3d9156a64c03313a0d4ad981e5, and the hash of my commit is dc1add6e94e212eff37bb3619e1422fb3c6d8dc8. In order to use this, you need to install `black` (from github master) and `typed-ast` using pip, preferably into python3, and then you can run `grey.py` with that Python and you'll have my patched version of black, which is how we've been formatting the codebase. Once my PR is merged, I'll follow up by removing this fork and updating instructions in the example config. # no-check-commit bad style Differential Revision: https://phab.mercurial-scm.org/D7002

rust-cpython: change license of ref_sharing.rs to MIT Since we plan to upstream this feature, it's better to continue further refactoring under the same license as rust-cpython crate. According to the file history, copyright holders are: - Raphaël Gomès <rgomes@octobus.net> - Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> - Yuya Nishihara <yuya@tcha.org>

contrib: fix check-code to be able to detect missing _() with bytestrings Differential Revision: https://phab.mercurial-scm.org/D6997

cleanup: mark some ui.(status|note|warn|write) calls as not needing i18n These used to be marked with no-op parens, but black removes those now and this is more explicit. # skip-blame: fallout from mass reformatting Differential Revision: https://phab.mercurial-scm.org/D6996

ui: define (write|status|warn|note)noi18n aliases We currently use `write(('...'))` to suppress check-code warnings about not using translated strings. However, when we run black, it will strip the `((...))`. In order to placate black, we'll need to use a different mechanism to pass untranslatable strings. This commit introduces a `writenoi18n` alias (and friends) to `write` for that purpose. Differential Revision: https://phab.mercurial-scm.org/D6994

mangler: stop rewriting string constants to be bytes literals We've rewritten everything, so we no longer require this step. Differential Revision: https://phab.mercurial-scm.org/D6973

formatting: byteify all mercurial/ and hgext/ string literals Done with python3.7 contrib/byteify-strings.py -i $(hg files 'set:mercurial/**.py - mercurial/thirdparty/** + hgext/**.py - hgext/fsmonitor/pywatchman/** - mercurial/__init__.py') black -l 80 -t py33 -S $(hg files 'set:**.py - mercurial/thirdparty/** - "contrib/python-zstandard/**" - hgext/fsmonitor/pywatchman/**') # skip-blame mass-reformatting only Differential Revision: https://phab.mercurial-scm.org/D6972

formatting: blacken the codebase This is using my patch to black (https://github.com/psf/black/pull/826) so we don't un-wrap collection literals. Done with: hg files 'set:**.py - mercurial/thirdparty/** - "contrib/python-zstandard/**"' | xargs black -S # skip-blame mass-reformatting only # no-check-commit reformats foo_bar functions Differential Revision: https://phab.mercurial-scm.org/D6971

style: run a patched black on a subset of mercurial This applied black to the 20 smallest files in mercurial/: ls -S1 mercurial/*.py | tail -n20 | xargs black --skip-string-normalization Note that a few files failed to format, presumably due to a bug in my patch. The intent is to be able to compare results to D5064 with https://github.com/python/black/pull/826 applied to black. I skipped string normalization on this patch for clarity - in reality I think we'd want one pass without string normalization, followed by another to normalize strings (which is basically replacing ' with " globally.) # skip-blame mass-reformatting only Differential Revision: https://phab.mercurial-scm.org/D6342

tests: conditionalize output for Python 3 It appears that the random.randint() behavior is different between Python 2 and Python 3. So make the test conditional on that. This makes the test pass on Python 3.7 (and presumably other Python 3 versions). Differential Revision: https://phab.mercurial-scm.org/D6964

hg: remove HGUNICODEPEDANTRY and RTUNICODEPEDANTRY This was added in 73e4a02e6d23 and 89822d7a9d5f as a preliminary way to better support Python 3. With the Python 3 port nearly done and better procedures in place, we shouldn't need it. Differential Revision: https://phab.mercurial-scm.org/D6991

convert: use pycompat.fsencode() This avoids a `TypeError: Can't mix strings and bytes in path components`. Differential Revision: https://phab.mercurial-scm.org/D6965