crecord: drop duplicated set of firstlineofpadtoprint attribute The attribute is already set a couple of lines above, in curseschunkselector.__init__().

patchbomb: use mail.Generator alias for py2/py3 compat

py3: use email.generator.BytesGenerator in patch.split() This fixes test-import.t on python3. We add Generator alias in mail module to handle python2/python3 compatibility.

py3: only flush before prompting during interactive patch filtering Follows up on c9093ae8d6c4. It's enough to flush just before each prompt.

py3: add a missing b'' prefix in test extension for chg Differential Revision: https://phab.mercurial-scm.org/D7038

fuzz: new target to fuzz jsonescapeu8fast This code just feels complicated enough we should go ahead and give it a dedicated fuzzer: we've found bugs in similar things before. Differential Revision: https://phab.mercurial-scm.org/D7034

fuzz: new fuzzer for fncache-related functions Not all of these are strictly fncache-related, but they all have th same signature and similar-enough behavior that we may as well fuzz them together. No obvious bugs for once, but these felt like they were just complicated enough to cover. Differential Revision: https://phab.mercurial-scm.org/D7033

fuzz: exercise a little more revlog code Differential Revision: https://phab.mercurial-scm.org/D7032

fuzz: new fuzzer for dirs.c This found a six-year-old bug immediately, and then I put it through a few CPU-days of time before sending it. Differential Revision: https://phab.mercurial-scm.org/D7031

dirs: fix trivial over-read of input data This code, introduced in 8c0a7eeda06d, was intentionally over-reading an input string to avoid getting a shared string object for a one-byte input. Unfortunately with an empty input (like in the case of a fuzzer getting started) this was a trivial over-read and triggered an AddressSanitizer failure. I went out of my way to make sure the code still does the copy-avoidance tricks. I don't think this change will cost us much performance since the one-character strings should be cached aggressively anyway. Differential Revision: https://phab.mercurial-scm.org/D7030

sidedatacopies: deal with upgrading and downgrading to that format This is quite useful to test this on real life data. Differential Revision: https://phab.mercurial-scm.org/D6955

sidedatacopies: move various copies related function to the copies modules We will need to access these logic form the copies module. So we move them from their higher level module to the lower level `copies` module. We cannot use them from their top level module as it would create cycles. Differential Revision: https://phab.mercurial-scm.org/D6954

sidedatacopies: read rename information from sidedata Repository using the new format now use changeset centric algorithm and read the copies information from the changelog sidedata. Differential Revision: https://phab.mercurial-scm.org/D6953

changelog: make copies related function return None or a valid value With the previous code, existing but empty value were not "decoded", leading to the method returning one of `None`, some valid value (`list` or `dict`) or `b''`. On a general basis, not explicitly checking for None is a source of bugs. Having a clean return types will help the side-data copies code in future changesets. Differential Revision: https://phab.mercurial-scm.org/D7037

test: fix zstd related output in pure tests Differential Revision: https://phab.mercurial-scm.org/D7036

sidedatacopies: get and store sidedata in the changelogrevision object The object provide a simple way to access changelog entry, we need it to also bear the sidedata value. Since the sidedata are retrieved at the same time as the revision, we can do that without extra cost. Differential Revision: https://phab.mercurial-scm.org/D6951

sidedatacopies: write copies information in sidedata when applicable If the format of the repository indicate it stores copies information into changeset's sidedata, then we actually write that information into sidedata at commit time. It will be put to use in later changesets. Currently, we store all field unconditionally, but that is likely to change in the future for the sake of efficiency. Differential Revision: https://phab.mercurial-scm.org/D6950

sidedatacopies: introduce a sidedata testcase for test-copies-in-changeset.t For now it is equivalent to the filelog case, but introducing this early helps make the comings changesets clearer. Differential Revision: https://phab.mercurial-scm.org/D6949

sidedatacopies: introduce a sidedata testcase for test-copies-unrelated.t For now it is equivalent to the filelog case, but introducing this early helps make the comings changesets clearer. Differential Revision: https://phab.mercurial-scm.org/D6948

sidedatacopies: introduce a sidedata testcase for test-copies.t For now it is equivalent to the filelog case, but introducing this early helps make the comings changesets clearer. Differential Revision: https://phab.mercurial-scm.org/D6947

sidedatacopies: teach upgrade about the new requirement The `debugformat` and `debugupgraderepo` command now detect the requirement. (upgrade to and from are not currently possible). Differential Revision: https://phab.mercurial-scm.org/D6946

sidedatacopies: add a new requirement for storing copies into sidedata The end goal is to have changesets centric sidedata information stored into changelog sidedata. This make it possible to use the changeset based copy tracing algorithm on any repository without affecting hashes. The actual implementation is coming. The feature is marked as experimental (do not use in production) until we stabilise details about the format. Differential Revision: https://phab.mercurial-scm.org/D6945

debugsidedata: small doc improvement The command has a mode to actually show the sidedata, but it wasn't documented. Differential Revision: https://phab.mercurial-scm.org/D6944

upgrade: allow for `sidedata` removal If the side-date feature is removed, we need to drop them. Differential Revision: https://phab.mercurial-scm.org/D6943

upgrade: allow upgrade to repository using sidedata Repository can now be migrated to support sidedata. More requirements and migration will be needed to actual side-data usage. This is a step in that direction. To test the feature, we leverage the test extension. It make sure the `update` part of the side-data migration actually works. Differential Revision: https://phab.mercurial-scm.org/D6942

revlog: add a way to control sidedata changes during revlog.clone We introduce a new argument to pass a callable dealing with the side data logic. Since the side data are usually associated to higher level logic, the revlog code itself is unlikely to know that to do itself. As a result the higher level upgrade code will be responsible to decide what needs to changes. The lower level revlog.clone just get simple instructions to apply. Differential Revision: https://phab.mercurial-scm.org/D6941

sidedata: use only changegroup3 if sidedata is in use This is necessary to exchange the flags over the wire (and within bundles). Differential Revision: https://phab.mercurial-scm.org/D6940

sidedata: apply basic but tight security around exchange We don't currently have code to deal with exchange between repository using sidedata and repository not using sidedata. Until we implement such code (eg: dropping side data when pushing to a non-sidedata repo) we prevent the two kind of repo to speak to each other. This is somewhere similar to what 'treemanifest' does. Note that sidedata exchange is broken unless one use changegroup v3 anyway. See next changeset for details. Differential Revision: https://phab.mercurial-scm.org/D6939

perf: use `setup` function in `perfdirstatewrite` The command seems to pre-date the introduction of the `setup` support in timer. We move the line that is obviously about benchmark setup in such `setup` function.

perf: document `perfdirstatewrite`

perf: use `setup` function in `perfdirfoldmap` The command seems to pre-date the introduction of the `setup` support in timer. We move the line that is obviously about benchmark setup in such `setup` function.

perf: document `perfdirfoldmap`

perf: use `setup` function in `perfdirstatefoldmap` The command seems to pre-date the introduction of the `setup` support in timer. We move the line that is obviously about benchmark setup in such `setup` function.

perf: document `perfdirstatefoldmap`

perf: use `setup` function in `perfdirstatedirs` The command seems to pre-date the introduction of the `setup` support in timer. We move the line that is obviously about benchmark setup in such `setup` function.

perf: document `perfdirstatedirs`

perf: use `setup` function in `perfdirstate` The command seems to pre-date the introduction of the `setup` support in timer. We move the line that is obviously about benchmark setup in such `setup` function.

perf: document `perfdirstate`

perf: document `perfstatus`

py3: flush ui after each message in interactive patch filtering Otherwise, actions from ui.write() are buffered and displayed at end of interactive session.

py3: decode prompt string before calling rawinput Calling input() (aka pycompat.rawinput() on python3) with a byte string displays a byte string in the console. E.g. in interactive commit, we get: b"examine changes to '<file>'?\n(enter ? for help) [Ynesfdaq?]". Similarly, "hg email" prompts are messed up. We thus decode the prompt string before running rawinput().

cleanup: join string literals that are already on one line Thanks to Kyle for noticing this and for providing the regular expression to run on the codebase. This patch has been reviewed by the test suite and they approved of it. # skip-blame: fallout from mass reformatting Differential Revision: https://phab.mercurial-scm.org/D7028

debugcommands: add a few more writenoi18n() Differential Revision: https://phab.mercurial-scm.org/D7027

py3: delete b'' prefix from safehasattr arguments Differential Revision: https://phab.mercurial-scm.org/D7029

destutil: provide hint on rebase+merge for how to specify destination/rev Without a destination specified, rebase and merge attempt to identify a good candidate; this fails if there's too many heads, the heads have a bookmark, or other reasons. Complicating the issue, users may have specified -t, thinking it means target, and are confused when the error message says that they need to specify an explicit rev or explicit destination - they feel like they already have. Differential Revision: https://phab.mercurial-scm.org/D7024

hghave: document format for version feature checks as <name><vers>, no dots I had a bit of a difficult time when attempting to fix a use of `py-38` when trying to figure out what the correct way of specifying the feature was. By having the strings 'py-3*' and 'py3.8' in the hghave.checkvers docstring, I would have had a much easier time of it instead of having to trace exactly what was happening here. Differential Revision: https://phab.mercurial-scm.org/D7023

check-commit: allow foo_bar naming in functions nameswithallthewordssmashedtogetherarehardtoread. especiallyifenglishisnotyourprimarylanguage. Let's align with the rest of the programming universe and allow_the_use_of_underscores_in_names. We took a hand poll at the 5.2 sprint regarding this change and all but 1 person supported it. The person who didn't expressed concerns around excessive API breakage if we mass renamed things. But we're not planning to mass rename things for the sake of renaming, so all should be well. Differential Revision: https://phab.mercurial-scm.org/D2010

infinitepush: mark extension as likely to be deleted .. bc:: The infinitepush extension is believed to be unused, and will be deleted at the end of 2020 unless users contact mercurial-devel@mercurial-scm.org. Differential Revision: https://phab.mercurial-scm.org/D7022

tests: use range() in generate-churning-module.py This is a test-only script. Performance on Python 2 for creating a full list instead of a generator doesn't matter. With this change, test-check-pyflakes.t passes on Python 3! Differential Revision: https://phab.mercurial-scm.org/D7019

tests: allow warning about file The code is protected by a block that makes it Python 2 only, so use of file is acceptable. Differential Revision: https://phab.mercurial-scm.org/D7018

lsprof: remove __main__ functionality I'm pretty sure nobody uses this. I noticed it because Python 3 linting is complaining about execfile. Differential Revision: https://phab.mercurial-scm.org/D7017

tests: use proper Python 3.8 feature Fix typo introduced in 830eacef67f8. Differential Revision: https://phab.mercurial-scm.org/D7016

py3: finish porting iteritems() to pycompat and remove source transformer This commit finishes porting .iteritems() to pycompat.iteritems() for the mercurial package. The translation of .iteritems() to .items() was the last conversion performed by the source transformer. With the porting to pycompat complete, we no longer have a need for the source transformer. So the source transformer has been removed. Good riddance! The code base is now compatible with Python 2 and Python 3. For the record, as the person who introduced the source transformer, it brings me joy to delete it. It accomplished its goal to facilitate a port to Python 3 without overly burdening people on some painful low-level differences between Python 2 and 3. It is unfortunate we still have to wallpaper over many differences with the pycompat shim. But it is what it is. Differential Revision: https://phab.mercurial-scm.org/D7015

py3: define and use pycompat.iteritems() for hgext/ .iteritems() -> .items() is the last source transform being performed. But it is also the most widely used. This commit adds a pycompat.iteritems symbol and imports it in place of .iteritems() for usage in hgext/. I chose to stop at just hgext/ because the patch will be large and it is an easy boundary to stop at since we can disable source transformation on a per-package basis. There are places where the type does implement items() and we could call items() directly. However, this would require critical thought and I thought it would be easier to just blindly change the code. We know which call sites need to be audited in the future because they have "pycompat.iteritems." With this change, we no longer perform source transformation on hgext! Differential Revision: https://phab.mercurial-scm.org/D7014

py3: define and use pycompat.itervalues() .itervalues() only exists on Python 2. Python 3's equivalent is .values(). But we don't want to blindly use .values() everywhere because on Python 2, it will create a list, which will have performance implications. This commit introduces pycompat.itervalues() which will call the appropriate method on the passed object. We update all callers of obj.itervalues() to pycompat.itervalues(obj) instead. With this commit, the only source tranforming remaining is for iteritems(). Victory is near... Differential Revision: https://phab.mercurial-scm.org/D7013

py3: stop normalizing 2nd argument of *attr() to unicode Now that we don't byteify strings, we can stop normalizing the 2nd string argument to getattr() and remove explicit overrides we were using in the code base. We no longer use some helper functions in the source transformer, so we remove those as well. Differential Revision: https://phab.mercurial-scm.org/D7012

log: populate keywords if specified in custom -Tjson(...) or -Tcbor(...) To make things simple, early return for ui.quiet is disabled if the formatter is templated and provides some datahint().

formatter: map -Tjson(...) and -Tcbor(...) to templater Even though custom JSON output could be generated by using --config template.json="{dict(...)|json}" ..., doing that is tedious because of the trailing comma handling. This patch introduces special syntax for JSON/CBOR formats. -Tjson(...) is translated to template as if function-style template definition were supported: [templates] json(...) = "{dict(...)|json}" json(...):docheader = "[\n " json(...):docfooter = "\n]\n" json(...):separator = ",\n "

formatter: parse name of built-in formatter templates in standard way This slightly makes it easier to add "-Tjson(...)" handling, which should be enabled only if the template specifier doesn't look like a literal template. In other words, it should be handled after "if '{' in tmpl". This makes "log -Tpickle" and "log -Tdebug" abort, which I think is better than just printing "picklepicklepickle...".

formatter: pass in template spec to templateformatter as argument Prepare for the next patch, which will unify handling of the formatter names and the template names.