fastannotate: use stringutil.pprint() to dump diffopts to be hashed

test-check-py3-compat: update "python3 check-py3-compat.py" output

test-check-py3-compat: exclude thirdparty and cffi which are known to fail Otherwise the following errors would be spilled out: mercurial/cffi/bdiff.py: error importing: <ImportError> cannot import name '_bdiff' (error at bdiff.py:*) mercurial/cffi/bdiffbuild.py: error importing: <ModuleNotFoundError> No module named 'cffi' (error at bdiffbuild.py:*) mercurial/cffi/mpatch.py: error importing: <ImportError> cannot import name '_mpatch' (error at mpatch.py:*) mercurial/cffi/mpatchbuild.py: error importing: <ModuleNotFoundError> No module named 'cffi' (error at mpatchbuild.py:*) mercurial/cffi/osutilbuild.py: error importing: <ModuleNotFoundError> No module named 'cffi' (error at osutilbuild.py:*) mercurial/thirdparty/concurrent/futures/_base.py: invalid syntax: invalid syntax (<unknown>, line *) mercurial/thirdparty/concurrent/futures/process.py: error importing: <SyntaxError> invalid syntax (_base.py, line 416) (error at __init__.py:*) mercurial/thirdparty/concurrent/futures/thread.py: error importing: <SyntaxError> invalid syntax (_base.py, line 416) (error at __init__.py:*) mercurial/thirdparty/zope/interface/_flatten.py: error importing: <ImportError> cannot import name 'Declaration' (error at _flatten.py:*) Some of them can be suppressed by building cffi modules for example, but I don't think it's worth keeping these modules covered by the compatibility checker.

test-check-py3-compat: remove false output from "python3 check-py3-compat.py" If python3 were python2, these errors would be reported, but we're running check-py3-compat.py on Python 3 here.

test: add a test file that displays ssh behavior in front of various errors Prior to changeset ac0a87160012, these errors were suboptimal since they claim the repository is missing while the actual issue was about permission. This extra test file provides additional documented coverage for the various corner cases.

tests: update test-casefolding.t with recent changes This is caught by the mac-buildbot. This was not found initially because this test requires a case insensitive filesystem. I have edited the test file by hand and I suggest to test this on such a system before pushing. Differential Revision: https://phab.mercurial-scm.org/D4430

py3: don't use dict.iterkeys() in hgext/fastannotate/context.py dict.iterkeys() is not present on Python 3. Differential Revision: https://phab.mercurial-scm.org/D4431

run-tests: replace '/dev/null' with os.devnull for Windows Many of the *.py tests were dying on: File "c:\Users\Matt\projects\hg\hgdemandimport\tracing.py", line 27, in log _pipe = open(os.environ['HGCATAPULTSERVERPIPE'], 'w', 1) IOError: [Errno 2] $ENOENT$: '/dev/null'

stringutil: teach pprint() to indent This will make data structure dumping in various places a bit easier to read and diff. Since I wanted this for `hg debugwireproto` output, I added indentation to it. A more advanced pretty printer implementation would conditionally add newlines if output is too long. But it is vastly simpler to be consistent and always add newlines when indenting. Again, I'm not crazy about the verbosity of the code and there is room to consolidate logic for "print a collection." But this isn't the most complicated code in the world and I'm not convinced it is worth doing. Differential Revision: https://phab.mercurial-scm.org/D4399

zsh_completion: complete merge tools for -t/--tool Differential Revision: https://phab.mercurial-scm.org/D4362

shelve: add an "internal" extra Keeping shelve changeset around increase the risk of collision with normal changesets. To prevent such collision and help with overall clarity, we add an 'internal' key in extra that mark the changeset as created by "shelve". Node changes in tests are expected.

shelve: write metadata file on the fly if they are missing Keeping an explicit reference to the shelve node in order to reuse it directly if the unshelved is repeated (eg: unshelve --keep).

shelve: look for shelved node in the repository before unbundling This prepares the version of shelve that would not strip the shelved node from the repository. If we have the node information, search for it in the repository and only fallback on unbundling if it is missing. To be able to find such nodes, we operate on an unfiltered repository.

shelve: handle shelved node on unfiltered repository To build a version of shelve that use hiding instead of stripping, we need shelve to be able to find these revisions. This it does not hurt the bundle version, so we introduce the necessary code in the relevant place.

shelve: store shelved node in a new data file It is useful for a version of shelve not based on bundle and strip. Having more data does not hurt the bundle based case so we introduce it in all cases before doing more rework. We also keep storing the patch using another method. note: We could have this metadata file list all the file involved in the shelve and use that for cleanup. This would be more future proof than having a hard-coded list of file to purge. However, this is an adventure for another series.

templatekw: add {path} keyword to host documentation It's hidden since I'm not sure if filectx templates can be a thing. The plan is to add {status}, {size}, etc., which are usable within {files % ...} context.

status: rename {copy} to {source} for compatibility with {file_copies} (BC) .. bc:: ``{copy}`` in status command template is renamed to ``{source}``.

formatter: rename {abspath}/{file} to {path}, and drop relative {path} (BC) Note that {path} in status is either relative-to-cwd or repository-absolute depending on the command argument and config knob, which can't be reproduced by using the {path|relpath} filter. The default template is updated to always use a relative path. .. bc:: ``{abspath}`` and ``{file}`` in generic templates are renamed to ``{path}``. Any ``{path}`` is a repository-absolute path. Use ``{path|relpath}`` to convert it to a filesystem path.

templatekw: alias {name} of file copies dict to {path} For the same reason as the previous patch. We might want some hack to support {source.path}, {source.rev}, etc., but that's a different issue.

templatekw: alias {file} of files list to {path} This is a part of the name unification. All {path}s will be changed to repository-absolute (i.e. canonical) paths. https://www.mercurial-scm.org/wiki/GenericTemplatingPlan#Dictionary

copies: improve logic of deciding copytracing on based of config options Few months ago or maybe a year ago, I imported Fb's heuristics based copytracing algorithms. While importing that, I renamed `experimental.disablecopytrace` with `experimental.copytrace` and the behavior of the new config option was like this: * "heuristics" : Fb's heuristic copytracing algorithm * "off" : copytracing is turned off * something else: copytracing is on This is the behavior right now also and this is bad because it hardcodes the string 'off' to turn off the copytracing. On big repositories, copytracing is very slow and people wants to turn copytracing off. However if the user sets it to 'False', 'Off', '0', none of them is going to disbale copytracing while they should. I lacked the understanding of why this can be bad when I coded it. After this patch, the new behavior of the config option will be: * "heuristics": Fb's heuristic copytracing algorithm * '0', 'false', 'off', 'never', 'no', 'NO', all the values which repo.ui.configbool() evaluates to False: copytracing in turned off * something else: copytracing is on Since 'off' still evaluates to copytracing being turned off, this is not BC. Also the config option is experimental. Differential Revision: https://phab.mercurial-scm.org/D4416

relnotes: enable extension when running releasenotes command Last night, when I was working on different machine, the test worked, now on a different machine it does not. Looks like we should make sure releasenotes extension is loaded. Differential Revision: https://phab.mercurial-scm.org/D4425

narrow: rename getbundlechangegrouppart_nonellipsis function The function is renamed to getbundlechangegrouppart_widen as per suggestion in D4383. The motivation is such that we can evolve the function into a handler for new wire-protocol command. Differential Revision: https://phab.mercurial-scm.org/D4424

tests: rename test-narrow-widen-non-ellipsis to match current names The test is renamed to test-narrow-widen-no-ellipsis (note non -> no), to match with test-narrow-clone-no-ellipsis.t. Follows up on D4382. Differential Revision: https://phab.mercurial-scm.org/D4423

contrib: byteify perf.py file This is done using contrib/byteify-strings.py. There is just one extra change done to make sure lines don't get longer than 80 chars. # skip-blame because just b'' prefixes. Differential Revision: https://phab.mercurial-scm.org/D4422

tests: use {rev} instead of {node|short} and (glob) in narrow tests I think we used {node|short} because they were first written for only flat manifests and we didn't think to change the format when we started testing tree manifests. Differential Revision: https://phab.mercurial-scm.org/D4421

shortest: fix an off-by-1 when disambiguating against revnums Revision "5" is an unambiguous hex prefix in a repo with 5 revisions; it's only "4" and lower that are ambiguous. Differential Revision: https://phab.mercurial-scm.org/D4420

filemerge: fix an i18n comment typo

tests: add flat manifest case in test-narrow-widen-non-ellipsis.t We had this test with tree manifest only because the tests were broken and we were fixing them and maintaing two different cases of broken tests was bit hard. Now that things work fine, let's add the flat manifest case. The test shows that things work fine in both cases. Differential Revision: https://phab.mercurial-scm.org/D4385

narrow: update the narrowspecs to new ones after pulling when widening The ellipsis case updates the new narrowspecs at bundle2 handling code, and non-ellipsis widening didn't use to work and never set the new narrowpats. Previous pacth added server side logic for widening without ellipsis. This patch adds the missing repo.setnarrowpats() so that we set the new narrowpats after widening in non-ellipsis cases also. This makes widening with non-ellipsis case work but there are certain things which we can improve. Differential Revision: https://phab.mercurial-scm.org/D4384

narrow: add server logic to send cg while widening without ellipsis Before this patch, if you try to widen a narrow clone without ellipsis enabled, it will be broken and the exchange.pull() done by tracked command to widen the clone will be no-op because no custom logic exists for this and server sees that we have all csets and it says `no changes found`. The widening with ellipsis send KILL for existing changegroups and send new changegroups because of the change in ellipsis hash, but we can prevent that in non-ellipsis cases. This patch adds server side logic to send the changegroups for the changesets which are on the client again with filelogs and manifests for the new includes. This is a very starting implementation and we send changegroups and manifests too while we can prevent them. Following things can definitely be improved in the logic this patch adds: 1) Send just the filelogs and treemanifests 2) Send the filelogs only for the additions in the include I tried 1) here but the code is coupled tightly and the way I was able to do that was hacking into the changegroup generation code in a very dirty way, like adding conditionals and preventing the yield. This patch also adds a 'widen' kwarg to prevent other commands except widening to go through that codepath. The test changes demonstrate that the new implementation is correct and fixes things. Differential Revision: https://phab.mercurial-scm.org/D4383

tests: add test to show widening is broken without ellipsis This patch adds a test to show that widening a narrow clone is broken if ellipsis is disabled. I don't think I can add cases to existing test-narrow-widen.t and check-in a failing version of that. So I created a copy. Once the test is fixed, we can merge this new test file back into the original one using testcases. Also, this is just testing treemanifest case because having two cases and both or are failing with different outputs is a bit hard to manage. This is important because upcoming patches will try to fix the broken part. Differential Revision: https://phab.mercurial-scm.org/D4382