remotefilelog: remove support for fetching packs We've never used this at Google, and it's strictly an optimization that indygreg and I are confident we can handle more elegantly later. Let's rip it out to keep the surface area of the imported remotefilelog small. Some tests have changes because the tests for regular packs and repacking were mixed with tests of fetching packs. I'm pretty sure I've done the right thing here. Differential Revision: https://phab.mercurial-scm.org/D5122

remotefilelog: jettison example cache client We can always reconstruct it from history if it becomes interesting. Differential Revision: https://phab.mercurial-scm.org/D4941

remotefilelog: prune obsolete method Differential Revision: https://phab.mercurial-scm.org/D4940

logtoprocess: use new runbgcommand from procutil Differential Revision: https://phab.mercurial-scm.org/D4939

procutil: port over windows encoding fixes from logtoprocess Differential Revision: https://phab.mercurial-scm.org/D5121

remotefilelog: transplant runbgcommand to procutil While cleaning up the deprecated runshellcommand I noticed a near-clone of this in logtoprocess, so I'm standardizing on what appears to be the newer one by moving it to procutil. Differential Revision: https://phab.mercurial-scm.org/D4938

remotefilelog: remove function that was described as deprecated Differential Revision: https://phab.mercurial-scm.org/D4937

remotefilelog: import pruned-down remotefilelog extension from hg-experimental This is remotefilelog as of my recent patches for compatibility with current tip of hg, minus support for old versions of Mercurial and some FB-specific features like their treemanifest extension and fetching linkrev data from a patched phabricator. The file extutil.py moved from hgext3rd to remotefilelog. This is not yet ready to be landed, consider it a preview for now. Planned changes include: * replace lz4 with zstd * rename some capabilities, requirements and wireproto commands to mark them as experimental * consolidate bits of shallowutil with related functions (eg readfile) I'm certainly open to other (small) changes, but my rough mission is to land this largely as-is so we can use it as a model of the functionality we need going forward for lazy-fetching of file contents from a server. # no-check-commit because of a few foo_bar functions Differential Revision: https://phab.mercurial-scm.org/D4782

store: introduce _matchtrackedpath() and use it to filter store files This patch introduces a function to filter store files on the basis of the path which they are tracking. The function assumes that the entries can be of two types, 'meta/*' and 'data/*' which means it will just work on revlog based storage and not with another storage ways. For the 'data/*' entries, we remove the 'data/' part and '.i/.d' part from the beginning and the end then pass that to matcher. For the 'meta/*' entries, we remove the 'meta/' and '/00manifest.(i/d)' part from beginning and end then call matcher.visitdir() with it to make sure all the parent directories are also downloaded. Since the storage filtering for narrow stream clones is implemented with this patch, we remove the un-implemented error message, add some more tests and add the treemanifest case to tests too. The tests demonstrate that it works correctly. After this patch, we have now narrow stream clones working. Narrow stream clones are a very important feature for large repositories who have good internet connection because they use streamclones for cloning and if they do normal narrow clone, that takes more time then a full streamclone. Also narrow-stream clone will drastically speed up clone timings. Differential Revision: https://phab.mercurial-scm.org/D5139

narrow: only send includepats and excludepats if they are not empty If we send an empty includepats or excludepats argument to getbundle, it's translated to `['']` on the server which causes problems because even though it's empty, bool of that value if True and we end up creating differencematcher with narrowspec.match() which results in unexpected behavior. Differential Revision: https://phab.mercurial-scm.org/D5138

exchange: pass includepats and excludepats as arguments to getbundle() This will help in implementing narrow stream clones. Also narrow extension used to add these arguments, now we add them by default if they are not empty. Since reading includepats and excludepats on the server only works when narrow is enabled, we check if narrow if enabled or not before passing them. Differential Revision: https://phab.mercurial-scm.org/D5119

catapult: add a bit more documentation on how to use catapult tracing Differential Revision: https://phab.mercurial-scm.org/D5217

catapult: introduce HGTESTCATAPULTSERVERPIPE to control run-tests' tracing If unset, it will inherit its value from HGCATAPULTSERVERPIPE (which hg itself also respects). By setting only HGTESTCATAPULTSERVERPIPE, we can get per-command breakdowns of the test runtimes for the whole test suite without overloading the trace file with the contents of the tracing from hg (such as demandimport, etc.) Differential Revision: https://phab.mercurial-scm.org/D5216

catapult: rename 'active' to 'activetrace'; this isn't storing a boolean state (This is an attempt to make it more obvious that the indentation is correct, to prevent a repeat of 97f52862b1bd) Differential Revision: https://phab.mercurial-scm.org/D5215

catapult: fix broken run-tests catapult tracing Commit 97f52862b1bd thought this block was misindented, but it was not; the indentation after that commit means we never log a catapult trace for individual commands. Also, we've never emitted the "end" trace line when the test was finished, so the trace viewer claims that these "did not finish", fixing that as well. Differential Revision: https://phab.mercurial-scm.org/D5214

test: fix self._testdir to use the right mercurial library during testing Currently if you run tests from some other directory other than the `../tests/`, you will get a warning stating ``` warning: Testing with unexpected mercurial lib: mercurial (expected /tmp/hgtests.xxxxxx/install/lib/python/mercurial) ``` This is because the current directory being added to the 'PATH', if the `self._testdir != runtestdir`, owing to this line ``` if self._testdir != runtestdir: path = [self._testdir] + path ``` Also say you ran the tests from the hg base directory, because directory is being added in the PATH (see the above snippet, at that stage the `self._testdir` has the value as `cwd`, owing to a faulty initialization). And since the current directory already has the 'hg', that is used in place of the hg that is installed for the testing purposes in `/tmp/hgtests.xxxxxx/...`. Differential Revision: https://phab.mercurial-scm.org/D5199

tests: conditonalize a difference in test-merge-tools.t on Windows

statprof: clean up unicode/bytes a little I'm not really sure how this worked before, but something perturbed it and what I've got in this change I believe is a little tidier. This fixes test-profile.t on Python 3. Differential Revision: https://phab.mercurial-scm.org/D5210

statprof: add a couple of asserts to avoid storing unicodes I don't feel strongly about this change, so it'd be fine to drop it during review, but it was _extremely_ helpful in drafting the next change. Differential Revision: https://phab.mercurial-scm.org/D5209

tests: add some helpful `|| cat` bits to test-profile.t This way if the hg invocation crashes, you get to see a stacktrace without having to edit the test. Differential Revision: https://phab.mercurial-scm.org/D5208

tests: skip wireproto clientreactor tests on Python 3.6.0-3.6.3 inclusive See comment next to the if statement for sad details. Differential Revision: https://phab.mercurial-scm.org/D5207

contrib: tweak import-checker to always use bytes for module names We could go the other route and make embedded() return this as a bytes, but then it would be harder to portably doctest, so I prefer this option. Differential Revision: https://phab.mercurial-scm.org/D5206

contrib: fix import-checker to not b'' module names on Python 3 Caught by the doctests. Differential Revision: https://phab.mercurial-scm.org/D5205

tests: add a critical flush() to run-tests.py to make output stable on py3 Differential Revision: https://phab.mercurial-scm.org/D5204

merge-tools: when calling external merge tool, describe the resolve inputs It is a common complaint that a user will be running some operation (histedit, rebase, evolve, etc.), get into a merge-conflict situation, and not understand what they are seeing - it is possible that the merge tool is configured to display the hash, but it's difficult for most merge tools to display a good snippet of the description. In the worst case, configuring this template will lead to output that is immediately covered by a terminal application, maybe the user can hit ctrl-z to see it. In the common case, the output will be in a terminal window and a GUI program will start, and it should be possible to view both the terminal and the GUI program at the same time. Differential Revision: https://phab.mercurial-scm.org/D5094

templatekw: deprecate p1rev/p2rev/p1node/p2node in favor of p1/p2

templatekw: add p1/p2 keywords which switches the current ctx This is just an example of mappingdict. I have no idea what should be displayed as {p1|json}. Currently it is an empty dict since {'ctx'} isn't displayable, which is clearly useless and should be changed later.

templater: add wrapper for a single template mapping This can be used to nest template mappings without inserting a sequence-like layer. See the next patch for example.