match: add visitchildrenset complement to visitdir `visitdir(d)` lets a caller query whether the directory is part of the matcher. It can receive a response of 'all' (yes, and all children, you can stop calling visitdir now), False (no, and no children either), or True (yes, either something in this directory or a child is part of the matcher). `visitchildrenset(d)` augments that by instead of returning True, it returns a list of items to actually investigate. With this, code can be modified from: for f in self.all_items: if match.visitdir(self.dir + '/' + f): <do stuff> to be: for f in self.all_items.intersect(match.visitchildrenset(self.dir)): <do stuff> use of this function can provide significant performance improvements, especially when using narrow (so that the matcher is much smaller than the stuff we see on disk) and/or treemanifests (so that we can avoid loading manifests for trees that aren't part of the matcher). Differential Revision: https://phab.mercurial-scm.org/D4130

includematcher: separate "parents" from "dirs" A future patch will make use of this separation so that we can make more intelligent decisions about what to investigate/load when the matcher is in use. Currently, even with this patch, we typically use the 'visitdir' call to identify if we can skip some directory, something along the lines of: for f in all_items: if match.visitdir(f): <do stuff> This can be slower than we'd like if there are a lot of items; it requires N calls to match.visitdir in the best case. Commonly, especially with 'narrow', we have a situation where we do some work for the directory, possibly just loading it from disk (when using treemanifests) and then check if we should be interacting with it at all, which can be a huge slowdown in some pathological cases. Differential Revision: https://phab.mercurial-scm.org/D4129

match: add tests for visitdir functionality There are a few cases that we could have done better with some additional logic; I tried to annotate these when I noticed them, but may have missed some. The tests are not exhaustive; there are certainly some patterns that I didn't test well, and many that I didn't test at all. The primary motivation was to get coverage on visitdir so that I can cover identical cases in a similar method I'm working on, to make sure that this new method behaves the same (or better). Differential Revision: https://phab.mercurial-scm.org/D4128

mergetool: warn if ui.merge points to nonexistent tool This adds a warning when ui.merge is configured but points to an executable that doesn't exist. It gets printed once per fail, but that seems to be how our other warnings about merge tools are reported. Differential Revision: https://phab.mercurial-scm.org/D3975

tests: demonstrate that no requested merge tool is ignored if missing If you explicitly configure a merge tool, it seems wrong that we don't even warn if we can't find it. This patch adds a test case that demonstrates that. Differential Revision: https://phab.mercurial-scm.org/D3974

fix: correctly set wdirwritten given that the dict item is deleted Differential Revision: https://phab.mercurial-scm.org/D4146

fix: pull out flag definitions to make them re-usable from extensions This makes it cleaner to implement fix-related commands with additional functionality while sharing some flags with the core implementation. Differential Revision: https://phab.mercurial-scm.org/D4145

templatekw: copy {author} to {user} and document {author} as an alias In other places including "log -Tjson" and revset, "user" is the canonical name. Let's standardize it. This is a part of the name unification of the Generic Templating Plan. https://www.mercurial-scm.org/wiki/GenericTemplatingPlan#Dictionary

templates: rename "user" to "luser" defined in default map file (API) "user" will be shadowed by the {user} keyword to be added by the next patch. I think the naming of template fields is a sort of an internal API, so this patch is flagged as an API change. .. api:: Rewrite ``{user}`` to ``{luser}`` in log templates inherited from map-cmdline.default.

grep: add MULTIREV support to --allfiles flag This patch facilitates passing multiple revisions with all-files flag. It's assumed that if you are passing multiple revisions to --allfiles, you want hits from all of them. Differential Revision: https://phab.mercurial-scm.org/D3976

phabricator: convert description into local The description from conduit is a unicode. Differential Revision: https://phab.mercurial-scm.org/D3980

index: move index_clearcaches() further down I want to add a call from it to a new function (nt_dealloc) that will be inserted below its current position. Differential Revision: https://phab.mercurial-scm.org/D4117

index: move all "nt_*" functions to one place Differential Revision: https://phab.mercurial-scm.org/D4116

index: rename "nt_*(indexObject *self,...)" functions to "index_*" These functions do something with the nodetree, but they're less generic and won't make sense as methods on the nodetree when it becomes a Python type. Differential Revision: https://phab.mercurial-scm.org/D4115

index: split up nt_init() in two I'd like to make nt_init() take a pointer to a nodetree to initialize, but it currently also allocates the nodetree. This patch prepares for that change by making nt_init() be about initializing an existing node tree and by creating a new index_init_nt() that creates the nodetree. Differential Revision: https://phab.mercurial-scm.org/D4114

index: make most "nt_*" functions take a nodetree Now that the nodetree has a pointer to the index, we can pass the nodtree instead of the index. There are few "nt_*" functions left after this. I'll deal with them soon. Differential Revision: https://phab.mercurial-scm.org/D4113

index: add pointer from nodetree back to index This is always a cycle right now, but it will not be for the nodetree instances I'm planning to add later (see earlier patch). Differential Revision: https://phab.mercurial-scm.org/D4112

index: remove side-effect from failed nt_new() As pointed out by Yuya in the review of D4108, if realloc() fails, we would end up with an invalid nodetree instance (with nt->nodes set to NULL), which means that if it was later accessed again it would likely segfault. It's probably unlikely that much else happens in the process if it ran out memory, but we should of course do our best to handle it. This patch makes it so we don't update the nodetree in this case. Differential Revision: https://phab.mercurial-scm.org/D4154

index: remove side-effect from failed nt_init() As pointed out by Yuya in the review of D4108, if we run into the "overflow in nt_init" case (which I think normally happens only in repos with at least 2^26=64Mi revisions), we would leave the node tree half-initialized. Differential Revision: https://phab.mercurial-scm.org/D4153

index: use PyMem_Free() to free nodeetree instance As Yuya pointed out in the review of D4108, PyMem_Malloc() and PyMem_Free() should be paired. IIUC, PyMem_Malloc() may use a different allocator than malloc(), so using free() with a pointer from PyMem_Malloc() may be very wrong. Differential Revision: https://phab.mercurial-scm.org/D4152

linelog: fix infinite loop vulnerability Checking `len(lines)` is not a great way of detecting infinite loops, as demonstrated in the added test. Therefore check instruction count instead. The original C implementation does not have this problem. There are a few other places where the C implementation enforces more strictly, like `a1 <= a2`, `b1 <= b2`, `rev > 0`. But they are optional. Test Plan: Add a test. The old code forces the test to time out. Differential Revision: https://phab.mercurial-scm.org/D4151

tests: fix bytes/str issues in run-tests.py caught by python3 Differential Revision: https://phab.mercurial-scm.org/D4143

changegroup: assign to proper attribute 0548f696795b accidentally assigned to self.clrevtolocalrev instead of self._clrevtolocalrev. Surprisingly, no tests failed as a result of this mistake. Curious. Differential Revision: https://phab.mercurial-scm.org/D4144

absorb: remove sf alias for command I'm not even sure what it is supposed to stand for. Differential Revision: https://phab.mercurial-scm.org/D4126

hgweb: catch ParseError that's raised by revset.match() Some queries, like the demonstrated "first(::)", fail earlier than we call mfunc(), and that results in a "500 Internal Server Error". To prevent it, revset.match() also needs to be in a try-except block.

templatekw: deprecate old-style template keyword function (API) .. api:: `f(**kwargs)` style template keyword function is deprecated. Switch to new `(context, mapping)` API by declaring resource requirements. The new-style API will be the default in Mercurial 4.9. See registrar.templatekeyword for details.

hgweb: mark all lambda template keywords as new-style function This is just a temporary workaround, and will be removed in Mercurial 4.9.

hgweb: use registrar to add "motd" template keyword This prepares for deprecation of old-style keyword functions.

fileset: load core predicates directly to symbols dict We no longer have any side effect in loadpredicate().

fileset: turn on listclean conditionally This is just a micro optimization.

changegroup: always use the treemanifest-enabled version of _packmanifests() It works for flat manifests too. We just cannot use cg1 or cg2 if we have subdirectory manifests. Differential Revision: https://phab.mercurial-scm.org/D4124

linelog: add replacelines_vec for fastannotate # no-check-commit because we're conforming to an existing interface Differential Revision: https://phab.mercurial-scm.org/D3993

absorb: drop wrapper around the amend command We can reinstate this later if we want. Differential Revision: https://phab.mercurial-scm.org/D3992

absorb: note some TODOs from the code review Differential Revision: https://phab.mercurial-scm.org/D4047

absorb: use ui.debug() instead of open-coding it Differential Revision: https://phab.mercurial-scm.org/D4046

absorb: use pycompat to get xrange Differential Revision: https://phab.mercurial-scm.org/D4045

absorb: use set literal to avoid intermediate list Differential Revision: https://phab.mercurial-scm.org/D4044

absorb: avoid mutable default arg Differential Revision: https://phab.mercurial-scm.org/D4043

absorb: import extension from Facebook's hg-experimental absorb is a wicked-fast command to use blame information to automatically amend edits to the correct draft revision. Originally written by Jun Wu, this import is hgext3rd/absorb/__init__.py with: * the `testedwith` value changed * the linelog import updated * some missing configitems registered * some imports reordered per check-code.py * some missing __future__ imports added per check-code.py Differential Revision: https://phab.mercurial-scm.org/D3991

resolve: organize 'if confirm' conditionals Differential Revision: https://phab.mercurial-scm.org/D4123

index: pass only nodetree to nt_new() The function now only depends on the nodetree, not the index. Differential Revision: https://phab.mercurial-scm.org/D4111

index: drop now-redundant "nt" prefix of fields in nodetree struct Differential Revision: https://phab.mercurial-scm.org/D4110

index: move more fields onto nodetree type The fields moves are the ones that are not related to how the nodetree is used in the index and that will make sense for the new nodetree instance for a subset of the index that I'll add later. Differential Revision: https://phab.mercurial-scm.org/D4109

index: extract a type for the nodetree This is a first step towards exposing the nodetree as a Python type. Differential Revision: https://phab.mercurial-scm.org/D4108

index: make "nt_*" functions work on an initialized nodetree I want to be able to reuse these functions with another nodetree instance later (for disambiguating node prefix within a revset). That other nodetree instance won't want to be fully populated from the index, so this commit moves that part to the callers. Differential Revision: https://phab.mercurial-scm.org/D4107

changegroup: inline _packellipsischangegroup It now does nothing special. The logic is simple enough to inline in the 2 callers in narrow that need it. The changegroup generation APIs could probably be a bit simpler. But that's for another time. Differential Revision: https://phab.mercurial-scm.org/D4092

changegroup: move fullnodes into cgpacker And with this change, the narrow packer no longer defines any addition attributes on packer instances! Differential Revision: https://phab.mercurial-scm.org/D4091

changegroup: specify ellipses mode explicitly Currently, code throughout changegroup relies on the presence of self._full_nodes to enable ellipses mode. This is a very tenuous check. And the check may be wrong once we move _full_nodes into cgpacker. Let's capture the enabling of ellipses mode explicitly as a constructor argument and as an instance variable. We could probably derive ellipses mode by presence of other variables. But for now, this explicit approach seems simplest since it is most compatible with existing code. Differential Revision: https://phab.mercurial-scm.org/D4090