index: create function for deleting node from nodetree A node is marked deleted by nt_insert(self, node, -1), but "-1" could mean so many things (notably nullrev), so let's hide it in a function. The function could possibly also remove nodes from the tree in the future (it currently only marks them deleted). Differential Revision: https://phab.mercurial-scm.org/D4004

revlog: remove some knowledge of sentinel nullid in index I think the "-2" to mean "last position in index, not counting the null revision at the end" is an implementation detail of the index that we should avoid spreading knowledge of. I hope we can even remove support for index[-2]. Differential Revision: https://phab.mercurial-scm.org/D4016

scmutil: make shortest() respect disambiguation revset The previous patch would let you use a shorter prefix if the prefix is unique within a configured revset. However, that's not very useful if there's no simple way of knowing what that shorter prefix is. This patch adapts the shortest() template function to use the shorter prefixes for nodes in the configured revset. This is currently extremely slow, because it calculates the revset for each call to shortest(). To make this faster, the next patch will start caching the revset instance. Ideally we'd cache a prefix tree instance instead. Differential Revision: https://phab.mercurial-scm.org/D4038

lookup: add option to disambiguate prefix within revset When resolving a nodeid prefix that is not unique within the repo and the user has configured a revset that they want to disambiguate within, we now try to look up within that revset before we fail. If there is a unique match within the revset, we use that. This is of course most effective at allowing a short prefix if the revset contains few nodes. For most of our internal users at Google, "not public()" is sufficiently small that a hex digit or two is enough. The implementation is currently pretty slow, but good enough for small revsets (which is the expected use case). The scan in the revset is linear. We may want to use a prefix tree if we want to allow users to use a larger revset. Credit for the idea goes to Kyle Lippincott. Differential Revision: https://phab.mercurial-scm.org/D4037

revlog: use specialized exception for ambiguous prefix lookup It's useful to be able to catch a specific exception for this case. We'll use it soon. Differential Revision: https://phab.mercurial-scm.org/D4036

narrowspec: remove the unused _parsestoredpatterns() function Previous patch started using sparse.parseconfig() to read narrowspec and now we don't need this function. Differential Revision: https://phab.mercurial-scm.org/D4058

narrowspec: use sparse.parseconfig() to parse narrowspec file (BC) This also make narrow files use 'include' and 'exclude' instead of plural forms which are 'includes' and 'excludes'. This is BC because existing narrowspecs have to replace excludes, includes words with exclude and include i.e. their singular versions. Differential Revision: https://phab.mercurial-scm.org/D4057

sparse: add an action argument to parseconfig() This will help us in reusing this function to parse narrow config files and unfiying the config file parsing logic. Differential Revision: https://phab.mercurial-scm.org/D4056

tests: avoid unnecessarily looking inside .hg/ in test-narrow* Many of the narrow tests were looking at .hg/narrowspec. I think most of them were written before we added `hg tracked`, but now that we have that command, we should fix the tests. There were also a few instances of tests looking at .hg/dirstate. I fixed those to use `hg files` instead. Differential Revision: https://phab.mercurial-scm.org/D4093

narrow: extract part of narrowspec backup to core Just a little preparation for the next patch. Differential Revision: https://phab.mercurial-scm.org/D4095

narrow: move requirement constant from changegroup to repository As suggested by Gregory Szorc. Differential Revision: https://phab.mercurial-scm.org/D4094

merge with stable

fileset: optimize 'x and not y' to 'x - y' 'x - y' is first rewritten to 'x and not y' so that x and y are reordered by weight.

fileset: reorder 'and' expression to evaluate basic patterns first Timing of a crafted example (when disk cache is warm): $ hg files set:'binary() and path:contrib' (orig) time: real 0.140 secs (user 0.120+0.000 sys 0.020+0.000) (new) time: real 0.040 secs (user 0.030+0.000 sys 0.010+0.000)

fileset: roughly adjust weights of functions ... in order to move status predicates far away from basic patterns. I don't know if each weight is appropriate, but it should be good enough as a start.

fileset: add stub for weight-based optimization The main purpose of this change is to group basic patterns, which can be mapped to a plain matcher. I'm not so interested in a weight of each function.

fileset: reject 'negate' node early while transforming parsed tree That's how a 'negate' node is processed in revset.

fileset: drop 'group' node from tree to be evaluated This helps rewriting a parsed tree.

fileset: add phase to transform parsed tree This isn't strictly necessary, but I decided to just follow the strategy of the revset parsing.

py3: whitelist another 3 tests caught by the ratchet Differential Revision: https://phab.mercurial-scm.org/D4069

testrunner: allow multiple #testcases This lets you have multiple #testcases declarations and they're taken to be different dimensions of the test, so their cross product becomes the total set of test cases. For example: #testcases obsstore-on obsstore-off #testcases manifest-flat manifest-tree $ hg rebase ... ... #if obsstore-on $ hg log ... #endif Note that this is an excellent way to slow down the test suite, so use it with care. The feature is implemented by replacing most of the "case" variables that were strings before by an array of strings with each item a different dimension of the test case. The file names are created by joining the dimensions by "#" (e.g. test-foo.t#obsstore-on#manifest-flat). Differential Revision: https://phab.mercurial-scm.org/D4049

testrunner: use "#" for "test cases" suffix in .err filename too This seems like a natural follow-up to b865bba56db1 (run-tests: update the test case name format, 2018-05-13). Differential Revision: https://phab.mercurial-scm.org/D4052

resolve: add confirm config option This config setting gives a functionality to confirm before it re-merge all unresolved files. If this config is enabled, when you run 'hg resolve --all' it will prompt with a msg "re-merge all unresolved files (yn)?" To enable this functionality: [commands] resolve.confirm = True Differential Revision: https://phab.mercurial-scm.org/D3988

revlog: remove side effect from failed nt_init() If nt_init() successfully allocates memory for the node tree but then runs out of memory while trying to insert nullid into it, it will leave the node tree pointer set on the index object. That means that future node tree operations will think that the tree is properly initialized. It seems very unlikely to make a difference in practice (if nt_init() runs out of memory, not much else will probably work), but since I spotted it, I figured I might as well fix it. Differential Revision: https://phab.mercurial-scm.org/D4028

revlog: remove micro-optimization for looking up only nullid index_find_node() would call nt_find() before initializing the node tree. nt_find() would then return -2 unless the requested revision was the null revision. I can't imagine what scenario that is optimizing for, and doing the initialization earlier makes the code simpler and easier to follow, so that's what this patch does. Differential Revision: https://phab.mercurial-scm.org/D4027

revlog: remove unnecessary output parameter from node_check() The "nodelen" output parameter is always set to 20 if the function returns successfully. Differential Revision: https://phab.mercurial-scm.org/D4026

narrow: move status-filtering to core and to ctx One of my recent changes from repo.status(ctx1, ctx2) to ctx1.status(ctx2) broke some of our Google-internal tests. The problem turned out to be that the narrow extension was overriding repo.status() to make it filter out paths outside the narrowspec. When I changed to ctx1.status(ctx2), then that filtering obviously got lost. ctx.status() seems like a better method to do the filtering in, so this patch moves the filtering into that method, thereby also moving it out of the extension and into core. Differential Revision: https://phab.mercurial-scm.org/D4068

amend: support "history-editing-backup" config option Now, amend is in the list of those history editing commands which support `history-editing-backup` config option. If you don't want to store any backup then just use this config. [ui] hisotry-editing-backup = False Current status of list of history editing commands which support this config: 1. histedit 2. rebase 3. amend Differential Revision: https://phab.mercurial-scm.org/D3968

rebase: move "backup" flag to rebaseruntime It was getting messy to populate "backup" flag to calls of `_finishrebase()` and `_prepareabortorcontinue`, so made some changes to move "backup" flag to rbsrt. Differential Revision: https://phab.mercurial-scm.org/D4055

i18n: format warning of hggettext in standard compiler error style Now, hggettext specific warning messages are formatted in: FILENAME:LINENO:MESSAGE This allows editors to jump into corresponded line easily.

i18n: omit redundant translatable synopsis text to avoid xgettext warning This empty translatable synopsis text causes xgettext warning below: Empty msgid. It is reserved by GNU gettext: gettext("") returns the header entry with meta information, not the empty string. This "synopsis" argument of @command annotation is omitable.

i18n: avoid substitution of PYFILES at runtime for readability of output This substitution decreases readability of "make update-pot" output, because PYFILES consists of many files. This patch makes "make update-pot" show "find mercurial hgext doc -name '*.py'" instead of many *.py files at runtime.

util: make new timedcmstats class Python 3 compatible

fix: add a monkey-patchable point after all new revisions have been committed Differential Revision: https://phab.mercurial-scm.org/D4048

help: add quotes to a few commands we point to I didn't know that 'hg help "revsets.x or y"' was valid syntax, so the quoting is extra useful here to make it clear that that is an actual command. Differential Revision: https://phab.mercurial-scm.org/D4059

linelog: port to Python 3 Differential Revision: https://phab.mercurial-scm.org/D4051

exchange: move simple narrow changegroup generation from extension The narrow extension completely replaced the function generating the changegroup part when a narrow changegroup was requested. Previous commits have taught the in-core changegroup code how to filter files based on a matcher. This commit teaches the in-core bundle2 part generation code to construct a matcher based on arguments. It will also emit a bundle2 part describing the narrow spec. I believe the changegroup part generation code in the narrow extension is now limited to ellipsis serving mode. i.e. core is now capable of narrow changegroup generation when ellipsis mode is disabled. Differential Revision: https://phab.mercurial-scm.org/D4014

exchange: make narrow ACL presence imply narrow=True And refactor the logic for determining when to invoke our custom changegroup part function so it is only conditional on narrow being set. This makes it more obvious under which conditions we should invoke the custom implementation. Also use raw strings so **kwargs works on Python 3. Differential Revision: https://phab.mercurial-scm.org/D4013

changegroup: inline prune() logic from narrow prune() needs to ellide manifests that aren't part of the narrow matcher. The code is violating storage abstractions, so a comment has been added. Keep in mind the impetus for moving this code to core is so changegroup code can be refactored to be storage agnostic. Differential Revision: https://phab.mercurial-scm.org/D4012

fileset: extract language processing part to new module (API) I'll add a couple more functions that work on parsed tree. % wc -l mercurial/fileset*.py 559 mercurial/fileset.py 135 mercurial/filesetlang.py 694 total

fileset: flatten 'or' nodes to unnest unionmatchers This also makes it easier to compile a union of basic patterns into a single regexp pattern.

fileset: flatten arguments list Just prepares for flattening 'or' nodes. This change would have no impact on performance.

debugfileset: add option to show matcher representation

debugfileset: backport --show-stage option from debugrevspec I'll add some static optimizations to fileset.

lfs: remove callstatus property from 'lfs()' fileset It was added at 91aac8e6604d, but is no longer needed since a fileset expression is now compiled into an "open" matcher. See ff5b6fca1082 for details.

rebase: support "history-editing-backup" config option If you don't want to store any backup while rebasing, you can use `history-editing-backup` config option. [ui] history-editing-backup = # True or False Current status of list of commands which supports this config: 1. histedit 2. rebase Differential Revision: https://phab.mercurial-scm.org/D3887

extensions: add detailed loading information This lets you track down what exactly is happening during extension loading, and how long various steps took.

util: create a context manager to handle timing The context manager is pulled out of the timed decorator function, and refactored to provide a stats instance, with added tests.

linelog: fix import statements to pass the import checker on py3 Differential Revision: https://phab.mercurial-scm.org/D4050

linelog: add a Python implementation of the linelog datastructure This datastructure was originally developed by Jun Wu at Facebook, inspired by SCCS weaves. It's useful as a cache for blame information, but also is the magic that makes `hg absorb` easy to implement. In service of importing the code to Mercurial, I wanted to actually /understand/ it, and once I did I decided to take a run at implementing it. The help/internals/linelog.txt document is the README from Jun Wu's implementaiton. It all applies to our linelog implementation. Differential Revision: https://phab.mercurial-scm.org/D3990

changegroup: move file matcher from narrow extension Sparse changegroup generation requires the use of a matcher to filter which files are relevant. This commit moves the file matcher from the narrow extension to core and updates the narrow extension to use it. I'm not sure why the narrow extension was storing the matcher as a callable that resolved to a matcher. So I changed it to be a simple matcher instance. In addition, code from narrow to intersect the matcher with the local narrow spec is now performed automatically when the changegroup packer is created. If a matcher is not passed into getbundler() an alwaysmatcher() is assumed. This ensures that a matcher is always defined for all operations. Differential Revision: https://phab.mercurial-scm.org/D4011

resolve: add option to warn/abort on -m with unresolved conflict markers When a user is dropped out of Mercurial to a terminal to resolve files, we emit messages like: conflicts while merging file1! (edit, then use 'hg resolve --mark') conflicts while merging file2! (edit, then use 'hg resolve --mark') We don't mention a file name in the hint, so some users might do something like `$EDITOR file1; hg resolve --mark`, see that it says "(no more unresolved files)" and forget to deal with file2 before running the next command. Even if we did mention a file name in the hint, it's too easy to forget it (maybe the merge spans a couple days or something). This option lets us inform the user that they might have missed something. In the scenario above, the output would be something like: warning: the following files still have conflict markers: file2 (no more unresolved files) Differential Revision: https://phab.mercurial-scm.org/D4035

exchange: refactor control flow of _getbundlechangegrouppart() The use of early return makes the control flow of this function much easier to reason about IMO. Differential Revision: https://phab.mercurial-scm.org/D4010

exchange: move _computeellipsis() from narrow This is also referenced as part of the narrow changegroup code and therefore needs to move to core before we can integrate the narrow changegroup code into core. Differential Revision: https://phab.mercurial-scm.org/D4009

exchange: move narrow acl functionality into core This function is called by the custom changegroup generation code in the narrow extension. I want to move that changegroup code into core. That means we need to move this function. The code is kinda hacky in that assumes existence of REMOTE_USER, which is only present on authenticated HTTP requests. I've added a comment indicating that. Differential Revision: https://phab.mercurial-scm.org/D4008

exchange: move disabling of rev-branch-cache bundle part out of narrow I'm attempting to refactor changegroup code in order to better support alternate storage backends. The narrow extension is performing a lot of monkeypatching to this code and it is making it difficult to reason about how everything works. I'm reasonably certain I would be unable to abstract storage without requiring extensive rework of narrow. I believe it is less effort to move narrow code into core so it can be accounted for when changegroup code is refactored. So I'll be doing that. The first part of this is integrating the disabling of the cache:rev-branch-cache bundle2 part into core. This doesn't seem like it is related to changegroup, but narrow's modifications to changegroup are invasive and also require taking its code for bundle generation and exchange into core in order for the changegroup code to work. Differential Revision: https://phab.mercurial-scm.org/D4007

dispatch: show a short error message when invalid global option given Similar reasoning as the previous patch. Differential Revision: https://phab.mercurial-scm.org/D4025

dispatch: don't show list of commands on bogus command If a command is ambiguous, you get this: $ hg ve hg: command 've' is ambiguous: verify version [255] If you typo a command, you get this: $ hg comit hg: unknown command 'comit' (did you mean one of commit, incoming, mycommit?) [255] But if you completely mistype a command so it no longer looks like any existing commands, you get a full list of commands. That might be useful the first time you use Mercurial, but after that it's probably more annoying than help, especially if you have the pager enabled and have a short terminal. Let's instead give a short hint telling the user to run `hg help` for more help. Differential Revision: https://phab.mercurial-scm.org/D4024

histedit: avoid repeating name of state file in a few places We can depend on the state object instead. Differential Revision: https://phab.mercurial-scm.org/D4006

py3: stop rewriting xrange() to pycompat.xrange() We now require the use of pycompat.xrange() in source. One less feature in the module importer gets us one step closer to Python 3 native source code. Differential Revision: https://phab.mercurial-scm.org/D4034