debugupgraderepo: add a --no-backup mode The process has been around for a while and is pretty safe now. Having an automated way to clean up the old data is useful when running many different conversion to compare delta algorithm.

upgrade: add '-' in optimization name The older name `redeltaall` was hard to type and read. The newer form should be more user-friendly. We keep backward compatibility with the old form (at least for a while). Having to use different form depending on the version is very impractical and error prone.

revlog: add test case for _findsnapshots Testing the method directly is more robust.

revlog: use the native implementation of issnapshot In some sparserevlog case where a lot of the history has to be searched for a snapshot, the cost of issnashot cost becomes significant. The computation done by the method is fairly low level, a native implementation provide a very significant speedup. example affected manifest write before: 0.490375s after: 0.114989s (-76%)

revlog: add a native implementation of issnapshot This will be used in the next changesets

revlog: more efficient implementation for issnapshot We avoid multiple method calls and tuple creation, this provides a significant speedup in some case: example affected manifest write before: 0.815520s after: 0.487767s (-40%)

revlog: clarify the non sparse-revlog case in `issnapshot` If we are not using sparse-revlog, there is only one type of snapshot, the full one.

exthelper: switch to using the registrar merging method

registrar: add a method to merge registrar instances This provides sanity checking beyond simply merging the underlying dictionaries.

phabricator: ensure that the return of urlopener.open() is closed No problem observed, just an oversight noticed while reading documentation.

revlog: fix pure python slicing test when chain contains nullrev The revlog mock used in the test was not behaving the same as its C counterpart.

merge with stable

remotefilelog: fix bug in maybesparsematch returning alwaysmatcher The description of the method says that it should return None if sparse is not used in this repository; since sparse.matcher() returns alwaysmatcher if sparse is not enabled, I'm using that as the signal to return None here to preserve the previous behavior. Differential Revision: https://phab.mercurial-scm.org/D5487

procutil: correct spelling of uninterruptable -> uninterruptible Differential Revision: https://phab.mercurial-scm.org/D5488

help: show advanced, experimental and deprecated extensions with --verbose I noticed that phabricator and absorb weren't showing up, no matter what. There's a related problem where commands for these extensions *do* show up without --verbose. I'm not sure what the point of hiding the extensions, but not the commands is.

help: remove a duplicate category name entry for TOPIC_CATEGORY_NONE

phabricator: assign commands to help categories `phabsend` and `phabupdate` seem pretty close to `email`. `phabread` is a little less close, but can be trivially used to actually import patches.

largefiles: eliminate an unnecessary import of configitems Thanks to Yuya for pointing this out.

exthelper: add some examples for using registrar aliases Maybe it's my general lack of python knowledge, but how to use these would be way too obscure for me otherwise.

exthelper: reintroduce the ability to register filesets Same mechanism as revsets earlier in this series. The LFS extension is updated to provide test coverage.

exthelper: reintroduce the ability to register templates Same mechanism as revsets earlier in this series. The LFS extension is updated to provide test coverage. This also seems to make the test failure around issue6033 mentioned in 0a7f582f6f1f much less regular on Windows.

extensions: deprecate extsetup without a `ui` argument (API) 9.5 years should be enough time, but there were some tests for the old style still (which are now updated). Exthelper doesn't fallback to the old API, so this is for consistency. .. api:: The extension hook ``extsetup`` without a `ui` argument has been deprecated, and will be removed in the next version. Add a `ui` argument to avoid the deprecation warning.

largefiles: port revset registration to exthelper This tests the merge code that wasn't tested in the previous patch.

exthelper: reintroduce the ability to register revsets I think this is what Yuya and Boris agreed on.[1] This happens *after* the extsetup phase now (and after the _aftercallback notifications). But this is trivial, mergeable between exthelper instances, and doesn't need to have the extension name supplied when registering. The test needed updating so that extsetup() takes a `ui` argument, as exthelper isn't trying to be backward compatible with 1.3.1. [1] https://www.mercurial-scm.org/pipermail/mercurial-devel/2018-December/125888.html

largefiles: drop the uisetup module This is small enough to live in the __init__ module.

branches: add -r option to show branch name(s) of a given rev (issue5948) Differential Revision: https://phab.mercurial-scm.org/D5486

progress: avoid ui.configbool() lookup when progress bar is active Profiling revealed that the ui.configbool('progress', 'debug') during progress bar updates was consuming a significant amount of overhead. This commit adds an attribute on progress bar instances that caches this config option. The impact on `hg perfprogress` with default options is significant: before: ! wall 4.641942 comb 4.580000 user 4.210000 sys 0.370000 (best of 3) after: ! wall 1.948626 comb 1.950000 user 1.950000 sys 0.000000 (best of 5) After this change, profiling reveals that progress.progbar.progress() is now consuming ~73% of time. This change does not improve the execution time if the progress bar is disabled. We may want a more comprehensive solution for that case, as the progress bar won't be enabled in a number of scenarios (e.g. servers and processes not attached to an interactive TTY). I also think that overhead of ~2.0s for 1M updates is a bit high. I suspect further refactoring of the progress bar can significantly reduce overhead. I don't have plans to do this, however. Differential Revision: https://phab.mercurial-scm.org/D5408

largefiles: port wrapped functions to exthelper Things get interesting in the commit. I hadn't seen issue6033 on Windows, and yet it is now reproducible 100% of the time on Windows 10 with this commit. I didn't test Linux. (For comparison, after seeing this issue, I tested on the parent with --loop, and it failed 5 times out of over 1300 tests.) The strange thing is that largefiles has nothing to do with that test (it's not even mentioned there). It isn't autoloading run amuck- it occurs even if largefiles is explicitly disabled, and also if the entry in afterhgrcload() is commented out. It's also not the import of lfutil- I disabled that by copying the function into lfs and removing the import, and the problem still occurs. Experimenting further, it seems that the problem is isolated to 3 entries: exchange.pushoperation, hg.clone, and cmdutil.revert. If those decorators are all commented out, the test passes when run in a loop for awhile. (Obviously, some largefiles tests will fail.) But if any one is commented back in, the test fails immediately. I left one method related to wrapping the wire protocol, because it seemed more natural with the TODO. Also, exthelper doesn't support wrapping functions from another extension, only commands in another extension. I didn't try to figure out why rebase is both command wrapped and function wrapped.

largefiles: port commands to exthelper One subtle change here is that the purge, rebase and transplant extensions are wrapped in extsetup() instead of uisetup().

exthelper: support the option argument when registering a command Largefiles uses this 5th argument with push and pull, so this will be tested in the next commit. I assume the reason for unrolling and reforming the tuple in each finalxxxsetup() is to validate that something proper was passed in when registering. But it's better to explode when decorating than during the delayed actual registration.

largefiles: port configitems to exthelper It looks like dynamicdefault was referenced in a non-standard way.

remotefilelog: use repo.local() instead of isinstance()

exthelper: add a cautionary note about adding attributes to containers

exthelper: drop a compatibility shim with '^command' syntax This syntax was dropped in fa88170c10bb.

wireproto: in batch queries, support queries with immediate responses listkeys and pushkey return without querying the remote when the remote doesn't support such queries. Before this change, the batching code didn't handle this convention, resulting in this kind of error: $ hg pull ssh://user@dummy/repo1 -r tip -B a pulling from ssh://user@dummy/repo1 remote: ** Unknown exception encountered with possibly-broken third-party extension disable-lookup remote: ** which supports versions unknown of Mercurial. remote: ** Please disable disable-lookup and try your action again. remote: ** If that fixes the bug please report it to the extension author. remote: ** Python 2.7.15+ (default, Oct 2 2018, 22:12:08) [GCC 8.2.0] remote: ** Mercurial Distributed SCM (version 4.8.1+586-ef54bd33b476+20181224) remote: ** Extensions loaded: disable-lookup remote: Traceback (most recent call last): remote: File "/tmp/hgtests.i66Npc/install/bin/hg", line 43, in <module> remote: dispatch.run() remote: File "/tmp/hgtests.i66Npc/install/lib/python/mercurial/dispatch.py", line 99, in run remote: status = dispatch(req) remote: File "/tmp/hgtests.i66Npc/install/lib/python/mercurial/dispatch.py", line 225, in dispatch remote: ret = _runcatch(req) or 0 remote: File "/tmp/hgtests.i66Npc/install/lib/python/mercurial/dispatch.py", line 376, in _runcatch remote: return _callcatch(ui, _runcatchfunc) remote: File "/tmp/hgtests.i66Npc/install/lib/python/mercurial/dispatch.py", line 384, in _callcatch remote: return scmutil.callcatch(ui, func) remote: File "/tmp/hgtests.i66Npc/install/lib/python/mercurial/scmutil.py", line 166, in callcatch remote: return func() remote: File "/tmp/hgtests.i66Npc/install/lib/python/mercurial/dispatch.py", line 367, in _runcatchfunc remote: return _dispatch(req) remote: File "/tmp/hgtests.i66Npc/install/lib/python/mercurial/dispatch.py", line 1021, in _dispatch remote: cmdpats, cmdoptions) remote: File "/tmp/hgtests.i66Npc/install/lib/python/mercurial/dispatch.py", line 756, in runcommand remote: ret = _runcommand(ui, options, cmd, d) remote: File "/tmp/hgtests.i66Npc/install/lib/python/mercurial/dispatch.py", line 1030, in _runcommand remote: return cmdfunc() remote: File "/tmp/hgtests.i66Npc/install/lib/python/mercurial/dispatch.py", line 1018, in <lambda> remote: d = lambda: util.checksignature(func)(ui, *args, **strcmdopt) remote: File "/tmp/hgtests.i66Npc/install/lib/python/mercurial/util.py", line 1670, in check remote: return func(*args, **kwargs) remote: File "/tmp/hgtests.i66Npc/install/lib/python/mercurial/commands.py", line 5257, in serve remote: s.serve_forever() remote: File "/tmp/hgtests.i66Npc/install/lib/python/mercurial/wireprotoserver.py", line 797, in serve_forever remote: self.serveuntil(threading.Event()) remote: File "/tmp/hgtests.i66Npc/install/lib/python/mercurial/wireprotoserver.py", line 804, in serveuntil remote: _runsshserver(self._ui, self._repo, self._fin, self._fout, ev) remote: File "/tmp/hgtests.i66Npc/install/lib/python/mercurial/wireprotoserver.py", line 656, in _runsshserver remote: rsp = wireprotov1server.dispatch(repo, proto, request) remote: File "/tmp/hgtests.i66Npc/install/lib/python/mercurial/wireprotov1server.py", line 74, in dispatch remote: return func(repo, proto, *args) remote: File "/tmp/hgtests.i66Npc/install/lib/python/mercurial/wireprotov1server.py", line 195, in batch remote: data[k] = vals[k] remote: KeyError: 'namespace' abort: unexpected response: empty string [255] Differential Revision: https://phab.mercurial-scm.org/D5482

rust: core implementation for lazyancestors Once exposed through appropriate bindings, this should be able to replace ancestor.lazyancestors entirely. Differential Revision: https://phab.mercurial-scm.org/D5440

rust-cpython: binding for AncestorsIterator It's now reachable from Python as rustext.ancestor.AncestorsIterator Tests are provided in the previously introduced Python testcase: this is much more convenient that writing lengthy Rust code to call into Python. Differential Revision: https://phab.mercurial-scm.org/D5439

rust-cpython: implement Graph using C parents function We introduce the `Index` struct that wraps the C index. It is not intrinsically protected by the GIL (see the lengthy discussion in its docstring). Improving on this seems prematurate at this point. A pointer to the parents function is stored on the parsers C extension module as a capsule object. This is the recommended way to export a C API for consumption from other extensions. See also: https://docs.python.org/2.7/c-api/capsule.html In our case, we use it in cindex.rs, retrieving function pointer from the capsule and storing it within the CIndex struct, alongside with a pointer to the index. From there, the implementation is very close to the one from hg-direct-ffi. The naming convention for the capsule is inspired from the one in datetime: >>> import datetime >>> datetime.datetime_CAPI <capsule object "datetime.datetime_CAPI" at 0x7fb51201ecf0> although in datetime's case, the capsule points to a struct holding several type objects and methods. Differential Revision: https://phab.mercurial-scm.org/D5438

pull: fix inconsistent view of bookmarks during pull (issue4700) I had a share where a pull apparently pulled a bookmark but not the revision pointed to by the bookmark, which I suspect is due to this (and if not, we might as well remove known issues in this area). I do this by combining doing all the queries that could read the bookmarks in one round trip. I had to change the handling of the case where the server doesn't support the lookup query, because if it fails, it would otherwise make fremotebookmark.result() block forever. This is due to wireprotov1peer.peerexecutor.sendcommands's behavior (it fills a single future if any query fails synchronously and leaves all other futures unchanged), but I don't know if the fix is to cancel all other futures, or to keep going with the other queries. Differential Revision: https://phab.mercurial-scm.org/D5449

merge: modify the logical statement Differential Revision: https://phab.mercurial-scm.org/D5476

exthelper: correct a documentation typo

lfs: convert to using exthelper to wrap functions I'm not 100% sure that upgraderequirements() can be double annotated safely, but it seems OK based on printing the address of the function being wrapped. One thing I've noticed is that @eh.reposetup doesn't do the usual check to ensure that it's a local repo. Should that be baked into @eh.reposetup() somehow, possibly with a non-default option to skip the check? It seems like a gaping hole if every function that gets registered needs to add this check.

tests: convert a test extension to use exthelper This provides test coverage to uipopulate().

exthelper: drop fileset/revset/template support for now Yuya raised concerns about duplicating registrar functionality. There are a couple of ideas to work around this, which would allow bringing them back, and then backporting to evolve. For now, I just want to get the subsequent changes landed before the bulk b'' rewrite makes rebasing too hard.

exthelper: simplify configitem registration

extensions: import the exthelper class from evolve 980565468003 (API) This should help make extensions that wrap a lot of stuff more comprehendible. It was copied unmodified, except: - fix up the imports - rename final_xxxsetup() -> finalxxxsetup() to appease checkcode - avoid a [] default arg to wrapcommand() .. api:: Add `exthelper` class to simplify extension writing by allowing functions, commands, and configitems to be registered via annotations. The previous APIs are still available for use.

narrow: detect if narrowspec was changed in a different share With this commit, `hg share` should be usable with narrow repos. Design explained on https://www.mercurial-scm.org/wiki/NarrowSharePlan I was running into cache invalidation problems when updating the narrowspec. After spending a day trying to figure out a good solution, I resorted to just assigning repo.narrowpats and repo._narrowmatch after invalidating them. Differential Revision: https://phab.mercurial-scm.org/D5278

tests: add test for narrow+share For how narrow+share is supposed to work, see https://www.mercurial-scm.org/wiki/NarrowSharePlan. Differential Revision: https://phab.mercurial-scm.org/D5276

narrow: keep narrowspec backup in store As suggested by Yuya in review of D4099. Differential Revision: https://phab.mercurial-scm.org/D5470

tests: update narrowspec when narrowspec, not dirstate, is accessed test-narrow-expanddirstate.t mimics a Google-internal extension that updates the narrowspec whenever the dirstate is accessed. Since 1d09ba0d2ed3 (narrow: move remaining narrow-limited dirstate walks to core, 2018-10-01) and a few commits before it, we no longer restrict repo.dirstate.walk() to the narrowspec. It is instead done at a higher level (e.g. context.status()). We were running into problems with the Google-internal extension when importing those commits. The issue was that the narrowspec was read before the first dirstate access. I believe the right fix is to instead update the narrowspec when trying to read it (not when reading the dirstate), so that's what this patch does. Differential Revision: https://phab.mercurial-scm.org/D5275

merge: extract helper for creating empty "actions" dict Replicating the set of actions in multiple places is bad. Differential Revision: https://phab.mercurial-scm.org/D5472

manifest: accept narrowmatch into constructor instead of getting from repo The manifest should ideally not know at all about the repo, so this is just a little step towards cleaning that up. Differential Revision: https://phab.mercurial-scm.org/D5469

py3: byteify one more sys.argv in gendoc.py

test: introduce a new flag to display env variable line per line It's easier to conditionalize some of the environment variables per Mercurial version once there is only one value per line. Differential Revision: https://phab.mercurial-scm.org/D5453

revlog: add an explicit test for `issnapshot` We test the method on a real revlog containing "real" data.

revlog: add some direct testing of the slicing logic This test check slicing backed by an actual revlog. It will test the C version of slicing (if the test are run with the C extensions).

revlog: limit base to rev size ratio to 500 instead of 50 While a value of 50 provided large speedup in some case (400s → 7s) it also creates a slow down for a whole class of revision we are seeing in a private repository (0.1s → 3s). A value of 500 makes them disappear improving the total runtime (the slower revision still improve significantly (400s → 21s)). We need to run a wider array of tests on various repositories to see the effect on speed and size of different values for this acceptable constant. However, in the meantime, it seems safer to move back to a less impactful value.

contrib: remove unused version of os Caught by test-check-pyflakes.t Differential Revision: https://phab.mercurial-scm.org/D5471

py3: convert `'{}'.format(foo)` to `'%s' % foo` in the bookflow extension Byte strings don't have the former. Converting these to byte strings is waiting on the mass rewrite.

py3: byteify sys.argv in gendoc.py