discovery: minor fix to some conditionals Since `size` is the upper limit of the sample, we should include it in the check. Otherwize the `more` variable will be zero and the sampling will be useless

delta: reuse _findsnapshot call from previous stage Two different stage of the sparse-revlog logic needs the _findsnapshot data. To avoid recomputing it twice, make it possible to reuse the first computation in the second step. example affected manifest write before: 0.067141s after: 0.064252s (-5%) (total gain since start of series: 95%)

delta: have a native implementation of _findsnapshot The function might traverse a lot of revision, a native implementation get significantly faster. example affected manifest write before: 0.114989 after: 0.067141 (-42%)

transaction: do not overwrite atomic-temp files on error Even though the original files can be restored from the backup, it should be better to not write back a temporary file if we know it can be corrupted.

help: fix typo

rust: use 'impl Trait' in method argument of AncestorsIterator I just didn't know it's stabilized. Let's switch to new convenient syntax. Differential Revision: https://phab.mercurial-scm.org/D5451

rust: use .rev() for reverse range Differential Revision: https://phab.mercurial-scm.org/D5450

histedit: add warning message on editing tagged commits (issue4017) Differential Revision: https://phab.mercurial-scm.org/D5489

strip: extract bookmark movement into a separate function We will need it for the soft-strip case.

strip: compute bookmark target only if we have bookmark to move This is a small change that seems to make sense.

strip: extract code to create strip backup We will reuse this for soft stripping.

strip: clarify comment around bundle create We may create two bundles, one for the changeset actually stripped, and one for the changeset affected by the strip we want to keep. (Changesets with a higher rev number than the striped ones). For soft stripping, we still need to the first one (used for "restoring" the changesets), but not the second one. We start with clarifying which bits of the code is used for what.

strip: move attributes shortcut assigned earlier This series will add support for "soft" stripping using the archived phase in addition to the usual heavy weight revlog stripping. First, we need to refactor the strip function to extract logic common to the "soft-strip" case. We are about to extract bits of the strip function into separate functions. Those blocks of code contain variables for frequently accessed attributes. We move those variables outside of the soon to be moved blocks.

log: fix line wrap on diffstat with -G/--graph (issue5800) Differential Revision: https://phab.mercurial-scm.org/D5485

registrar: minor cleanup to the merge function Pointed out by Yuya.

vfs: raise NotImplementedError in abstractvfs._auditvfs This make sure that auditing is implemented.

vfs: handle _auditpath in proxyvfs Just forward the call to the underlying vfs.

vfs: fix proxyvfs inheritance The proxyvfs class is designed to overwrite some of the vfs logic. Yet, it did not use normal class inheritance. This is becoming an issue as `abstractvfs` method could take precedence over their `proxyvfs` version. We fix the inheritance chain to be as expected.

vfs: also audit rename Renaming through the vfs is not used in many places, and none of them seems to be a security risk. However, it is still worthwhile to run the auditing on rename file to perform developer-warning level checks.

vfs: add a `_auditpath` to abstract vfs We are about to make `rename` audit path. Since rename lives in the `abstractvfs` layer, we need it to be aware of auditing to some extent. The default implementation is no-op because multiple existing vfs are not using auditing at all right now (eg: fncachevfs).

vfs: makes all audited path relative Only auditing relative path helps the vfs warning logic.

debugupgraderepo: add a --no-backup mode The process has been around for a while and is pretty safe now. Having an automated way to clean up the old data is useful when running many different conversion to compare delta algorithm.

upgrade: add '-' in optimization name The older name `redeltaall` was hard to type and read. The newer form should be more user-friendly. We keep backward compatibility with the old form (at least for a while). Having to use different form depending on the version is very impractical and error prone.

revlog: add test case for _findsnapshots Testing the method directly is more robust.

revlog: use the native implementation of issnapshot In some sparserevlog case where a lot of the history has to be searched for a snapshot, the cost of issnashot cost becomes significant. The computation done by the method is fairly low level, a native implementation provide a very significant speedup. example affected manifest write before: 0.490375s after: 0.114989s (-76%)

revlog: add a native implementation of issnapshot This will be used in the next changesets

revlog: more efficient implementation for issnapshot We avoid multiple method calls and tuple creation, this provides a significant speedup in some case: example affected manifest write before: 0.815520s after: 0.487767s (-40%)

revlog: clarify the non sparse-revlog case in `issnapshot` If we are not using sparse-revlog, there is only one type of snapshot, the full one.

exthelper: switch to using the registrar merging method

registrar: add a method to merge registrar instances This provides sanity checking beyond simply merging the underlying dictionaries.

phabricator: ensure that the return of urlopener.open() is closed No problem observed, just an oversight noticed while reading documentation.

revlog: fix pure python slicing test when chain contains nullrev The revlog mock used in the test was not behaving the same as its C counterpart.

merge with stable

remotefilelog: fix bug in maybesparsematch returning alwaysmatcher The description of the method says that it should return None if sparse is not used in this repository; since sparse.matcher() returns alwaysmatcher if sparse is not enabled, I'm using that as the signal to return None here to preserve the previous behavior. Differential Revision: https://phab.mercurial-scm.org/D5487

procutil: correct spelling of uninterruptable -> uninterruptible Differential Revision: https://phab.mercurial-scm.org/D5488

help: show advanced, experimental and deprecated extensions with --verbose I noticed that phabricator and absorb weren't showing up, no matter what. There's a related problem where commands for these extensions *do* show up without --verbose. I'm not sure what the point of hiding the extensions, but not the commands is.

help: remove a duplicate category name entry for TOPIC_CATEGORY_NONE

phabricator: assign commands to help categories `phabsend` and `phabupdate` seem pretty close to `email`. `phabread` is a little less close, but can be trivially used to actually import patches.

largefiles: eliminate an unnecessary import of configitems Thanks to Yuya for pointing this out.

exthelper: add some examples for using registrar aliases Maybe it's my general lack of python knowledge, but how to use these would be way too obscure for me otherwise.

exthelper: reintroduce the ability to register filesets Same mechanism as revsets earlier in this series. The LFS extension is updated to provide test coverage.

exthelper: reintroduce the ability to register templates Same mechanism as revsets earlier in this series. The LFS extension is updated to provide test coverage. This also seems to make the test failure around issue6033 mentioned in 0a7f582f6f1f much less regular on Windows.

extensions: deprecate extsetup without a `ui` argument (API) 9.5 years should be enough time, but there were some tests for the old style still (which are now updated). Exthelper doesn't fallback to the old API, so this is for consistency. .. api:: The extension hook ``extsetup`` without a `ui` argument has been deprecated, and will be removed in the next version. Add a `ui` argument to avoid the deprecation warning.

largefiles: port revset registration to exthelper This tests the merge code that wasn't tested in the previous patch.

exthelper: reintroduce the ability to register revsets I think this is what Yuya and Boris agreed on.[1] This happens *after* the extsetup phase now (and after the _aftercallback notifications). But this is trivial, mergeable between exthelper instances, and doesn't need to have the extension name supplied when registering. The test needed updating so that extsetup() takes a `ui` argument, as exthelper isn't trying to be backward compatible with 1.3.1. [1] https://www.mercurial-scm.org/pipermail/mercurial-devel/2018-December/125888.html

largefiles: drop the uisetup module This is small enough to live in the __init__ module.

branches: add -r option to show branch name(s) of a given rev (issue5948) Differential Revision: https://phab.mercurial-scm.org/D5486

progress: avoid ui.configbool() lookup when progress bar is active Profiling revealed that the ui.configbool('progress', 'debug') during progress bar updates was consuming a significant amount of overhead. This commit adds an attribute on progress bar instances that caches this config option. The impact on `hg perfprogress` with default options is significant: before: ! wall 4.641942 comb 4.580000 user 4.210000 sys 0.370000 (best of 3) after: ! wall 1.948626 comb 1.950000 user 1.950000 sys 0.000000 (best of 5) After this change, profiling reveals that progress.progbar.progress() is now consuming ~73% of time. This change does not improve the execution time if the progress bar is disabled. We may want a more comprehensive solution for that case, as the progress bar won't be enabled in a number of scenarios (e.g. servers and processes not attached to an interactive TTY). I also think that overhead of ~2.0s for 1M updates is a bit high. I suspect further refactoring of the progress bar can significantly reduce overhead. I don't have plans to do this, however. Differential Revision: https://phab.mercurial-scm.org/D5408

largefiles: port wrapped functions to exthelper Things get interesting in the commit. I hadn't seen issue6033 on Windows, and yet it is now reproducible 100% of the time on Windows 10 with this commit. I didn't test Linux. (For comparison, after seeing this issue, I tested on the parent with --loop, and it failed 5 times out of over 1300 tests.) The strange thing is that largefiles has nothing to do with that test (it's not even mentioned there). It isn't autoloading run amuck- it occurs even if largefiles is explicitly disabled, and also if the entry in afterhgrcload() is commented out. It's also not the import of lfutil- I disabled that by copying the function into lfs and removing the import, and the problem still occurs. Experimenting further, it seems that the problem is isolated to 3 entries: exchange.pushoperation, hg.clone, and cmdutil.revert. If those decorators are all commented out, the test passes when run in a loop for awhile. (Obviously, some largefiles tests will fail.) But if any one is commented back in, the test fails immediately. I left one method related to wrapping the wire protocol, because it seemed more natural with the TODO. Also, exthelper doesn't support wrapping functions from another extension, only commands in another extension. I didn't try to figure out why rebase is both command wrapped and function wrapped.

largefiles: port commands to exthelper One subtle change here is that the purge, rebase and transplant extensions are wrapped in extsetup() instead of uisetup().

exthelper: support the option argument when registering a command Largefiles uses this 5th argument with push and pull, so this will be tested in the next commit. I assume the reason for unrolling and reforming the tuple in each finalxxxsetup() is to validate that something proper was passed in when registering. But it's better to explode when decorating than during the delayed actual registration.

largefiles: port configitems to exthelper It looks like dynamicdefault was referenced in a non-standard way.

remotefilelog: use repo.local() instead of isinstance()

exthelper: add a cautionary note about adding attributes to containers

exthelper: drop a compatibility shim with '^command' syntax This syntax was dropped in fa88170c10bb.

wireproto: in batch queries, support queries with immediate responses listkeys and pushkey return without querying the remote when the remote doesn't support such queries. Before this change, the batching code didn't handle this convention, resulting in this kind of error: $ hg pull ssh://user@dummy/repo1 -r tip -B a pulling from ssh://user@dummy/repo1 remote: ** Unknown exception encountered with possibly-broken third-party extension disable-lookup remote: ** which supports versions unknown of Mercurial. remote: ** Please disable disable-lookup and try your action again. remote: ** If that fixes the bug please report it to the extension author. remote: ** Python 2.7.15+ (default, Oct 2 2018, 22:12:08) [GCC 8.2.0] remote: ** Mercurial Distributed SCM (version 4.8.1+586-ef54bd33b476+20181224) remote: ** Extensions loaded: disable-lookup remote: Traceback (most recent call last): remote: File "/tmp/hgtests.i66Npc/install/bin/hg", line 43, in <module> remote: dispatch.run() remote: File "/tmp/hgtests.i66Npc/install/lib/python/mercurial/dispatch.py", line 99, in run remote: status = dispatch(req) remote: File "/tmp/hgtests.i66Npc/install/lib/python/mercurial/dispatch.py", line 225, in dispatch remote: ret = _runcatch(req) or 0 remote: File "/tmp/hgtests.i66Npc/install/lib/python/mercurial/dispatch.py", line 376, in _runcatch remote: return _callcatch(ui, _runcatchfunc) remote: File "/tmp/hgtests.i66Npc/install/lib/python/mercurial/dispatch.py", line 384, in _callcatch remote: return scmutil.callcatch(ui, func) remote: File "/tmp/hgtests.i66Npc/install/lib/python/mercurial/scmutil.py", line 166, in callcatch remote: return func() remote: File "/tmp/hgtests.i66Npc/install/lib/python/mercurial/dispatch.py", line 367, in _runcatchfunc remote: return _dispatch(req) remote: File "/tmp/hgtests.i66Npc/install/lib/python/mercurial/dispatch.py", line 1021, in _dispatch remote: cmdpats, cmdoptions) remote: File "/tmp/hgtests.i66Npc/install/lib/python/mercurial/dispatch.py", line 756, in runcommand remote: ret = _runcommand(ui, options, cmd, d) remote: File "/tmp/hgtests.i66Npc/install/lib/python/mercurial/dispatch.py", line 1030, in _runcommand remote: return cmdfunc() remote: File "/tmp/hgtests.i66Npc/install/lib/python/mercurial/dispatch.py", line 1018, in <lambda> remote: d = lambda: util.checksignature(func)(ui, *args, **strcmdopt) remote: File "/tmp/hgtests.i66Npc/install/lib/python/mercurial/util.py", line 1670, in check remote: return func(*args, **kwargs) remote: File "/tmp/hgtests.i66Npc/install/lib/python/mercurial/commands.py", line 5257, in serve remote: s.serve_forever() remote: File "/tmp/hgtests.i66Npc/install/lib/python/mercurial/wireprotoserver.py", line 797, in serve_forever remote: self.serveuntil(threading.Event()) remote: File "/tmp/hgtests.i66Npc/install/lib/python/mercurial/wireprotoserver.py", line 804, in serveuntil remote: _runsshserver(self._ui, self._repo, self._fin, self._fout, ev) remote: File "/tmp/hgtests.i66Npc/install/lib/python/mercurial/wireprotoserver.py", line 656, in _runsshserver remote: rsp = wireprotov1server.dispatch(repo, proto, request) remote: File "/tmp/hgtests.i66Npc/install/lib/python/mercurial/wireprotov1server.py", line 74, in dispatch remote: return func(repo, proto, *args) remote: File "/tmp/hgtests.i66Npc/install/lib/python/mercurial/wireprotov1server.py", line 195, in batch remote: data[k] = vals[k] remote: KeyError: 'namespace' abort: unexpected response: empty string [255] Differential Revision: https://phab.mercurial-scm.org/D5482

rust: core implementation for lazyancestors Once exposed through appropriate bindings, this should be able to replace ancestor.lazyancestors entirely. Differential Revision: https://phab.mercurial-scm.org/D5440

rust-cpython: binding for AncestorsIterator It's now reachable from Python as rustext.ancestor.AncestorsIterator Tests are provided in the previously introduced Python testcase: this is much more convenient that writing lengthy Rust code to call into Python. Differential Revision: https://phab.mercurial-scm.org/D5439

rust-cpython: implement Graph using C parents function We introduce the `Index` struct that wraps the C index. It is not intrinsically protected by the GIL (see the lengthy discussion in its docstring). Improving on this seems prematurate at this point. A pointer to the parents function is stored on the parsers C extension module as a capsule object. This is the recommended way to export a C API for consumption from other extensions. See also: https://docs.python.org/2.7/c-api/capsule.html In our case, we use it in cindex.rs, retrieving function pointer from the capsule and storing it within the CIndex struct, alongside with a pointer to the index. From there, the implementation is very close to the one from hg-direct-ffi. The naming convention for the capsule is inspired from the one in datetime: >>> import datetime >>> datetime.datetime_CAPI <capsule object "datetime.datetime_CAPI" at 0x7fb51201ecf0> although in datetime's case, the capsule points to a struct holding several type objects and methods. Differential Revision: https://phab.mercurial-scm.org/D5438

pull: fix inconsistent view of bookmarks during pull (issue4700) I had a share where a pull apparently pulled a bookmark but not the revision pointed to by the bookmark, which I suspect is due to this (and if not, we might as well remove known issues in this area). I do this by combining doing all the queries that could read the bookmarks in one round trip. I had to change the handling of the case where the server doesn't support the lookup query, because if it fails, it would otherwise make fremotebookmark.result() block forever. This is due to wireprotov1peer.peerexecutor.sendcommands's behavior (it fills a single future if any query fails synchronously and leaves all other futures unchanged), but I don't know if the fix is to cancel all other futures, or to keep going with the other queries. Differential Revision: https://phab.mercurial-scm.org/D5449

merge: modify the logical statement Differential Revision: https://phab.mercurial-scm.org/D5476

exthelper: correct a documentation typo

lfs: convert to using exthelper to wrap functions I'm not 100% sure that upgraderequirements() can be double annotated safely, but it seems OK based on printing the address of the function being wrapped. One thing I've noticed is that @eh.reposetup doesn't do the usual check to ensure that it's a local repo. Should that be baked into @eh.reposetup() somehow, possibly with a non-default option to skip the check? It seems like a gaping hole if every function that gets registered needs to add this check.

tests: convert a test extension to use exthelper This provides test coverage to uipopulate().