largefiles: port wrapped functions to exthelper Things get interesting in the commit. I hadn't seen issue6033 on Windows, and yet it is now reproducible 100% of the time on Windows 10 with this commit. I didn't test Linux. (For comparison, after seeing this issue, I tested on the parent with --loop, and it failed 5 times out of over 1300 tests.) The strange thing is that largefiles has nothing to do with that test (it's not even mentioned there). It isn't autoloading run amuck- it occurs even if largefiles is explicitly disabled, and also if the entry in afterhgrcload() is commented out. It's also not the import of lfutil- I disabled that by copying the function into lfs and removing the import, and the problem still occurs. Experimenting further, it seems that the problem is isolated to 3 entries: exchange.pushoperation, hg.clone, and cmdutil.revert. If those decorators are all commented out, the test passes when run in a loop for awhile. (Obviously, some largefiles tests will fail.) But if any one is commented back in, the test fails immediately. I left one method related to wrapping the wire protocol, because it seemed more natural with the TODO. Also, exthelper doesn't support wrapping functions from another extension, only commands in another extension. I didn't try to figure out why rebase is both command wrapped and function wrapped.

largefiles: port commands to exthelper One subtle change here is that the purge, rebase and transplant extensions are wrapped in extsetup() instead of uisetup().

exthelper: support the option argument when registering a command Largefiles uses this 5th argument with push and pull, so this will be tested in the next commit. I assume the reason for unrolling and reforming the tuple in each finalxxxsetup() is to validate that something proper was passed in when registering. But it's better to explode when decorating than during the delayed actual registration.

largefiles: port configitems to exthelper It looks like dynamicdefault was referenced in a non-standard way.

remotefilelog: use repo.local() instead of isinstance()

exthelper: add a cautionary note about adding attributes to containers

exthelper: drop a compatibility shim with '^command' syntax This syntax was dropped in fa88170c10bb.

wireproto: in batch queries, support queries with immediate responses listkeys and pushkey return without querying the remote when the remote doesn't support such queries. Before this change, the batching code didn't handle this convention, resulting in this kind of error: $ hg pull ssh://user@dummy/repo1 -r tip -B a pulling from ssh://user@dummy/repo1 remote: ** Unknown exception encountered with possibly-broken third-party extension disable-lookup remote: ** which supports versions unknown of Mercurial. remote: ** Please disable disable-lookup and try your action again. remote: ** If that fixes the bug please report it to the extension author. remote: ** Python 2.7.15+ (default, Oct 2 2018, 22:12:08) [GCC 8.2.0] remote: ** Mercurial Distributed SCM (version 4.8.1+586-ef54bd33b476+20181224) remote: ** Extensions loaded: disable-lookup remote: Traceback (most recent call last): remote: File "/tmp/hgtests.i66Npc/install/bin/hg", line 43, in <module> remote: dispatch.run() remote: File "/tmp/hgtests.i66Npc/install/lib/python/mercurial/dispatch.py", line 99, in run remote: status = dispatch(req) remote: File "/tmp/hgtests.i66Npc/install/lib/python/mercurial/dispatch.py", line 225, in dispatch remote: ret = _runcatch(req) or 0 remote: File "/tmp/hgtests.i66Npc/install/lib/python/mercurial/dispatch.py", line 376, in _runcatch remote: return _callcatch(ui, _runcatchfunc) remote: File "/tmp/hgtests.i66Npc/install/lib/python/mercurial/dispatch.py", line 384, in _callcatch remote: return scmutil.callcatch(ui, func) remote: File "/tmp/hgtests.i66Npc/install/lib/python/mercurial/scmutil.py", line 166, in callcatch remote: return func() remote: File "/tmp/hgtests.i66Npc/install/lib/python/mercurial/dispatch.py", line 367, in _runcatchfunc remote: return _dispatch(req) remote: File "/tmp/hgtests.i66Npc/install/lib/python/mercurial/dispatch.py", line 1021, in _dispatch remote: cmdpats, cmdoptions) remote: File "/tmp/hgtests.i66Npc/install/lib/python/mercurial/dispatch.py", line 756, in runcommand remote: ret = _runcommand(ui, options, cmd, d) remote: File "/tmp/hgtests.i66Npc/install/lib/python/mercurial/dispatch.py", line 1030, in _runcommand remote: return cmdfunc() remote: File "/tmp/hgtests.i66Npc/install/lib/python/mercurial/dispatch.py", line 1018, in <lambda> remote: d = lambda: util.checksignature(func)(ui, *args, **strcmdopt) remote: File "/tmp/hgtests.i66Npc/install/lib/python/mercurial/util.py", line 1670, in check remote: return func(*args, **kwargs) remote: File "/tmp/hgtests.i66Npc/install/lib/python/mercurial/commands.py", line 5257, in serve remote: s.serve_forever() remote: File "/tmp/hgtests.i66Npc/install/lib/python/mercurial/wireprotoserver.py", line 797, in serve_forever remote: self.serveuntil(threading.Event()) remote: File "/tmp/hgtests.i66Npc/install/lib/python/mercurial/wireprotoserver.py", line 804, in serveuntil remote: _runsshserver(self._ui, self._repo, self._fin, self._fout, ev) remote: File "/tmp/hgtests.i66Npc/install/lib/python/mercurial/wireprotoserver.py", line 656, in _runsshserver remote: rsp = wireprotov1server.dispatch(repo, proto, request) remote: File "/tmp/hgtests.i66Npc/install/lib/python/mercurial/wireprotov1server.py", line 74, in dispatch remote: return func(repo, proto, *args) remote: File "/tmp/hgtests.i66Npc/install/lib/python/mercurial/wireprotov1server.py", line 195, in batch remote: data[k] = vals[k] remote: KeyError: 'namespace' abort: unexpected response: empty string [255] Differential Revision: https://phab.mercurial-scm.org/D5482

rust: core implementation for lazyancestors Once exposed through appropriate bindings, this should be able to replace ancestor.lazyancestors entirely. Differential Revision: https://phab.mercurial-scm.org/D5440

rust-cpython: binding for AncestorsIterator It's now reachable from Python as rustext.ancestor.AncestorsIterator Tests are provided in the previously introduced Python testcase: this is much more convenient that writing lengthy Rust code to call into Python. Differential Revision: https://phab.mercurial-scm.org/D5439

rust-cpython: implement Graph using C parents function We introduce the `Index` struct that wraps the C index. It is not intrinsically protected by the GIL (see the lengthy discussion in its docstring). Improving on this seems prematurate at this point. A pointer to the parents function is stored on the parsers C extension module as a capsule object. This is the recommended way to export a C API for consumption from other extensions. See also: https://docs.python.org/2.7/c-api/capsule.html In our case, we use it in cindex.rs, retrieving function pointer from the capsule and storing it within the CIndex struct, alongside with a pointer to the index. From there, the implementation is very close to the one from hg-direct-ffi. The naming convention for the capsule is inspired from the one in datetime: >>> import datetime >>> datetime.datetime_CAPI <capsule object "datetime.datetime_CAPI" at 0x7fb51201ecf0> although in datetime's case, the capsule points to a struct holding several type objects and methods. Differential Revision: https://phab.mercurial-scm.org/D5438

pull: fix inconsistent view of bookmarks during pull (issue4700) I had a share where a pull apparently pulled a bookmark but not the revision pointed to by the bookmark, which I suspect is due to this (and if not, we might as well remove known issues in this area). I do this by combining doing all the queries that could read the bookmarks in one round trip. I had to change the handling of the case where the server doesn't support the lookup query, because if it fails, it would otherwise make fremotebookmark.result() block forever. This is due to wireprotov1peer.peerexecutor.sendcommands's behavior (it fills a single future if any query fails synchronously and leaves all other futures unchanged), but I don't know if the fix is to cancel all other futures, or to keep going with the other queries. Differential Revision: https://phab.mercurial-scm.org/D5449

merge: modify the logical statement Differential Revision: https://phab.mercurial-scm.org/D5476

exthelper: correct a documentation typo

lfs: convert to using exthelper to wrap functions I'm not 100% sure that upgraderequirements() can be double annotated safely, but it seems OK based on printing the address of the function being wrapped. One thing I've noticed is that @eh.reposetup doesn't do the usual check to ensure that it's a local repo. Should that be baked into @eh.reposetup() somehow, possibly with a non-default option to skip the check? It seems like a gaping hole if every function that gets registered needs to add this check.

tests: convert a test extension to use exthelper This provides test coverage to uipopulate().

exthelper: drop fileset/revset/template support for now Yuya raised concerns about duplicating registrar functionality. There are a couple of ideas to work around this, which would allow bringing them back, and then backporting to evolve. For now, I just want to get the subsequent changes landed before the bulk b'' rewrite makes rebasing too hard.

exthelper: simplify configitem registration

extensions: import the exthelper class from evolve 980565468003 (API) This should help make extensions that wrap a lot of stuff more comprehendible. It was copied unmodified, except: - fix up the imports - rename final_xxxsetup() -> finalxxxsetup() to appease checkcode - avoid a [] default arg to wrapcommand() .. api:: Add `exthelper` class to simplify extension writing by allowing functions, commands, and configitems to be registered via annotations. The previous APIs are still available for use.

narrow: detect if narrowspec was changed in a different share With this commit, `hg share` should be usable with narrow repos. Design explained on https://www.mercurial-scm.org/wiki/NarrowSharePlan I was running into cache invalidation problems when updating the narrowspec. After spending a day trying to figure out a good solution, I resorted to just assigning repo.narrowpats and repo._narrowmatch after invalidating them. Differential Revision: https://phab.mercurial-scm.org/D5278

tests: add test for narrow+share For how narrow+share is supposed to work, see https://www.mercurial-scm.org/wiki/NarrowSharePlan. Differential Revision: https://phab.mercurial-scm.org/D5276

narrow: keep narrowspec backup in store As suggested by Yuya in review of D4099. Differential Revision: https://phab.mercurial-scm.org/D5470

tests: update narrowspec when narrowspec, not dirstate, is accessed test-narrow-expanddirstate.t mimics a Google-internal extension that updates the narrowspec whenever the dirstate is accessed. Since 1d09ba0d2ed3 (narrow: move remaining narrow-limited dirstate walks to core, 2018-10-01) and a few commits before it, we no longer restrict repo.dirstate.walk() to the narrowspec. It is instead done at a higher level (e.g. context.status()). We were running into problems with the Google-internal extension when importing those commits. The issue was that the narrowspec was read before the first dirstate access. I believe the right fix is to instead update the narrowspec when trying to read it (not when reading the dirstate), so that's what this patch does. Differential Revision: https://phab.mercurial-scm.org/D5275

merge: extract helper for creating empty "actions" dict Replicating the set of actions in multiple places is bad. Differential Revision: https://phab.mercurial-scm.org/D5472

manifest: accept narrowmatch into constructor instead of getting from repo The manifest should ideally not know at all about the repo, so this is just a little step towards cleaning that up. Differential Revision: https://phab.mercurial-scm.org/D5469

py3: byteify one more sys.argv in gendoc.py

test: introduce a new flag to display env variable line per line It's easier to conditionalize some of the environment variables per Mercurial version once there is only one value per line. Differential Revision: https://phab.mercurial-scm.org/D5453

revlog: add an explicit test for `issnapshot` We test the method on a real revlog containing "real" data.

revlog: add some direct testing of the slicing logic This test check slicing backed by an actual revlog. It will test the C version of slicing (if the test are run with the C extensions).

revlog: limit base to rev size ratio to 500 instead of 50 While a value of 50 provided large speedup in some case (400s → 7s) it also creates a slow down for a whole class of revision we are seeing in a private repository (0.1s → 3s). A value of 500 makes them disappear improving the total runtime (the slower revision still improve significantly (400s → 21s)). We need to run a wider array of tests on various repositories to see the effect on speed and size of different values for this acceptable constant. However, in the meantime, it seems safer to move back to a less impactful value.