rust: using policy.importrust from Python callers This commit converts all current Python callers of mercurial.rustext to the new policy.importrust system. After this point, going through policy.importrust or policy.importmod (in some more distant future) is mandatory for callers of Rust code outside of Python tests. We felt it to be appropriate to keep Rust-specific tests run inconditionally if the Rust extensions are present.

rust: module policy with importrust We introduce two rust+c module policies and a new `policy.importrust()` that makes use of them. This simple approach provides runtime switching of implementations, which is crucial for the performance measurements such as those Octobus does with ASV. It can also be useful for bug analysis. It also has the advantage of making conditionals in Rust callers more uniform, in particular abstracting over specifics like `demandimport` At this point, the build stays unchanged, with the rust-cpython based `rustext` module being built if HGWITHRUSTEXT=cpython. More transparency for the callers, i.e., just using `policy.importmod` would be a much longer term and riskier effort for the following reasons: 1. It would require to define common module boundaries for the three or four cases (pure, c, rust+ext, cffi) and that is premature with the Rust extension currently under heavy development in areas that are outside the scope of the C extensions. 2. It would imply internal API changes that are not currently wished, as the case of ancestors demonstrates. 3. The lack of data or property-like attributes (tp_member and tp_getset) in current `rust-cpython` makes it impossible to achieve direct transparent replacement of pure Python classes by Rust extension code, meaning that the caller sometimes has to be able to make adjustments or provide additional wrapping.

help: add help entry for internals.mergestate This patch adds an entry for `internals.mergestate` as suggested by @marmoute. Most of the help text is taken from `merge.mergestate`. Differential Revision: https://phab.mercurial-scm.org/D6448 Differential Revision: https://phab.mercurial-scm.org/D6528

phabricator: use parents.set to always set dependencies Now that Mercurial's Phabricator instance has been updated to a version that supports the parents.set transaction on revision.edit we can use that to set dependency relationships in patch stacks instead of abusing the summary. This has the advantage that we can use it on every `phabsend` so commit reordering is picked up without spamming changes like abusing the summary would, and using parents.set will clear previous parents unlike parents.add. Differential Revision: https://phab.mercurial-scm.org/D6514

help: remove repeated word in 'hg help rebase' Specifically, the second 'with' in 'with which to merge with'. Differential Revision: https://phab.mercurial-scm.org/D6483

rebase: tweak description of inmemory working even w/ dirty working dir One of our users was confused because they read this, and then attempted to run `hg rebase` with a dirty working directory, and it still complained. The reason was that they were attempting to rebase the commit they currently had checked out, which (at least with evolve workflows enabled) involves updating the working directory to be based on the newly rebased commit. Differential Revision: https://phab.mercurial-scm.org/D6507

revlog: speed up isancestor Currently, it is implemented on top of commonancestorsheads. Implement it on top of reachableroots instead, as reachableroots could stop walking the graph much sooner than commonancestorsheads. Measuring repo.changelog.isancestorrev on two revisions in a private repository: before: ! wall 0.005175 comb 0.010000 user 0.010000 sys 0.000000 (best of 550) after : ! wall 0.000072 comb 0.000000 user 0.000000 sys 0.000000 (best of 36199) When hg does this kind of operations 1500 times in pull -> bookmarks.comparebookmarks -> bookmarks.validdest, that's 11s that drop from the --profile output. Differential Revision: https://phab.mercurial-scm.org/D6506

dagop: fix documentation of reachableroots The previous revset couldn't be correct as it is symmetric in <roots> and <heads>, but reachableroots has no such symmetry. It makes a difference with for instance reachableroots(2, 3) where 2 and 3 are both children of 1. Differential Revision: https://phab.mercurial-scm.org/D6505

phabricator: add --blocker argument to phabsend to specify blocking reviewers The way to signal to Conduit that a reviewer is considered blocking is just to wrap their PHID in "blocking()" when including it in the list of PHIDs passed to `reviewers.add`. arc doesn't have a --blocker, instead one is supposed to append a '!' to the end of reviewer names (I think reviewers are usually added in an editor rather than the command line, where '!'s can be more hazardous). moz-phab (Mozilla's arcanist wrapper) does have a --blocker argument, and being explicit like this is also more discoverable. Even `arc diff`'s help doesn't seem to mention the reviewer! syntax. Differential Revision: https://phab.mercurial-scm.org/D6512

phabricator: auto-sanitise API tokens and HTTP cookies from VCR recordings Currently when making VCR recordings one needs to manually sanitise sensitive credentials before committing and submitting them as part of tests. It is easy to imagine this being accidentally missed one time by a fallible human and said credentials being leaked. It is also possible that it wouldn't be noticed to alert the user to the leak since the recording files are so large and practically unreviewable. Thus do so automatically, so the only place that needs checking is in the test-phabricator.t file. Differential Revision: https://phab.mercurial-scm.org/D6513

py3: use .startswith() instead of bytes[0] Doing bytes[0] will return the ascii value of that position which breaks comparison with a bytechar. This makes test-absorb.t work again on py3. Differential Revision: https://phab.mercurial-scm.org/D6508

revset: fix merge() to fall back to changectx API if wdir specified I have a code which basically runs "0:wdir() & <user-revset>", and it crashed if merge() were passed in.

revset: use nullrev constant in merge()

mixedrepostorecache: fix a silly redundant updating of set Differential Revision: https://phab.mercurial-scm.org/D6470

rust-regex: fix shortcut for exact matches The current shortcut for rootglobs that can be simplified to exact matches does not work, it instead treats the pattern as a regex, which is not the same thing. This changes fixes the behavior and introduces a test for this behavior. Differential Revision: https://phab.mercurial-scm.org/D6489

rust-filepatterns: use bytes instead of String In my initial patch, I introduced an unnecessary hard constraint on UTF-8 filenames and patterns which I forgot to remove. Although the performance penalty for using String might be negligible, we don't want to break compatibility with non-UTF-8 encodings for no reason. Moreover, this change allows for a cleaner Rust core API. This patch introduces a new utils module that is used with this fix. Finally, PatternError was not put inside the Python module generated by Rust, which would have raised a NameError. Differential Revision: https://phab.mercurial-scm.org/D6485

doc: fix description of "predecessors" to match reality Differential Revision: https://phab.mercurial-scm.org/D6467

phabricator: make `hg debugcallconduit` work outside a hg repo I am trying to write some automations around phabricator and having debugcallconduit work outside a hg repo will be nice! Marking command as optionalrepo instead of norepo because we might to load repo/.hg/hgrc. Differential Revision: https://phab.mercurial-scm.org/D6499

phabricator: pass ui instead of repo to callconduit This will help us make `hg debugcallconduit` work outside a hg repo as next patch will mark that command as no repo. Differential Revision: https://phab.mercurial-scm.org/D6498

phabricator: pass ui into readurltoken instead of passing repo The goal of this series is to make `hg debugcallconduit` work outside of a hg repo. This patch, removes requirement of repo object from readurltoken as we only need ui there. It also updates the callers to pass in ui instead of repo. Differential Revision: https://phab.mercurial-scm.org/D6497

py3: add test-contrib-emacs.t to passing tests list I installed emacs on the server running buildbot and the test started passing on Python 3. Lets add it to the list of passing test. Differential Revision: https://phab.mercurial-scm.org/D6500

phabricator: add commenting to phabsend for new/updated Diffs Especially useful when sending updates to existing Revisions so one can specify the sort of changes e.g. "Address review comments" or "Rebase to tip" If the diff content hasn't changed then it only needs a metadata update which doesn't show in the Phabricator updates UI, so don't add a comment that will. Differential Revision: https://phab.mercurial-scm.org/D6496

py3: fix test-bookmarks-corner-case.t For some reasons, the output of print was not going through. Replaced that ui.status(). Differential Revision: https://phab.mercurial-scm.org/D6481

py3: fix test-fix-metadata.t # skip-blame as just b'' prefixes Differential Revision: https://phab.mercurial-scm.org/D6480

py3: add b'' prefix at one place in run-tests.py #skip-blame because just b'' prefix Differential Revision: https://phab.mercurial-scm.org/D6482

copies: separate added/removed files by newline instead of null This makes it more consistent with how we encode copies (newline-separated lists of null-separated pairs). This perhaps makes {extras} a little less readable (?) despite avoiding the escaping. I don't know how I feel about this patch. I'm okay with it being queued or dropped. Differential Revision: https://phab.mercurial-scm.org/D6486

copies: also encode p[12]copies destination as index into "files" list This is mostly for consistency with the filesaddes/filesremoved fields, but it should also save a bit of space. Differential Revision: https://phab.mercurial-scm.org/D6431

discovery: be more conservative when adjusting the sample size Since 5b34972a0094, the discovery will increase the sample size when it detect a "complex" undecided set. However this detection focussed on the number of roots only, this could regress discovery performance when the undecided set has many roots that eventually get merged into a few heads. To prevent such misbehavior, we adjust the logic to take in account both heads and roots. The sample size will be increased only if both are especially large. Performance testing on the same case as 5b34972a0094, does not show a significant difference.

rust-dirstate: create dirstate submodule This change is here to facilitate a future patch that is written in a different file. I expect this module to grow a few different files. Differential Revision: https://phab.mercurial-scm.org/D6389

profiling: show actual time spent in hotpath display To get, for instance: ... \ 6.6% 4.08s lock.py: __exit__ line 1566: ... | 6.5% 4.01s exchange.py: close line 1191: ... | 6.5% 4.01s transaction.py: _active line 1443: ... | 6.5% 4.01s transaction.py: close line 47: ... | 6.2% 3.84s scmutil.py: wrapped line 529: ... | 6.2% 3.81s localrepo.py: wrapper line 2114: ... | 6.2% 3.81s localrepo.py: updatecaches line 177: ... ... instead of: ... \ 6.6% lock.py: __exit__ line 1566: ... | 6.5% exchange.py: close line 1191: ... | 6.5% transaction.py: _active line 1443: ... | 6.5% transaction.py: close line 47: ... | 6.2% scmutil.py: wrapped line 529: ... | 6.2% localrepo.py: wrapper line 2114: ... | 6.2% localrepo.py: updatecaches line 177: ... ... I find that if it's not displayed, I frequently end up estimating the numbers by hand. Differential Revision: https://phab.mercurial-scm.org/D6477