rust: hooking into Python code We introduce a new class called 'rustlazyancestors' in the ancestors module, which is used only if parsers.rustlazyancestors does exist. The implementation of __contains__ stays unchanged, but is now backed by the Rust iterator. It would probably be a good candidate for further development, though, as it is mostly looping, and duplicates the 'seen' set. The Rust code could be further optimized, however it already gives rise to performance improvements: median timing from hg perfancestors: - on pypy: before: 0.077566s after: 0.016676s -79% - on mozilla central: before: 0.190037s after: 0.082225s -58% - on a private repository (about one million revisions): before: 0.567085s after: 0.108816s -80% - on another private repository (about 400 000 revisions): before: 1.440918s after: 0.290116s -80% median timing for hg perfbranchmap base - on pypy: before: 1.383413s after: 0.507993s -63% - on mozilla central: before: 2.821940s after: 1.258902s -55% - on a private repository (about one million revisions): before: 77.065076s after: 16.158475s -80% - on another private repository (about 401 000 revisions): before: 7.835503s after: 3.545331s -54%

py3: fix test-propertycache.py Differential Revision: https://phab.mercurial-scm.org/D5101

py3: fix test-dirstate-race.t Differential Revision: https://phab.mercurial-scm.org/D5106

help: adding a proper declaration for shortlist/basic commands (API) We previously used the '^' prefix to indicate that a command should be shown on the short list (shown for just "hg"), but that's a horrible hack, so I'm removing it. Differential Revision: https://phab.mercurial-scm.org/D5069

help: assigning topic categories Differential Revision: https://phab.mercurial-scm.org/D5068

help: assigning categories to existing commands I'm separating this into its own commit so people can bikeshed over the actual categorization (vs the support for categories). These categories are based on the help implementation we've been using internally at Google, and have had zero complaints. Differential Revision: https://phab.mercurial-scm.org/D5067

help: splitting the topics by category Differential Revision: https://phab.mercurial-scm.org/D5066

help: adding support for command categories Differential Revision: https://phab.mercurial-scm.org/D5065

notify: just use email.errors email.Errors is a proxy object to email.errors on Python 2.

rust-chg: add struct holding information needed to spawn server process The Locator will handle the initialization of the connection. It will spawn server processes as needed.

rust-chg: install logger if $CHGDEBUG is set This is modeled after the example logger and debugmsg() of chg/util.c. https://docs.rs/log/0.4.5/log/#implementing-a-logger

rust-chg: depend on log and tokio_timer I'll start porting the daemon management functions from chg of C, which will be difficult to debug without some logging facility. AFAIK, the log crate is easy-to-use and widely used. tokio_timer provides sleep() helper to be used while spawning a server process.

rust-chg: suppress panic while writing chg error to stderr Otherwise "chg >/dev/full 2>&1" would exit with 101. Spotted by test-basic.t.

logcmdutil: add a helpful assertion to catch mistyped templates early This would have made a defect in test-notify.t much easier to figure out. Differential Revision: https://phab.mercurial-scm.org/D5097

notify: adapt to new location of email module's errors Differential Revision: https://phab.mercurial-scm.org/D5099

notify: add some b prefixes # skip-blame just b prefixes Differential Revision: https://phab.mercurial-scm.org/D5098

py3: fix test-diff-color.t Differential Revision: https://phab.mercurial-scm.org/D5095

py3: fix test-revlog.t The mpatchError has a trailing comma on Python 2 but not on Python 3, so use a glob to handle both Python 2 and Python 3. Differential Revision: https://phab.mercurial-scm.org/D5093

fuzz: try *even harder* to prevent Python from looking up usernames Differential Revision: https://phab.mercurial-scm.org/D5092

wireproto: fix incorrect function name in docstring The docstring for `iwireprotocolcommandcacher` references an `onoutputfinished` method. The actual name of the function is `onfinished`. Differential Revision: https://phab.mercurial-scm.org/D5090

py3: fix test-status.t Differential Revision: https://phab.mercurial-scm.org/D5089

formatter: make debug output prettier "(glob)" won't be needed since pprintgen() can print dict items in stable order.

stringutil: allow to specify initial indent level of pprint() I want to pprint() an inner object, which starts with level=1 indent.

stringutil: make level parameter of pprintgen() 0-origin I think this makes more sense in that the level is incremented where nesting goes one more deep.

formatter: use stringutil.pprint() in debug output to drop b''

rust: exposing in parsers module To build with the Rust code, set the HGWITHRUSTEXT environment variable. At this point, it's possible to instantiate and use a rustlazyancestors object from a Python interpreter. The changes in setup.py are obviously a quick hack, just good enough to test/bench without much refactoring. We'd be happy to improve on that with help from the community. Rust bindings crate gets compiled as a static library, which in turn gets linked within 'parsers.so' With respect to the plans at https://www.mercurial-scm.org/wiki/OxidationPlan this would probably qualify as "roll our own FFI". Also, it doesn't quite meet the target of getting rid of C code, since it brings actually more, yet: - the new C code does nothing else than parsing arguments and calling Rust functions. In particular, there's no complex allocation involved. - subsequent changes could rewrite more of revlog.c, this time resulting in an overall decrease of C code and unsafety.

rust: iterator bindings to C code In this changeset, still made of Rust code only, we expose the Rust iterator for instantiation and consumption from C code. The idea is that both the index and index_get_parents() will be passed from the C extension, hence avoiding a hard link dependency to parsers.so, so that the crate can still be built and tested independently. On the other hand, parsers.so will use the symbols defined in this changeset.

rust: pure Rust lazyancestors iterator This is the first of a patch series aiming to provide an alternative implementation in the Rust programming language of the _lazyancestorsiter from the ancestor module. This iterator has been brought to our attention by the people at Octobus, as a potential good candidate for incremental "oxydation" (rewriting in Rust), because it has shown performance issues lately and it merely deals with ints (revision numbers) obtained by calling the index, whih should be directly callable from Rust code, being itself implemented as a C extension. The idea behind this series is to provide a minimal example of Rust code collaborating with existing C and Python code. To open the way to gradually rewriting more of Mercurial's Python code in Rust, without being forced to pay a large initial cost of rewriting the existing fast core into Rust. This patch does not introduce any bindings to other Mercurial code yet. Instead, it introduces the necessary abstractions to address the problem independently, and unit-test it. Since this is the first use of Rust as a Python module within Mercurial, the hg-core crate gets created within this patch. See its Cargo.toml for more details. Someone with a rustc/cargo installation may chdir into rust/hg-core and run the tests by issuing: cargo test --lib The algorithm is a bit simplified (see details in docstrings), and at its simplest becomes rather trivial, showcasing that Rust has batteries included too: BinaryHeap, the Rust analog of Python's heapq does actually all the work. The implementation can be further optimized and probably be made more idiomatic Rust.

run-tests: restore quoting the python executable for running *.py tests This was accidentally dropped in 8cf459d8b111.

tests: replace `cd ..` with an absolute path in a couple ssh tests These tests are broken under py3 on Windows to the point where the `cd ..` was actually escaping into the system wide $TEMP. The subsequent `hg init` created a repo there, and then added a local extension to the hgrc. This breaks every single subsequent test when it tries to `hg init` in its $TESTTMP, and can't load the localwrite.py extension. And since I botched this the first time and replaced the wrong `cd ..`, this just replaces all of them. I've noticed test garbage in $TEMP recently, and maybe this will help. Perhaps `hg init` shouldn't load the config for the local repo, but this is an easy enough workaround for now.

lfs: register the flag processors per repository Previously, enabling the extension for any repo in commandserver or hgweb would enable the flags on all repos. Since localrepo.resolverevlogstorevfsoptions() is called so early, the check to see if the extension is enabled on the repo (which hasn't been instantiated yet) is a bit awkward. But I don't see a better way.

revlog: allow flag processors to be applied via store options This allows flag processors to be registered to specific repos in an extension by wrapping localrepo.resolverevlogstorevfsoptions(). I wanted to add the processors via a function on localrepo, but some of the places where the processors are globally registered don't have a repository available. This makes targeting specific repos in the wrapper awkward, but still manageable.