py3: reinvent print() function for contrib/hgclient.py

py3: work around unicode stdio streams in contrib/hgclient.py

py3: convert string literals to bytes in contrib/hgclient.py # skip-blame just many b'' prefixes

merge with stable

branchmap: remove redundant sort There is absoluty no benefit in sorting a list that's being merged into a set on the next line. The changelog.ancestors() call later on also doesn't benefit from a sorted sequence of revs. Differential Revision: https://phab.mercurial-scm.org/D5111

revset: drop special case of 'revset(...)' function in analyze We now have a valid no-op function. We no longer need the special case.

revset: document the `revset(...)` syntax We introduce a new "no-op" function to bear the documentation. In practice, the parsing step is skipping it so it is not even called. This will get fixed in the next changeset.

check-commit: update test expectation per removal of "double empty line" rule Follow up for 47084b5ffd80.

style: drop requirement to only use single lines between top-level objects Differential Revision: https://phab.mercurial-scm.org/D5105

py3: byteify extension in test-relink.t

f: fix a Python 3 bytes/string issue I suspect we should test this tool in isolation, but we don't yet. Oh well. Differential Revision: https://phab.mercurial-scm.org/D5061

tests: use regex instead of Python versions for archive hash changes It turns out this behavior changed between versions of Python 3. Let's just always accept either size or sha1, and move on. Differential Revision: https://phab.mercurial-scm.org/D5104

notify: a ton of encoding dancing to deal with the email module Almost fixes test-keyword.t on Python 3, but leaves us with some extremely confusing failures at the end of the test that seem related to the command server? Differential Revision: https://phab.mercurial-scm.org/D5100

tests: add missing b prefix in test-context-metadata.t # skip-blame just a b prefix Differential Revision: https://phab.mercurial-scm.org/D5109

context: raise runtime errors with sysstrs We should probably *not* use RuntimeError for this, but let's deal with that later, rather than as part of the Python 3 effort. Differential Revision: https://phab.mercurial-scm.org/D5108

rust: rustfmt config for hg-direct-ffi For now, we're duplicating it, but it would be probably a good idea to use a single one for the whole workspace (would have implications on the other crates as well)

rust: rustlazyancestors.__contains__ This changeset provides a Rust implementation of the iteration performed by lazyancestor.__contains__ It has the advantage over the Python iteration to use the 'seen' set encapsuled into the dedicated iterator (self._containsiter), rather than storing emitted items in another set (self._containsseen), and hence should reduce the memory footprint. Also, there's no need to convert intermediate emitted revisions back into Python integers. At this point, it would be tempting to implement the whole lazyancestor object in Rust, but that would lead to more C wrapping code (two objects) for little expected benefits.

help: fix a missing quote character in ui.tweakdefaults

rust: hooking into Python code We introduce a new class called 'rustlazyancestors' in the ancestors module, which is used only if parsers.rustlazyancestors does exist. The implementation of __contains__ stays unchanged, but is now backed by the Rust iterator. It would probably be a good candidate for further development, though, as it is mostly looping, and duplicates the 'seen' set. The Rust code could be further optimized, however it already gives rise to performance improvements: median timing from hg perfancestors: - on pypy: before: 0.077566s after: 0.016676s -79% - on mozilla central: before: 0.190037s after: 0.082225s -58% - on a private repository (about one million revisions): before: 0.567085s after: 0.108816s -80% - on another private repository (about 400 000 revisions): before: 1.440918s after: 0.290116s -80% median timing for hg perfbranchmap base - on pypy: before: 1.383413s after: 0.507993s -63% - on mozilla central: before: 2.821940s after: 1.258902s -55% - on a private repository (about one million revisions): before: 77.065076s after: 16.158475s -80% - on another private repository (about 401 000 revisions): before: 7.835503s after: 3.545331s -54%

py3: fix test-propertycache.py Differential Revision: https://phab.mercurial-scm.org/D5101

py3: fix test-dirstate-race.t Differential Revision: https://phab.mercurial-scm.org/D5106

help: adding a proper declaration for shortlist/basic commands (API) We previously used the '^' prefix to indicate that a command should be shown on the short list (shown for just "hg"), but that's a horrible hack, so I'm removing it. Differential Revision: https://phab.mercurial-scm.org/D5069

help: assigning topic categories Differential Revision: https://phab.mercurial-scm.org/D5068

help: assigning categories to existing commands I'm separating this into its own commit so people can bikeshed over the actual categorization (vs the support for categories). These categories are based on the help implementation we've been using internally at Google, and have had zero complaints. Differential Revision: https://phab.mercurial-scm.org/D5067

help: splitting the topics by category Differential Revision: https://phab.mercurial-scm.org/D5066

help: adding support for command categories Differential Revision: https://phab.mercurial-scm.org/D5065

notify: just use email.errors email.Errors is a proxy object to email.errors on Python 2.

rust-chg: add struct holding information needed to spawn server process The Locator will handle the initialization of the connection. It will spawn server processes as needed.

rust-chg: install logger if $CHGDEBUG is set This is modeled after the example logger and debugmsg() of chg/util.c. https://docs.rs/log/0.4.5/log/#implementing-a-logger

rust-chg: depend on log and tokio_timer I'll start porting the daemon management functions from chg of C, which will be difficult to debug without some logging facility. AFAIK, the log crate is easy-to-use and widely used. tokio_timer provides sleep() helper to be used while spawning a server process.

rust-chg: suppress panic while writing chg error to stderr Otherwise "chg >/dev/full 2>&1" would exit with 101. Spotted by test-basic.t.

logcmdutil: add a helpful assertion to catch mistyped templates early This would have made a defect in test-notify.t much easier to figure out. Differential Revision: https://phab.mercurial-scm.org/D5097

notify: adapt to new location of email module's errors Differential Revision: https://phab.mercurial-scm.org/D5099

notify: add some b prefixes # skip-blame just b prefixes Differential Revision: https://phab.mercurial-scm.org/D5098

py3: fix test-diff-color.t Differential Revision: https://phab.mercurial-scm.org/D5095

py3: fix test-revlog.t The mpatchError has a trailing comma on Python 2 but not on Python 3, so use a glob to handle both Python 2 and Python 3. Differential Revision: https://phab.mercurial-scm.org/D5093

fuzz: try *even harder* to prevent Python from looking up usernames Differential Revision: https://phab.mercurial-scm.org/D5092

wireproto: fix incorrect function name in docstring The docstring for `iwireprotocolcommandcacher` references an `onoutputfinished` method. The actual name of the function is `onfinished`. Differential Revision: https://phab.mercurial-scm.org/D5090

py3: fix test-status.t Differential Revision: https://phab.mercurial-scm.org/D5089

formatter: make debug output prettier "(glob)" won't be needed since pprintgen() can print dict items in stable order.

stringutil: allow to specify initial indent level of pprint() I want to pprint() an inner object, which starts with level=1 indent.

stringutil: make level parameter of pprintgen() 0-origin I think this makes more sense in that the level is incremented where nesting goes one more deep.

formatter: use stringutil.pprint() in debug output to drop b''

rust: exposing in parsers module To build with the Rust code, set the HGWITHRUSTEXT environment variable. At this point, it's possible to instantiate and use a rustlazyancestors object from a Python interpreter. The changes in setup.py are obviously a quick hack, just good enough to test/bench without much refactoring. We'd be happy to improve on that with help from the community. Rust bindings crate gets compiled as a static library, which in turn gets linked within 'parsers.so' With respect to the plans at https://www.mercurial-scm.org/wiki/OxidationPlan this would probably qualify as "roll our own FFI". Also, it doesn't quite meet the target of getting rid of C code, since it brings actually more, yet: - the new C code does nothing else than parsing arguments and calling Rust functions. In particular, there's no complex allocation involved. - subsequent changes could rewrite more of revlog.c, this time resulting in an overall decrease of C code and unsafety.

rust: iterator bindings to C code In this changeset, still made of Rust code only, we expose the Rust iterator for instantiation and consumption from C code. The idea is that both the index and index_get_parents() will be passed from the C extension, hence avoiding a hard link dependency to parsers.so, so that the crate can still be built and tested independently. On the other hand, parsers.so will use the symbols defined in this changeset.

rust: pure Rust lazyancestors iterator This is the first of a patch series aiming to provide an alternative implementation in the Rust programming language of the _lazyancestorsiter from the ancestor module. This iterator has been brought to our attention by the people at Octobus, as a potential good candidate for incremental "oxydation" (rewriting in Rust), because it has shown performance issues lately and it merely deals with ints (revision numbers) obtained by calling the index, whih should be directly callable from Rust code, being itself implemented as a C extension. The idea behind this series is to provide a minimal example of Rust code collaborating with existing C and Python code. To open the way to gradually rewriting more of Mercurial's Python code in Rust, without being forced to pay a large initial cost of rewriting the existing fast core into Rust. This patch does not introduce any bindings to other Mercurial code yet. Instead, it introduces the necessary abstractions to address the problem independently, and unit-test it. Since this is the first use of Rust as a Python module within Mercurial, the hg-core crate gets created within this patch. See its Cargo.toml for more details. Someone with a rustc/cargo installation may chdir into rust/hg-core and run the tests by issuing: cargo test --lib The algorithm is a bit simplified (see details in docstrings), and at its simplest becomes rather trivial, showcasing that Rust has batteries included too: BinaryHeap, the Rust analog of Python's heapq does actually all the work. The implementation can be further optimized and probably be made more idiomatic Rust.

run-tests: restore quoting the python executable for running *.py tests This was accidentally dropped in 8cf459d8b111.

tests: replace `cd ..` with an absolute path in a couple ssh tests These tests are broken under py3 on Windows to the point where the `cd ..` was actually escaping into the system wide $TEMP. The subsequent `hg init` created a repo there, and then added a local extension to the hgrc. This breaks every single subsequent test when it tries to `hg init` in its $TESTTMP, and can't load the localwrite.py extension. And since I botched this the first time and replaced the wrong `cd ..`, this just replaces all of them. I've noticed test garbage in $TEMP recently, and maybe this will help. Perhaps `hg init` shouldn't load the config for the local repo, but this is an easy enough workaround for now.

lfs: register the flag processors per repository Previously, enabling the extension for any repo in commandserver or hgweb would enable the flags on all repos. Since localrepo.resolverevlogstorevfsoptions() is called so early, the check to see if the extension is enabled on the repo (which hasn't been instantiated yet) is a bit awkward. But I don't see a better way.

revlog: allow flag processors to be applied via store options This allows flag processors to be registered to specific repos in an extension by wrapping localrepo.resolverevlogstorevfsoptions(). I wanted to add the processors via a function on localrepo, but some of the places where the processors are globally registered don't have a repository available. This makes targeting specific repos in the wrapper awkward, but still manageable.

py3: use str to query registry values on Windows This blew up launching any command if extdiff processed a tool with a regkey config.

py3: convert "usage" literal to bytes Here _() is practically an identity function, but we shouldn't pass in unicode to _().

churn: remove redundant round() To my surprise, the int() is required. Spotted by Mads when he reviewed D5063. Thanks! Differential Revision: https://phab.mercurial-scm.org/D5086

py3: use py3 as the test tag, dropping the k Differential Revision: https://phab.mercurial-scm.org/D5079

tests: fix inline extension in test-fncache.t for Python 3 # skip-blame just some b prefixing Differential Revision: https://phab.mercurial-scm.org/D5083

py3: 3 more passing tests Differential Revision: https://phab.mercurial-scm.org/D5082

simplemerge: port to Python 3 Differential Revision: https://phab.mercurial-scm.org/D5081

contrib: fix up output in check-config.py to use strs to avoid b prefixes Differential Revision: https://phab.mercurial-scm.org/D5059

context: open files in bytes mode I'm stunned this open() call has survived this long without the b in the mode - it seems like it should have been a source of bugs somewhere... Differential Revision: https://phab.mercurial-scm.org/D5080

tests: fix up test-hghave for recent run-tests change to use more CPUs Differential Revision: https://phab.mercurial-scm.org/D5085