py3: fix test-dirstate-race.t Differential Revision: https://phab.mercurial-scm.org/D5106

help: adding a proper declaration for shortlist/basic commands (API) We previously used the '^' prefix to indicate that a command should be shown on the short list (shown for just "hg"), but that's a horrible hack, so I'm removing it. Differential Revision: https://phab.mercurial-scm.org/D5069

help: assigning topic categories Differential Revision: https://phab.mercurial-scm.org/D5068

help: assigning categories to existing commands I'm separating this into its own commit so people can bikeshed over the actual categorization (vs the support for categories). These categories are based on the help implementation we've been using internally at Google, and have had zero complaints. Differential Revision: https://phab.mercurial-scm.org/D5067

help: splitting the topics by category Differential Revision: https://phab.mercurial-scm.org/D5066

help: adding support for command categories Differential Revision: https://phab.mercurial-scm.org/D5065

notify: just use email.errors email.Errors is a proxy object to email.errors on Python 2.

rust-chg: add struct holding information needed to spawn server process The Locator will handle the initialization of the connection. It will spawn server processes as needed.

rust-chg: install logger if $CHGDEBUG is set This is modeled after the example logger and debugmsg() of chg/util.c. https://docs.rs/log/0.4.5/log/#implementing-a-logger

rust-chg: depend on log and tokio_timer I'll start porting the daemon management functions from chg of C, which will be difficult to debug without some logging facility. AFAIK, the log crate is easy-to-use and widely used. tokio_timer provides sleep() helper to be used while spawning a server process.

rust-chg: suppress panic while writing chg error to stderr Otherwise "chg >/dev/full 2>&1" would exit with 101. Spotted by test-basic.t.

logcmdutil: add a helpful assertion to catch mistyped templates early This would have made a defect in test-notify.t much easier to figure out. Differential Revision: https://phab.mercurial-scm.org/D5097

notify: adapt to new location of email module's errors Differential Revision: https://phab.mercurial-scm.org/D5099

notify: add some b prefixes # skip-blame just b prefixes Differential Revision: https://phab.mercurial-scm.org/D5098

py3: fix test-diff-color.t Differential Revision: https://phab.mercurial-scm.org/D5095

py3: fix test-revlog.t The mpatchError has a trailing comma on Python 2 but not on Python 3, so use a glob to handle both Python 2 and Python 3. Differential Revision: https://phab.mercurial-scm.org/D5093

fuzz: try *even harder* to prevent Python from looking up usernames Differential Revision: https://phab.mercurial-scm.org/D5092

wireproto: fix incorrect function name in docstring The docstring for `iwireprotocolcommandcacher` references an `onoutputfinished` method. The actual name of the function is `onfinished`. Differential Revision: https://phab.mercurial-scm.org/D5090

py3: fix test-status.t Differential Revision: https://phab.mercurial-scm.org/D5089

formatter: make debug output prettier "(glob)" won't be needed since pprintgen() can print dict items in stable order.

stringutil: allow to specify initial indent level of pprint() I want to pprint() an inner object, which starts with level=1 indent.

stringutil: make level parameter of pprintgen() 0-origin I think this makes more sense in that the level is incremented where nesting goes one more deep.

formatter: use stringutil.pprint() in debug output to drop b''

rust: exposing in parsers module To build with the Rust code, set the HGWITHRUSTEXT environment variable. At this point, it's possible to instantiate and use a rustlazyancestors object from a Python interpreter. The changes in setup.py are obviously a quick hack, just good enough to test/bench without much refactoring. We'd be happy to improve on that with help from the community. Rust bindings crate gets compiled as a static library, which in turn gets linked within 'parsers.so' With respect to the plans at https://www.mercurial-scm.org/wiki/OxidationPlan this would probably qualify as "roll our own FFI". Also, it doesn't quite meet the target of getting rid of C code, since it brings actually more, yet: - the new C code does nothing else than parsing arguments and calling Rust functions. In particular, there's no complex allocation involved. - subsequent changes could rewrite more of revlog.c, this time resulting in an overall decrease of C code and unsafety.

rust: iterator bindings to C code In this changeset, still made of Rust code only, we expose the Rust iterator for instantiation and consumption from C code. The idea is that both the index and index_get_parents() will be passed from the C extension, hence avoiding a hard link dependency to parsers.so, so that the crate can still be built and tested independently. On the other hand, parsers.so will use the symbols defined in this changeset.

rust: pure Rust lazyancestors iterator This is the first of a patch series aiming to provide an alternative implementation in the Rust programming language of the _lazyancestorsiter from the ancestor module. This iterator has been brought to our attention by the people at Octobus, as a potential good candidate for incremental "oxydation" (rewriting in Rust), because it has shown performance issues lately and it merely deals with ints (revision numbers) obtained by calling the index, whih should be directly callable from Rust code, being itself implemented as a C extension. The idea behind this series is to provide a minimal example of Rust code collaborating with existing C and Python code. To open the way to gradually rewriting more of Mercurial's Python code in Rust, without being forced to pay a large initial cost of rewriting the existing fast core into Rust. This patch does not introduce any bindings to other Mercurial code yet. Instead, it introduces the necessary abstractions to address the problem independently, and unit-test it. Since this is the first use of Rust as a Python module within Mercurial, the hg-core crate gets created within this patch. See its Cargo.toml for more details. Someone with a rustc/cargo installation may chdir into rust/hg-core and run the tests by issuing: cargo test --lib The algorithm is a bit simplified (see details in docstrings), and at its simplest becomes rather trivial, showcasing that Rust has batteries included too: BinaryHeap, the Rust analog of Python's heapq does actually all the work. The implementation can be further optimized and probably be made more idiomatic Rust.

run-tests: restore quoting the python executable for running *.py tests This was accidentally dropped in 8cf459d8b111.

tests: replace `cd ..` with an absolute path in a couple ssh tests These tests are broken under py3 on Windows to the point where the `cd ..` was actually escaping into the system wide $TEMP. The subsequent `hg init` created a repo there, and then added a local extension to the hgrc. This breaks every single subsequent test when it tries to `hg init` in its $TESTTMP, and can't load the localwrite.py extension. And since I botched this the first time and replaced the wrong `cd ..`, this just replaces all of them. I've noticed test garbage in $TEMP recently, and maybe this will help. Perhaps `hg init` shouldn't load the config for the local repo, but this is an easy enough workaround for now.

lfs: register the flag processors per repository Previously, enabling the extension for any repo in commandserver or hgweb would enable the flags on all repos. Since localrepo.resolverevlogstorevfsoptions() is called so early, the check to see if the extension is enabled on the repo (which hasn't been instantiated yet) is a bit awkward. But I don't see a better way.

revlog: allow flag processors to be applied via store options This allows flag processors to be registered to specific repos in an extension by wrapping localrepo.resolverevlogstorevfsoptions(). I wanted to add the processors via a function on localrepo, but some of the places where the processors are globally registered don't have a repository available. This makes targeting specific repos in the wrapper awkward, but still manageable.

py3: use str to query registry values on Windows This blew up launching any command if extdiff processed a tool with a regkey config.

py3: convert "usage" literal to bytes Here _() is practically an identity function, but we shouldn't pass in unicode to _().

churn: remove redundant round() To my surprise, the int() is required. Spotted by Mads when he reviewed D5063. Thanks! Differential Revision: https://phab.mercurial-scm.org/D5086

py3: use py3 as the test tag, dropping the k Differential Revision: https://phab.mercurial-scm.org/D5079

tests: fix inline extension in test-fncache.t for Python 3 # skip-blame just some b prefixing Differential Revision: https://phab.mercurial-scm.org/D5083

py3: 3 more passing tests Differential Revision: https://phab.mercurial-scm.org/D5082

simplemerge: port to Python 3 Differential Revision: https://phab.mercurial-scm.org/D5081

contrib: fix up output in check-config.py to use strs to avoid b prefixes Differential Revision: https://phab.mercurial-scm.org/D5059

context: open files in bytes mode I'm stunned this open() call has survived this long without the b in the mode - it seems like it should have been a source of bugs somewhere... Differential Revision: https://phab.mercurial-scm.org/D5080

tests: fix up test-hghave for recent run-tests change to use more CPUs Differential Revision: https://phab.mercurial-scm.org/D5085

py3: fix test-parse-date.t Differential Revision: https://phab.mercurial-scm.org/D5084

obsolete: don't translate internal error message AFAIK, it's caught only by "hg debugobsolete", so it's pretty much a programming error.

py3: get around unicode docstrings in test-encoding-textwrap.t and test-help.t On Python 3, docstrings are converted back to utf-8 bytes, which practically disables the "if type(message) is pycompat.unicode" hack in gettext(). Let's add one more workaround for the Py3 path.

crecord: make enter move cursor down to the next item of the same type Let's replace experimental.spacemovesdown with a separate key: Enter, since it wasn't used for anything in crecord. Not sure if '\n' works on Windows though. nextsametype() strictly only moves to items of the same type as the current item. This, for example, allows to go over individual lines in a diff and skip hunk and file headers (which would toggle multiple lines).

py3: fix infinitepush extension tests Differential Revision: https://phab.mercurial-scm.org/D5078

py3: build help of compression engines in bytes Removes "b''" from help.

py3: do I/O in bytes in test-help.t

tests: accept slightly different zip file in Python 3 I added some `unzip -t` here and I *think* the only change is from Python 3 having more data in the zip file headers or something. Sigh. Differential Revision: https://phab.mercurial-scm.org/D5075

webcommands: use stringutil.pprint() to repr invalid archive types Differential Revision: https://phab.mercurial-scm.org/D5074

archival: don't try and fsdecode non-{bytes,str} objects This function accepts both bytes and file-like objects. Differential Revision: https://phab.mercurial-scm.org/D5073

tests: fix last failure in test-tools.t Differential Revision: https://phab.mercurial-scm.org/D5072

run-tests: run tests with as many processes as cores by default This seems like a useful default behavior so tests run faster by default* * Except in special circumstances where the OS/filesystem doesn't scale well to many CPU cores (like APFS *cough* *cough*). Differential Revision: https://phab.mercurial-scm.org/D5071

run-tests: print number of tests and parallel process count This seems like a useful output message to have. I also sneak in a change to lower the parallel process count if it is larger than the number of tests, as that makes no sense and output saying we're running more tests in parallel than there exists tests would be wonky. Differential Revision: https://phab.mercurial-scm.org/D5070

releasenotes: use stringutil.wrap() instead of handcrafted TextWrapper wrapper It's silly to splitlines() a joined string, but we don't care the performance here.

match: optimize matcher when all patterns are of rootfilesin kind Internally at Google, we use narrowspecs with only rootfilesin-kind patterns. Sometimes there are thousands of such patterns (i.e. thousands of tracked directories). In such cases, it can take quite long to build and evaluate the resulting matcher. This patch optimizes matchers that have only patterns of rootfilesin so it instead of creating a regular expression, it matches the given file's directory against the set of directories. In a repo with ~3600 tracked directories, it takes about 1.35 s to build the matcher and 2.7 s to walk the dirstate before this patch. After, it takes 0.04 s to create the matcher and 0.87 s to walk the dirstate. It may be worthwhile to do similar optimizations for e.g. patterns of type "kind:", but that's not a priority for us right now. Differential Revision: https://phab.mercurial-scm.org/D5058

churn: use integer division consistently This results in slight output changes, but it's at least consistent between Python 2 and 3. Since the output is just bar graphs anyway, I'm content with the changes. Differential Revision: https://phab.mercurial-scm.org/D5063

churn: fix stack traces on Python 3 Differential Revision: https://phab.mercurial-scm.org/D5062

py3: moar passing tests Differential Revision: https://phab.mercurial-scm.org/D5055

py3: one new passing tests Differential Revision: https://phab.mercurial-scm.org/D5006

tests: expand our coverage of errors in Python 3 for bad extensions What a mess. Differential Revision: https://phab.mercurial-scm.org/D5005