delta: exclude base candidate much smaller than the target If a revision's full text is that much bigger than a base candidate full text, we no longer consider that candidate. This solves a pathological case we encountered on a very specify repository. It contains a long series of changesets with a very small manifest (one file) co-existing with others changesets using a very large manifest. Without this filtering, we ended up considering a large number of tiny full snapshots as a potential base. It resulted in very large delta (the size of the full text) and mercurial spending 99% of its time compressing these deltas. The timing of a commit moved from about 400s to about 10s (still slow, but not ridiculously slow).

perfrevflogwrite: clear revlog cache between each write We want to measure write time from a cold cache (similar to commit). So we need to clear the cache to prevent computation from rev N-1 to interfere with rev N.

py3: enable legacy stdio mode in exewrapper This drops the test failure count from 166 to 117. The failures were typically in the form of `hg serve -d` spawning but crashing immediately, leaving clients with "bad http status" lines, connection refusals, and so forth. The underlying message on the server side was either "OSError: [WinError 6] The handle is invalid" or "OSError: [WinError 1] Incorrect function". Additionally, no output was rendered if the pager was activated. Thanks to Yuya for diagnosing the problem. The failure count drops to 107 when PYTHONLEGACYWINDOWSSTDIO=1 is defined in the environment. These failures seem to revolve around the dummyssh server process, and dumbhttp.py. So I'll probably add that to the test runner. One small regression here (only in py3) is that if hg.exe is already built, a messagebox appears when building it again saying that python37.dll can't be loaded. Python3 isn't in PATH by default, and setup.py tries running bare `hg` first. But MSYS prepends '.' to PATH, so it runs the local hg, but can't find the library. It falls back to the python used to invoke setup.py, so ultimately it works. I'm not sure if it's better to strip '.' from PATH or just skip right to `sys.executable hg` on Windows. Also, something seems to be wrong with run-tests._usecorrectpython(). I accidentially left off the 'PYTHON="py -3"' when building (thus making py2 stuff), and yet managed to invoke run-tests.py with "py -3". (And that only had 67 failures.)

run-tests: alias hg to hg.exe on Windows To enable legacy stdio mode on Windows, hg.exe needs to be updated. But before that, we actually have to use it when running the tests. I *think* what was happening before was when MSYS invoked `hg`, it looked at the shbang line and ran python.exe found there. The test harness must be updating $PATH to include the python used to launch it, and therefore it ran py3. As a side note, this also fixed `py -3 run-tests.py` (without --local), which complained about the space in the shbang line before this. This should also help in WSL, because the explicit '.exe' is needed to invoke a Windows app instead of invoking the Linux app. I have no idea why this change capitalizes Lib in the tests, as it was previously lowercase for both py2 and py3.

py3: conditionalize the python version in test-install.t

tests: followup on network related errors on Debian 9 First, update test-clonebundles.t which was failing since the introduction of the `$EADDRNOTAVAIL$` common pattern. Also update two tests from 8695fbe17f7c with a more compact condition line. Differential Revision: https://phab.mercurial-scm.org/D5445

py3: unimplement RevlogError.__str__() On Python 2, str(exc) would crash if __str__() returned a unicode containing non-ASCII characters.

py3: use forcebytestr() to stringify hook exception This fixes test-narrow-exchange.t.

color: fix a documentation typo

py3: quote $PYTHON in test-patchbomb.t for Windows I couldn't get the quoting right in the environment variable, so now it's a function.

rust-cpython: testing the bindings from Python This is easier and more convincing than doing the same tests from a Rust tests module. Differential Revision: https://phab.mercurial-scm.org/D5437

rust-cpython: build via HGWITHRUSTEXT=cpython The existing behaviour, building the direct ffi bindings if HGIWTHRUSTEXT is just set is unchanged, but if HGWITHRUSTEXT is cpython, then the cpython bindings (aka mercurial/rustext.so) are built. Differential Revision: https://phab.mercurial-scm.org/D5436

rust: better treatment of cargo/rustc errors Differential Revision: https://phab.mercurial-scm.org/D5435

rust-cpython: start cpython crate bindings This changeset introduces the hg-cpython crate, that compiles as a shared library holding a whole Python package (mercurial.rustext), with only the empty 'ancestor' submodule for now. Such bindings will be easier and safer to develop and maintain that those of `hg-direct-ffi`. They don't involve C code, only unsafe Rust that's mostly isolated within the cpython crate. The long-term goal would be to import the provided modules, such as rustext.ancestor with mercurial.policy.importmod, same as we already do with cext modules. Differential Revision: https://phab.mercurial-scm.org/D5434

rust-cpython: exclude hgcli from workspace hgcli uses a specific rust-cpython commit by indygreg, of which a PR has been derived which is not merged nor released yet. But we can't use several versions of the sys-python2.7 crate in a single workspace: it makes for a build error. Since hgcli does not at the time being need anything from hg-core, whereas the upcoming hg-cpython will. So for now we're moving hgcli aside, hoping we could base all of them on the same version of rust-cpython again in the future. Differential Revision: https://phab.mercurial-scm.org/D5433

sparse-revlog: protect C code against delta chain including nullrev For unclear reasons, some repositories include nullrev (-1). Re-computing delta for such repo remove nullrev from all chain, so some older versions have been creating them. This currently raise an IndexError with the new C code doing chain slicing as it expect all item to be positive. Both python and C code for reading delta chain preserve nullrev, and the Python code for chain slicing handle the case fine. So we take the safe route and make the new C code works fine in that case.

sparse-revlog: handle nullrev in index_get_length The more generic index_get method handle nullrev fine, we apply the same logic here.

sparse-revlog: handle nullrev in index_get_start The more generic index_get method handle nullrev fine, we apply the same logic here.

revlog: introduce a constant for nullrev in `revlog.c` The value is important enough to be explicitly tracked.

rust: translation of missingancestors This is as direct as possible a translation of the ancestor.missingancestors Python class in pure Rust. The goal for this changeset is to make it easy to compare with the Python version. We also add to Python tests the cases that helped us develop and debug this implementation. Some possible optimizations are marked along the way as TODO comments Differential Revision: https://phab.mercurial-scm.org/D5416

contrib: provide a small script that draw performance plot We have been using this script to look into the result of various runs of the `hg perfrevlogwrite` command. It seems useful enough to be shared more widely.

delta: filter nullrev out first When picking a potential candidate, we filter them on various criteria. The "different from nullrev" criteria is very fast to compute and we should process it first.

perf: report more of the higher range in perfrevlogwrite Since the delta chain length is limited to 1000 revisions, we get a new snapshot about every 1000 revisions. If we assume that the snapshot will be most of the slowest revision, the current display (99% and max) are not very precise in their area. We now include more information about this space in the default report.

help: present boolean arguments as "--[no-]foo" This should make it much more discoverable (we document it in `hg help flags`, but most users don't think to look there). Note that flags that default to None (and not False) will not get this new presentation. We can change the defaults to False later for flags where it makes sense (probably almost all boolean flags). Differential Revision: https://phab.mercurial-scm.org/D5432

help: show "default: off" for boolean flags that default to off Differential Revision: https://phab.mercurial-scm.org/D5431

help: use "default: on" instead of "default: True" "True" feels like a Python thing and not something that users should see. Differential Revision: https://phab.mercurial-scm.org/D5430

check-commit: disallow capitalization only right after topic It should be okay to write "topic: change 'default: True' to 'default: on'". Differential Revision: https://phab.mercurial-scm.org/D5429

absorb: don't prompt to apply changes when there are none to apply Differential Revision: https://phab.mercurial-scm.org/D5428

py3: add a b'' prefix in tests/test-remotefilelog-pull-noshallow.t This makes the test pass on Python 3. # skip-blame because just b'' prefix. Differential Revision: https://phab.mercurial-scm.org/D5427

error: implement __str__ on RevlogError to fix some output defects on Py3 We open-code encoding.unimethod here to avoid cycles, and do a local import of encoding when someone str()s a RevlogError. It's not my favorite solution, but it gets the job done. Differential Revision: https://phab.mercurial-scm.org/D5426