ci: Add a job testing with rhg installed as `hg` This significantly increases test coverage of rhg, without duplicating many tests that already exist. The `cargo build` command being remove only compiled a `rhg` executable (as shown by the preceding `cd` command) but since a previous patch `run-tests.py --rhg` now takes care of that. Differential Revision: https://phab.mercurial-scm.org/D10196

run-test: install rhg if --rhg is passed Before this, --rhg was only working with --local. Differential Revision: https://phab.mercurial-scm.org/D10195

makefile: add a install option This gives and easy way to install rhg that we can use in `run-test.py` in the next changesets. Differential Revision: https://phab.mercurial-scm.org/D10194

makefile: add a build-rhg option This gives an easy action to build the rhg-binary. This will be useful for the `install-rhg` action in the next changeset. Differential Revision: https://phab.mercurial-scm.org/D10193

makefile: add a install-chg option This is done as a gratuitous improvement on the way to add makefile entry to build and install rhg. It seems saner to have equivalent entry for chg too. Differential Revision: https://phab.mercurial-scm.org/D10192

makefile: add a build-chg option This is done as a gratuitous improvement on the way to add makefile entry to build and install rhg. It seems saner to have equivalent entry for chg too. Differential Revision: https://phab.mercurial-scm.org/D10191

tests: Disable for rhg remaining tests that fail in that mode These cases are in features not yet implemented by rhg for which triggering a fallback is not practical. Disabling some tests allows us to reach passing CI and catch any future regression in the rest of the tests. Differential Revision: https://phab.mercurial-scm.org/D10190

rhg: Remove `rhg.fallback-executable=hg` default configuration When `rhg.on-unsupported` is configured to `fallback` and an unsupported feature is encountered, the previous default was to look for an `hg` executable in `$PATH`. This default was fragile since it was easy to end up accidentally using an older version of Mercurial installed system-wide, when a local (perhaps patched) installation was intended. Instead, it is now an error to have `rhg.on-unsupported=fallback` without also configuring an explicit path or the fallback executable. Differential Revision: https://phab.mercurial-scm.org/D10189

rhg: Return an error code for `rhg config Section.idontexist` This is what Python-based hg does. Differential Revision: https://phab.mercurial-scm.org/D10145

rhg: Fall back to Python for --version Clap has some built-in support for printing something on --version, but it looks different than what Pytho-based hg does. Also, at the moment we’re not giving version numbers to rhg separate from the Mercurial version. Differential Revision: https://phab.mercurial-scm.org/D10144

rhg: Exit with an error code if `files` finds nothing This matches the behavior of Python-based hg. Differential Revision: https://phab.mercurial-scm.org/D10143

rhg: `cat` command: print error messages for missing files And exit with an error code if no file was matched. This matches the behavior of Python-based hg. Differential Revision: https://phab.mercurial-scm.org/D10142

rhg: Silently ignore missing files in config %include … instead of aborting with an error message. This is what Python-based hg does. Differential Revision: https://phab.mercurial-scm.org/D10141

rhg: Add support for environment variables in config include paths Some tests rely on this. Differential Revision: https://phab.mercurial-scm.org/D10140

rhg: Add support for the HGRCSKIPREPO environment variable It’s easy enough and affects tests, well, that test that feature. Differential Revision: https://phab.mercurial-scm.org/D10139

rhg: Don’t make repository path absolute too early Some error messages want to include a relative path, which affects the output of some tests. Differential Revision: https://phab.mercurial-scm.org/D10138

rhg: Fall back to Python if ui.relative-paths is configured This feature is not supported yet, and affects the output of some tests. Differential Revision: https://phab.mercurial-scm.org/D10137

tests: clarify some missing output in test-merge-subrepos This makes the test behavior clearer, especially, why the output are inconsistent. Differential Revision: https://phab.mercurial-scm.org/D10136

tests: Adapt expected output for minor differences with rhg Differential Revision: https://phab.mercurial-scm.org/D10135

rhg: Add support for --cwd This affect the meaning of relative paths in `--repository`, which are resolved "early" by rhg in order to load config which is needed before fallback to Python is considered. An incorrect path could cause errors when loading a non-existent repo, leading to failing tests even when fallback is enabled. Differential Revision: https://phab.mercurial-scm.org/D10134

rhg: Align with Python on some more error messages Differences can cause some tests to fail Differential Revision: https://phab.mercurial-scm.org/D10133

rhg: Add an allow-list of ignored extensions Because rhg doesn’t know how a Python extension would affect behavior it implements in Rust, when an unsupported extension is enabled it conservatively falls back to Python-based hg. However many users will have unsupported extensions enabled in practice. Maybe they don’t actually affect rhg behavior, but we don’t know. This adds a `rhg.ignored-extensions` configuration that lets users list extensions that rhg can safely ignore and proceed even if they’re not supported in Rust. Differential Revision: https://phab.mercurial-scm.org/D10188

rhg: Fall back to Python if unsupported extensions are enabled Extensions might affect behavior in ways we can’t anticipate, so just ignoring them is not correct. Later we’ll add opt-in configuration to ignore specific extensions. Differential Revision: https://phab.mercurial-scm.org/D10112

rhg: Sort config files when adding a directory For example in `/etc/mercurial/hgrc.d/` or with `HGRCPATH=some-directory`. Previously files where parsed in the order returned by the filesystem, which is undefined. But order is significant when multiple files define the same configuration key: the "last" one wins. Differential Revision: https://phab.mercurial-scm.org/D10111

rhg: Align config file parse error formatting with Python Differences can cause tests to fail Differential Revision: https://phab.mercurial-scm.org/D10110

rhg: Fall back to Python for bundle repositories rhg does not support bundles at all, yet. Differential Revision: https://phab.mercurial-scm.org/D10102

rhg: Fall back to Python on --repository with an URL A low-hanging fruit to improve on this would be to properly parse and handle `file:` URLs. But other Python-based hg supports some other URL schemes for features that rhg does not support yet. Differential Revision: https://phab.mercurial-scm.org/D10101

rhg: Print non-absolutized path in "repository {} not found" errors … like Python does. Differences in ouput can cause tests to fail. Differential Revision: https://phab.mercurial-scm.org/D10100

rhg: Align "malformed --config" error message with Python Differences in error message formatting can cause tests to fail. Differential Revision: https://phab.mercurial-scm.org/D10099

rhg: Fall back to Python on unsupported `rhg config <section>` Printing an entire section (as opposed to a single value with `rhg config foo.bar`) is not supported yet in Rust only. Differential Revision: https://phab.mercurial-scm.org/D10098

rhg: Fall back to Python for unsupported revset syntax rhg only supports a small subset of the syntax. On parse error, this gives Python a chance instead of aborting immediately. Differential Revision: https://phab.mercurial-scm.org/D10097

tests: Add `rhg` and `no-rhg` for #require and #if in .t files With this we can allow some small differences in expected output between Rust-based and Python-based code paths. Differential Revision: https://phab.mercurial-scm.org/D10096

tests: Enable rhg fallback to Python by default in tests This premise of `run-tests.py --rhg`: fallback should make `rhg` behave the same as `hg`, except faster in some cases. To test run the whole test suite with installed `rhg` as `hg` and with fallback enabled. Differential Revision: https://phab.mercurial-scm.org/D10187

rhg: Add environment variables for fallback configuration For the `rust-tests.py --rhg` we want every `hg` command in tests to run `rhg` with fallback enabled, but other methods of setting configuration are limited or disruptive as discussed in code comment. Differential Revision: https://phab.mercurial-scm.org/D10186

paths: add a `*` special path to define default sub option Differential Revision: https://phab.mercurial-scm.org/D10163

ui: pass a `ui` object to `paths.getpath` I want to introduce more path's suboption and make it possible to use default value for them. Processing theses sub-options might result in warnings. We need a `ui` object to issue such warnings. To make things simpler, we add an helper method on the `ui` object. Differential Revision: https://phab.mercurial-scm.org/D10162

configitems: add TODOs blocking the move out of experimental for revlogv2 These are the todos so far, we probably will have more, but we might as well list them while they're fresh in our minds. Differential Revision: https://phab.mercurial-scm.org/D10216

sidedata-exchange: rewrite sidedata on-the-fly whenever possible When a A exchanges with B, the difference of their supported sidedata categories is made, and the responsibility is always with the client to generated it: - If A pushes to B and B requires category `foo` that A does not have, A will need to generate it when sending it to B. - If A pulls from B and A needs category `foo`, it will generate `foo` before the end of the transaction. - Any category that is not required is removed. If peers are not compatible, abort. It is forbidden to rewrite sidedata for a rev that already has sidedata, since that would introduce unreachable (garbage) data in the data file, something we're not prepared for yet. Differential Revision: https://phab.mercurial-scm.org/D10032

revlog-index: add `replace_sidedata_info` method During a `pull` operation where the server does not provide sidedata, the client that requires it should generate them on-the-fly. In the generic case, we need to wait for the changelog + manifests + filelogs to be added, since we don't know what the sidedata computers might need: this means rewriting the sidedata of index entries from within the pull transaction (and no further back) right after we've added them. Both Python and C implementations only allow for rewriting the sidedata offset and length for revs within the transaction where they were created. Differential Revision: https://phab.mercurial-scm.org/D10031

revlogv2: temporarily forbid inline revlogs See inline comments. I plan to fix the underlying issue before revlogv2 is stabilized. Differential Revision: https://phab.mercurial-scm.org/D10030

changegroupv4: add sidedata helpers These helpers carry the information and computers needed to rewrite sidedata when generating/applying patches. We will be making use of them soon. Differential Revision: https://phab.mercurial-scm.org/D10029

revlog: add attribute on revlogs that specifies its kind The sidedata logic needs to check whether the revlog it's working on is a changelog, a manifest or a filelog. Furthermore, future versions of the revlog format will most likely see a split between the three types (i.e. they will store different information), so having this will be useful for other future endeavors as well. Differential Revision: https://phab.mercurial-scm.org/D10151

sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos Each repo will advertise the sidedata categories it requires (categories being unique and canonical), and have a set of "computers", functions to generate sidedata from `(repo, revlog, rev, previous_sidedata)`, for a given category. The set of computers can be a superset of the set of the wanted categories, but not smaller: repos are expected to be coherent in their handling of sidedata. Differential Revision: https://phab.mercurial-scm.org/D10028

delta: add sidedata field to revision delta When emitting revision delta, we need to also emit the sidedata information just added in the revlogv2 format if appropriate. Differential Revision: https://phab.mercurial-scm.org/D10027

changegroup: add v4 changegroup for revlog v2 exchange This change only adds the required infrastructure for the new changegroup format and does not do any actual exchange. This will be done in the next patches. Differential Revision: https://phab.mercurial-scm.org/D10026

revlogv2: don't assume that the sidedata of the last rev is right after data We are going to be rewriting sidedata soon, it's going to be appended to the revlog data file, meaning that the data and the sidedata might not be contiguous. Differential Revision: https://phab.mercurial-scm.org/D10025

sidedata: move to new sidedata storage in revlogv2 The current (experimental) sidedata system uses flagprocessors to signify the presence and store/retrieve sidedata from the raw revlog data. This proved to be quite fragile from an exchange perspective and a lot more complex than simply having a dedicated space in the new revlog format. This change does not handle exchange (ironically), so the test for amend - that uses a bundle - is broken. This functionality is split into the next patches. Differential Revision: https://phab.mercurial-scm.org/D9993

cext: add support for revlogv2 This enables the C code to retrieve/create entries in the revlog v2 format. This is mainly a matter of taking into account the additional slots for sidedata, etc. Differential Revision: https://phab.mercurial-scm.org/D9846

bitmanipulation: add utils to read/write bigendian 64bit integers Differential Revision: https://phab.mercurial-scm.org/D9845

format: remove sidedata format variant This format variant can only exist when also using revlog v2, so it's useless now. Differential Revision: https://phab.mercurial-scm.org/D10113

revlogv2: allow upgrading to v2 Revlogv2 implies sidedata. Right now sidedata is not really used in production, and Revlogv2 will be used for the first production-ready version of sidedata support. Differential Revision: https://phab.mercurial-scm.org/D9844

revlog: introduce v2 format As documented in [1], this is still tentative and could be subject to change, but we need to lay down the foundations in order to work on the next abstraction layers. [1] https://www.mercurial-scm.org/wiki/RevlogV2Plan Differential Revision: https://phab.mercurial-scm.org/D9843

rebase: inline simple function for finding obsolete subset of commits `_filterobsoleterevs()` is just one line long. It was introduced in 2d294dada4f8 (rebase: small refactoring to allow better extensibility from extensions, 2016-01-14), for use by the "inhibit" extension. That extension was removed from the evolve repo in 87e87881059d (compat: drop the inhibit hacky extension, 2017-10-24). Differential Revision: https://phab.mercurial-scm.org/D10198

ci: hook network-io tests into the pipeline This runs the "pip install" tests once for Python 2 and 3 each. Differential Revision: https://phab.mercurial-scm.org/D10075

merge: with stable

test: update expected output in test-http.t The output was introduced in a4c19a162615 and is wrong and unstable. So we glob it as other part of these tests already do. Differential Revision: https://phab.mercurial-scm.org/D10153

remotefilelog: remove unused import This is no longer used since a4c19a162615. Differential Revision: https://phab.mercurial-scm.org/D10152

wireprotov1peer: don't raise internal errors in some cases Specifically, when the peer is closed in the middle of a batch of rpcs. Differential Revision: https://phab.mercurial-scm.org/D10074

sshpeer: don't fail forwarding output from closed connections The test still shows an internal error, but one that happens further along. Differential Revision: https://phab.mercurial-scm.org/D10073

test: show internal exception with batchable rpcs over ssh Differential Revision: https://phab.mercurial-scm.org/D10072