bundle: optional multithreaded compression, ATM zstd-only Compression type can be a huge chunk of "hg bundle", especially when using the higher compression levels. With level=22 and threads=7, the NetBSD test repository took 28:39 wall time and 157:47 user time. Before, level=22 would take 129:20 wall time and 129:07 user time. Differential Revision: https://phab.mercurial-scm.org/D9283

rust: Preallocate the returned `Vec` in `utils::files::relativize_path` Profiling `rhg files > /dev/null` on an old snapshot of mozilla-central (with `perf` and the Firefox Profiler: https://github.com/firefox-devtools/profiler/blob/main/docs-user/guide-perf-profiling.md) showed non-trivial time spend in this function and in `realloc`. This change makes the wall-clock time for that process on my machine go from ~190 ms to ~150 ms. Differential Revision: https://phab.mercurial-scm.org/D10199

black: merge config into main pyproject.toml now that we have it This means that naive contributors who just run `black` on a source file will get reasonable behavior as long as they have a recent black. Yay! This was previously D9834 but was rolled back due to test failures. nbjoerg thinks it's time to try again, so let's give it a shot. Differential Revision: https://phab.mercurial-scm.org/D10185

pyproject: add config file This will tell pip et al to call our setup.py for the majority of packaging concerns, but also gives us a place to put standard config stuff like black. This was previously D9833, but was rolled back due to test breakage. nbjoerg thinks that breakage is now resolved, so we're trying again. Differential Revision: https://phab.mercurial-scm.org/D10184

perf-util: add an helper revset to use the same spec as the case search script This make result of the searcher script easier to use. Differential Revision: https://phab.mercurial-scm.org/D10228

perf-helpers: add a search-discovery-case script This a small script I built to look for interesting discovery case. It is fairly basic but could be useful in various situation so lets put it in the main repositories. Differential Revision: https://phab.mercurial-scm.org/D10225

debugdiscovery: add missing byte string marker to some help text This is causing crash when using --help. Differential Revision: https://phab.mercurial-scm.org/D10227

debugdiscovery: also integrate the discovery output in the json one We add a way for formatter to informs code that free output is unwanted, and we incorporate it in the json output. Differential Revision: https://phab.mercurial-scm.org/D10224

debugdiscovery: add support for Json output They are some pre-output that will be impractical, I'll take care of it in the next changeset. Differential Revision: https://phab.mercurial-scm.org/D10223

perf-helper: add a new sampling revset based on anti-chain See inline documentation for details. Differential Revision: https://phab.mercurial-scm.org/D10222

perf-helper: add a small extension with revsets to select repository subset Playing with discovery requires building interesting case. To do this we need revsets to try to generate them. We start with a quite simple one. See documentation for details. Differential Revision: https://phab.mercurial-scm.org/D10221

amend: mark commit obsolete after moving working copy We were doing it this way: 1. move working copy (repo.setparents) 2. add obsmarkers (scmutil.cleanupnodes) 3. fix dirstate (dirstate.normal/drop) Step 1 and 3 are closely related, so let's move them together. It seems safest to create the obsmarkers last. This patch thus makes the order 1, 3, 2. Differential Revision: https://phab.mercurial-scm.org/D10197

pull: allow to specify multiple sources I end up needing that on a regular basis and it turn out to be very simple to implement. See documentation and test for details. Differential Revision: https://phab.mercurial-scm.org/D10159

pull: pre-indent a part of the function We are about to run it in a loop, so lets pre-indent it to clarify the actual change in the next changesets. Differential Revision: https://phab.mercurial-scm.org/D10158

command: clarify `postincoming` return and that return handling The command should return None or a return code. The previous code was returning boolean directly relying on the fact that `True → 1` and `False → 0`. This is a good road to troubles, so lets be explicit about that return. Differential Revision: https://phab.mercurial-scm.org/D10157

hg: make `clean` return consistent with the `update` function The function return a boolean and is used as such. Lets be explicit about it. Differential Revision: https://phab.mercurial-scm.org/D10156

tests: rename `test-pull` to `test-pull-network.t` If is mostly about ssh and http interaction so lets avoid confusion with a generic `hg pull` test Differential Revision: https://phab.mercurial-scm.org/D10155

tests: move a test about update in test-pull-update.t Differential Revision: https://phab.mercurial-scm.org/D10154

split: close transaction in the unlikely event of a conflict while rebasing `hg split` *should* never result in conflicts, but in case there are bugs, we should at least commit the transaction so they can continue the rebase. One of our users ran into the regression fixed by D10120. They fixed the conflict and the tried to continue the rebase, but it failed with "abort: cannot continue inconsistent rebase" because the rebase state referred to commits written in a transaction that was never committed. Side note: `hg split` should probably turn off copy tracing to reduce the impact of such bugs, and to speed it up as well. Copies made in the rebased commits should still be respected because `hg rebase` calls `copies.graftcopies()`. Differential Revision: https://phab.mercurial-scm.org/D10164

rust-status: fix issue6456 for the Rust implementation also This implementation is being used by a few people now, so we need it fixed, even though a big rewrite will be coming quite soon. Differential Revision: https://phab.mercurial-scm.org/D10217

README: document requirement for builtin zstd Differential Revision: https://phab.mercurial-scm.org/D10226

ci: Add a job testing with rhg installed as `hg` This significantly increases test coverage of rhg, without duplicating many tests that already exist. The `cargo build` command being remove only compiled a `rhg` executable (as shown by the preceding `cd` command) but since a previous patch `run-tests.py --rhg` now takes care of that. Differential Revision: https://phab.mercurial-scm.org/D10196

run-test: install rhg if --rhg is passed Before this, --rhg was only working with --local. Differential Revision: https://phab.mercurial-scm.org/D10195

makefile: add a install option This gives and easy way to install rhg that we can use in `run-test.py` in the next changesets. Differential Revision: https://phab.mercurial-scm.org/D10194

makefile: add a build-rhg option This gives an easy action to build the rhg-binary. This will be useful for the `install-rhg` action in the next changeset. Differential Revision: https://phab.mercurial-scm.org/D10193

makefile: add a install-chg option This is done as a gratuitous improvement on the way to add makefile entry to build and install rhg. It seems saner to have equivalent entry for chg too. Differential Revision: https://phab.mercurial-scm.org/D10192

makefile: add a build-chg option This is done as a gratuitous improvement on the way to add makefile entry to build and install rhg. It seems saner to have equivalent entry for chg too. Differential Revision: https://phab.mercurial-scm.org/D10191

tests: Disable for rhg remaining tests that fail in that mode These cases are in features not yet implemented by rhg for which triggering a fallback is not practical. Disabling some tests allows us to reach passing CI and catch any future regression in the rest of the tests. Differential Revision: https://phab.mercurial-scm.org/D10190

rhg: Remove `rhg.fallback-executable=hg` default configuration When `rhg.on-unsupported` is configured to `fallback` and an unsupported feature is encountered, the previous default was to look for an `hg` executable in `$PATH`. This default was fragile since it was easy to end up accidentally using an older version of Mercurial installed system-wide, when a local (perhaps patched) installation was intended. Instead, it is now an error to have `rhg.on-unsupported=fallback` without also configuring an explicit path or the fallback executable. Differential Revision: https://phab.mercurial-scm.org/D10189

rhg: Return an error code for `rhg config Section.idontexist` This is what Python-based hg does. Differential Revision: https://phab.mercurial-scm.org/D10145

rhg: Fall back to Python for --version Clap has some built-in support for printing something on --version, but it looks different than what Pytho-based hg does. Also, at the moment we’re not giving version numbers to rhg separate from the Mercurial version. Differential Revision: https://phab.mercurial-scm.org/D10144

rhg: Exit with an error code if `files` finds nothing This matches the behavior of Python-based hg. Differential Revision: https://phab.mercurial-scm.org/D10143

rhg: `cat` command: print error messages for missing files And exit with an error code if no file was matched. This matches the behavior of Python-based hg. Differential Revision: https://phab.mercurial-scm.org/D10142

rhg: Silently ignore missing files in config %include … instead of aborting with an error message. This is what Python-based hg does. Differential Revision: https://phab.mercurial-scm.org/D10141

rhg: Add support for environment variables in config include paths Some tests rely on this. Differential Revision: https://phab.mercurial-scm.org/D10140

rhg: Add support for the HGRCSKIPREPO environment variable It’s easy enough and affects tests, well, that test that feature. Differential Revision: https://phab.mercurial-scm.org/D10139

rhg: Don’t make repository path absolute too early Some error messages want to include a relative path, which affects the output of some tests. Differential Revision: https://phab.mercurial-scm.org/D10138

rhg: Fall back to Python if ui.relative-paths is configured This feature is not supported yet, and affects the output of some tests. Differential Revision: https://phab.mercurial-scm.org/D10137

tests: clarify some missing output in test-merge-subrepos This makes the test behavior clearer, especially, why the output are inconsistent. Differential Revision: https://phab.mercurial-scm.org/D10136

tests: Adapt expected output for minor differences with rhg Differential Revision: https://phab.mercurial-scm.org/D10135

rhg: Add support for --cwd This affect the meaning of relative paths in `--repository`, which are resolved "early" by rhg in order to load config which is needed before fallback to Python is considered. An incorrect path could cause errors when loading a non-existent repo, leading to failing tests even when fallback is enabled. Differential Revision: https://phab.mercurial-scm.org/D10134

rhg: Align with Python on some more error messages Differences can cause some tests to fail Differential Revision: https://phab.mercurial-scm.org/D10133

rhg: Add an allow-list of ignored extensions Because rhg doesn’t know how a Python extension would affect behavior it implements in Rust, when an unsupported extension is enabled it conservatively falls back to Python-based hg. However many users will have unsupported extensions enabled in practice. Maybe they don’t actually affect rhg behavior, but we don’t know. This adds a `rhg.ignored-extensions` configuration that lets users list extensions that rhg can safely ignore and proceed even if they’re not supported in Rust. Differential Revision: https://phab.mercurial-scm.org/D10188

rhg: Fall back to Python if unsupported extensions are enabled Extensions might affect behavior in ways we can’t anticipate, so just ignoring them is not correct. Later we’ll add opt-in configuration to ignore specific extensions. Differential Revision: https://phab.mercurial-scm.org/D10112

rhg: Sort config files when adding a directory For example in `/etc/mercurial/hgrc.d/` or with `HGRCPATH=some-directory`. Previously files where parsed in the order returned by the filesystem, which is undefined. But order is significant when multiple files define the same configuration key: the "last" one wins. Differential Revision: https://phab.mercurial-scm.org/D10111

rhg: Align config file parse error formatting with Python Differences can cause tests to fail Differential Revision: https://phab.mercurial-scm.org/D10110

rhg: Fall back to Python for bundle repositories rhg does not support bundles at all, yet. Differential Revision: https://phab.mercurial-scm.org/D10102

rhg: Fall back to Python on --repository with an URL A low-hanging fruit to improve on this would be to properly parse and handle `file:` URLs. But other Python-based hg supports some other URL schemes for features that rhg does not support yet. Differential Revision: https://phab.mercurial-scm.org/D10101

rhg: Print non-absolutized path in "repository {} not found" errors … like Python does. Differences in ouput can cause tests to fail. Differential Revision: https://phab.mercurial-scm.org/D10100

rhg: Align "malformed --config" error message with Python Differences in error message formatting can cause tests to fail. Differential Revision: https://phab.mercurial-scm.org/D10099

rhg: Fall back to Python on unsupported `rhg config <section>` Printing an entire section (as opposed to a single value with `rhg config foo.bar`) is not supported yet in Rust only. Differential Revision: https://phab.mercurial-scm.org/D10098

rhg: Fall back to Python for unsupported revset syntax rhg only supports a small subset of the syntax. On parse error, this gives Python a chance instead of aborting immediately. Differential Revision: https://phab.mercurial-scm.org/D10097

tests: Add `rhg` and `no-rhg` for #require and #if in .t files With this we can allow some small differences in expected output between Rust-based and Python-based code paths. Differential Revision: https://phab.mercurial-scm.org/D10096

tests: Enable rhg fallback to Python by default in tests This premise of `run-tests.py --rhg`: fallback should make `rhg` behave the same as `hg`, except faster in some cases. To test run the whole test suite with installed `rhg` as `hg` and with fallback enabled. Differential Revision: https://phab.mercurial-scm.org/D10187

rhg: Add environment variables for fallback configuration For the `rust-tests.py --rhg` we want every `hg` command in tests to run `rhg` with fallback enabled, but other methods of setting configuration are limited or disruptive as discussed in code comment. Differential Revision: https://phab.mercurial-scm.org/D10186

paths: add a `*` special path to define default sub option Differential Revision: https://phab.mercurial-scm.org/D10163

ui: pass a `ui` object to `paths.getpath` I want to introduce more path's suboption and make it possible to use default value for them. Processing theses sub-options might result in warnings. We need a `ui` object to issue such warnings. To make things simpler, we add an helper method on the `ui` object. Differential Revision: https://phab.mercurial-scm.org/D10162

configitems: add TODOs blocking the move out of experimental for revlogv2 These are the todos so far, we probably will have more, but we might as well list them while they're fresh in our minds. Differential Revision: https://phab.mercurial-scm.org/D10216

sidedata-exchange: rewrite sidedata on-the-fly whenever possible When a A exchanges with B, the difference of their supported sidedata categories is made, and the responsibility is always with the client to generated it: - If A pushes to B and B requires category `foo` that A does not have, A will need to generate it when sending it to B. - If A pulls from B and A needs category `foo`, it will generate `foo` before the end of the transaction. - Any category that is not required is removed. If peers are not compatible, abort. It is forbidden to rewrite sidedata for a rev that already has sidedata, since that would introduce unreachable (garbage) data in the data file, something we're not prepared for yet. Differential Revision: https://phab.mercurial-scm.org/D10032

revlog-index: add `replace_sidedata_info` method During a `pull` operation where the server does not provide sidedata, the client that requires it should generate them on-the-fly. In the generic case, we need to wait for the changelog + manifests + filelogs to be added, since we don't know what the sidedata computers might need: this means rewriting the sidedata of index entries from within the pull transaction (and no further back) right after we've added them. Both Python and C implementations only allow for rewriting the sidedata offset and length for revs within the transaction where they were created. Differential Revision: https://phab.mercurial-scm.org/D10031

revlogv2: temporarily forbid inline revlogs See inline comments. I plan to fix the underlying issue before revlogv2 is stabilized. Differential Revision: https://phab.mercurial-scm.org/D10030

changegroupv4: add sidedata helpers These helpers carry the information and computers needed to rewrite sidedata when generating/applying patches. We will be making use of them soon. Differential Revision: https://phab.mercurial-scm.org/D10029

revlog: add attribute on revlogs that specifies its kind The sidedata logic needs to check whether the revlog it's working on is a changelog, a manifest or a filelog. Furthermore, future versions of the revlog format will most likely see a split between the three types (i.e. they will store different information), so having this will be useful for other future endeavors as well. Differential Revision: https://phab.mercurial-scm.org/D10151

sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos Each repo will advertise the sidedata categories it requires (categories being unique and canonical), and have a set of "computers", functions to generate sidedata from `(repo, revlog, rev, previous_sidedata)`, for a given category. The set of computers can be a superset of the set of the wanted categories, but not smaller: repos are expected to be coherent in their handling of sidedata. Differential Revision: https://phab.mercurial-scm.org/D10028