tests: partially fix test-transaction-rollback-on-revlog-split.t on Windows For whatever reason, the extra space causes different output on Windows: $ printf '% 20d' '1' | xxd - 0000000: 2020 2020 2020 2020 2020 2020 2020 2020 0000010: 2020 2020 31 1 vs WSL: /mnt/c/Users/Matt/hg/tests$ printf '% 20d' '1' | xxd - 00000000: 2020 2020 2020 2020 2020 2020 2020 2020 00000010: 2020 2031 Without the space, they both agree the last word is 2031. There are other failures on Windows that I don't understand that are introduced by 46b828b85eb7 and 18415fc918a1. Differential Revision: https://phab.mercurial-scm.org/D10763

tests: fix test-transaction-safety.t on Windows Mostly this is about `C:\dir\sub` style paths being turned into `C:dirsub` by MSYS, and not being about to execute shell scripts in hooks directly. See also a12819559ccb::9e3979a25bfe. Differential Revision: https://phab.mercurial-scm.org/D10762

docket: move the uid logic in the `revlogutils.docket` module We want to use it for revlog-v2 (& Co), it seems more logical to have the logic lives inside the `docket` file than the `nodemap` file. Differential Revision: https://phab.mercurial-scm.org/D10755

persistent-nodemap: use the intended uuid size I overlooked this in the initial implementation. The format already supporting any uuid size so, nothing horrible should happens. Differential Revision: https://phab.mercurial-scm.org/D10754

persistent-nodemap: add a way to make the picked uid predictable This will make the test less volatile and therefore easier to maintain. I am setting this is for the persistent nodemap docket first, but my target is to use similar logic for the revlogv2 (and co) test. For persisent nodemap, they are too many different in how each test variants (with rust, without rust, etc) needs new uid for this be really useful (right now). However it will be directly useful for the revlogv2 logic. Differential Revision: https://phab.mercurial-scm.org/D10753

run-tests: disable color in 'auto' mode on Windows if stdout is redirected Differential Revision: https://phab.mercurial-scm.org/D10761

run-tests: enable color on Windows In setting up the CI for Windows on heptapod, I noticed it was complaining about color not being enabled because pygments wasn't installed- even though it was. I had initially disabled color on Windows when using Windows 7, because that didn't understand ANSI color codes and made a mess of the output. But now that it's been unsupported for over a year, I don't think we should care about it either. It's admittedly a hack to depend on Mercurial proper to enable color support in the terminal, but I didn't feel like duplicating that code. I'm under the impression that 3rd party stuff is supposed to use this runner in the Mercurial repo instead of using their own copy, so I think it's safe to assume the Mercurial code is available. If it's not, it won't break anything. Differential Revision: https://phab.mercurial-scm.org/D10760

transaction: trivial refactoring Differential Revision: https://phab.mercurial-scm.org/D10714

test: remove some unnecessary dependency on repo format Differential Revision: https://phab.mercurial-scm.org/D10757

hg-cpython: fix new occuring TypeError dd339191f2dc introduced a minor refactoring of error types that highlighted a fragile error creation mechanism that was in place in the Rust `MixedIndex`. `PyErr::from_instance` also accepts a Python class, as long as it's an `Exception` class. Before the aforementioned commit, we never ran into a case where this duck-typing mechanism failed. We rectify this behavior by doing the instantiation ourselves. Differential Revision: https://phab.mercurial-scm.org/D10764

errors: make exit codes class variables instead Kyle pointed out to me that we can simply make the exit codes class variables. Python provides some magic for making them accessible as instance variables. This also makes it easier to let subclasses of existing errors override the exit codes by letting them simply define their own values as class variables. That means that there's no need to pass them into the superclass's constructor arguments, so the superclass doesn't need to expose the them as arguments. (Making a subclass set a different exit code for a subclass of `StorageError` was actually the goal with my recent series.) Differential Revision: https://phab.mercurial-scm.org/D10758

narrow: widen when files are excluded by sparse and not included by narrow In a repo where some directories are included by narrow and the complement are excluded by sparse, it was previously impossible to widen either because trying to widen narrow would complain that the requested files are outside the sparse checkout and trying to widen sparse would complain that the requested files are outside the narrow checkout. This changes the `hg tracked --addinclude` command to only actually update any newly accessible files in the dirstate if they are also accessible via sparse. Differential Revision: https://phab.mercurial-scm.org/D10734

hghave: make error output easier to diagnose I had a typo that meant the new bash check was throwing an exception (due to a missing argument), but it was very hard to diagnose without this change. Differential Revision: https://phab.mercurial-scm.org/D10733

tests: add req on bash for test-transaction-rollback-on-sigpipe (issue6429) I think we could work around this by rewriting the helper scripts in Python, but I don't want to deal with that now and this should prevent failures due to a lack of bash. Differential Revision: https://phab.mercurial-scm.org/D10732

updatecaches: deprecate the `full` argument Now that all users were migrated, we can use deprecate the old way. This would give potential extensions code a heads up on the API change. Differential Revision: https://phab.mercurial-scm.org/D10731

updatecaches: use the `caches` argument instead of a special `full` value After a clone we want to update most cachem, but not exactly all of them. We can now cleanly express this. Differential Revision: https://phab.mercurial-scm.org/D10730

updatecaches: use the caches argument in `hg debugupdatecaches` This is the new way. Differential Revision: https://phab.mercurial-scm.org/D10729

updatecaches: adds a `caches` parameters to `repo.updatecaches` It will superseed the `full` parameters (and its `post-clone` variant from stable). Various caller will be updated in the rest of this series. Differential Revision: https://phab.mercurial-scm.org/D10728

updatecaches: introduce a set of constants to control which are updated Passing around a set of constant to select what need warming will be cleaner and more flexible. We did not changed the API yet, as this changes is already large enough. In the rest of the rest we will change more code to actually use this constants (or more realistically pre-defined set of constant directly) Differential Revision: https://phab.mercurial-scm.org/D10727

errors: make StorageError subclass Error, attaching an exit code to it Differential Revision: https://phab.mercurial-scm.org/D10741

revlog: avoid raising no-arg RevlogError for internal flow control I'm about to make RevlogError require a `message` argument and this code was failing. This patch refactors it to not raise an exception for intra-function flow control. Differential Revision: https://phab.mercurial-scm.org/D10740

errors: catch the new Error class in scmutil and chgserver Differential Revision: https://phab.mercurial-scm.org/D10739

errors: create superclass for Abort exception I'd like to let extensions subclass `StorageError` to define a custom exit code. However, `StorageError` does not extend `Abort` (which is where the exit code currently lives), and it seems that it's not supposed to either (`StorageError` seems to be for lower-level errors and `Abort` is for command-level errors). This patch therefore extracts all the code from `Abort` into a new `Error` class, which I'll soon make `StorageError` also extend. Differential Revision: https://phab.mercurial-scm.org/D10738

errors: make InterventionRequired subclass Abort The docstring for `Abort` says that it's for errors raised by commands and `InterventionRequired` is definitely something raised by commands, so it seems that it should be an `Abort`. This patch makes it so. It adds a `coarse_exit_code` (in addition to the already existing `detailed_exit_code`) to `Abort` to achieve that, since `InterventionRequired` should result in a special exit code even when the `ui.detailed-exit-code` config is not set. Differential Revision: https://phab.mercurial-scm.org/D10737

errors: move Abort earlier, so more exceptions can subclass it I'd like to make at least `InterventionRequired` subclass `Abort` and Python requires the superclass to be defined before the subtype. Differential Revision: https://phab.mercurial-scm.org/D10736

errors: let each Abort subclass define its error code It's more flexible to have the error codes defined on the error types themselves. That way extensions can easily set their own exit code. It also means that we can reduce a bit of duplication betwen `scmutil.callcatch()` and `chgserver.chgcmdserver.validate()`. Differential Revision: https://phab.mercurial-scm.org/D10735

rust: Fix "panic message is not a string literal" warnings These deprecation warnings would not become errors until we actively port crates to the (not yet released) Rust 2021 edition, but fixing them anyway reduces console output noise. Differential Revision: https://phab.mercurial-scm.org/D10743

rust: Add type annotation to fix inference on Rust Nightly When compiling with Rust Nightly, the im-rs crate silently makes use of the experimental language feature for trait impl specialization. This apperently changes public its APIs in subtle ways such that type inference of some user code can fail where it succeeds when specialization is disabled. This made Mercurial’s Rust unit tests have compilation errors on Nightly. I have not managed to find the exactl root cause, but I wrote down my findings so far at https://github.com/bodil/im-rs/issues/188 This adds type annotation to make unit tests rely less on type inference and work around the issue. Differential Revision: https://phab.mercurial-scm.org/D10742

recover: only apply last journal record per file (issue6423) This got broken in 2019 when the size check was introduced. It is most noticable when dealing with transactions that involve an inline to non-inline revlog storage transaction. It wasn't seen as much at the time because the in-memory journal actually de-duplicated the entry implicity, but since 63edc384d3b7 the on-disk journal is used for rollback as well as recover. Differential Revision: https://phab.mercurial-scm.org/D10726

revlog: update data file record before index rename When migrating from inline to non-inline data storage, the data file is recorded initially as zero sized so that it is removed on failure. But the record has to be updated before the index is renamed, otherwise data is lost on rollback. Differential Revision: https://phab.mercurial-scm.org/D10725

revlog: fix index computation during inline->non-inline transition The computation in 63edc384d3b7 failed to factor in the index entries themselve as revlog.start() doesn't count them. Found by Valtenin Gatienbaron with a more precise test case from me. Differential Revision: https://phab.mercurial-scm.org/D10724

dirstate-v2: Change the on-disk format to be tree-shaped Nodes are stored not only for tracked files but also for their ancestor directories. A node has "pointers" (byte count from the start of the file) to its direct child nodes. Everything can be accessed with zero copy. Differential Revision: https://phab.mercurial-scm.org/D10722

dirstate-tree: Extract into a method sorting children of a given node A later changset will use this in another place. This is an associated function (that Python would call static method) instead of a free function so it doesn’t need to be imported separately. It’s on `Node` rather than `ChildNodes` because the latter is a type alias to an external type (`HashMap`) so that would require an extension trait which needs to be imported separately. Differential Revision: https://phab.mercurial-scm.org/D10721

dirstate-v2: Add a variant of some tests, that uses the new format With this, the new format receives some testing every time someone runs tests with Rust extensions enabled, including on CI. Differential Revision: https://phab.mercurial-scm.org/D10720

dirstate-v2: Change the on-disk format when the requirement is enabled For now, the format is the same except with an additional marker at the start. This marker is redundant: for existing repositories it is `.hg/requires` that determines which format to use. For new repositories, it is the new `format.exp-dirstate-v2` config. There is no upgrade or downgrade so far. Most of the changes are about plumbing a boolean through layers of APIs to indicate which format should be used. Differential Revision: https://phab.mercurial-scm.org/D10719

dirstate-v2: Update the expected output of some tests for new requirement Fix most test failures (except in test-narrow-debugrebuilddirstate.t and test-upgrade-repo.t) caused by the new entry in config or in .hg/requires when running `run-tests.py --extra-config-opt format.exp-dirstate-v2=1` There is no CI so far for this configuration. Differential Revision: https://phab.mercurial-scm.org/D10718

tests: More cleanly separate expected hexdump output There are more lines that differ than are in common, and dirstate-v2 will complicate that further. Differential Revision: https://phab.mercurial-scm.org/D10717

dirstate-v2: Add a new experimental `exp-dirstate-v2` repository requirement This requirement is added to `.hg/requires` when creating a new repository if Rust extensions are enabled and the `format.exp-dirstate-v2` config is set. Nothing yet changes based on this requirement, but its mere presence affects some tests (for example if they print `.hg/requires`). The next two changesets update tests’ expected outputs accordingly. There is no CI so far that enables this configuration. Differential Revision: https://phab.mercurial-scm.org/D10716

urlutil: provide some information about "bad url" when processing `pushurl` It appears pushurl only support `<proto>://` entries. This is not obvious and can lead to obscure error. We make the error less obscure as a start.. Differential Revision: https://phab.mercurial-scm.org/D10455

revset: document the `outgoing` behavior if the path resolve to multiple urls Differential Revision: https://phab.mercurial-scm.org/D10454

multi-urls: document the feature We have a feature lets make it visible to people. Differential Revision: https://phab.mercurial-scm.org/D10453

multi-urls: add a boolean suboption that unlock path specification as list When this option is set, a list of patch can be specifed as value for `[paths]` entries. For the command who support it, this behave the same as providing multiple destination of the command line. Differential Revision: https://phab.mercurial-scm.org/D10452

urlutil: move url "fixing" at the time of `ui.paths` initialization Doing such fixing at the time is simpler and will be necessary to deal with urls list in a sane manner. It also reduce the size of fix-config which is always better. I wish we could get ride of the hackish way to pass the root around, I suspect that the `root` variable could be stored as part of the config value, along side the source. However getting to the end of this `root` business is a far too large detours to make now. The test change to `tests/test-hgrc.t` and `test-config.t` are expectied since we are not longer altering the config itself, but the way it is interpreted when building path. This seems more correct. I also added a couple of test call to `test-config.t` and `test-globalopts.t` to clarify that the expanding process is properly happening a the right time. Differential Revision: https://phab.mercurial-scm.org/D10451

rewriteutil: add pointer to help text when rewrite would cause divergence The evolve extension's version of the hint has this pointer. I missed it when I moved it to core. Differential Revision: https://phab.mercurial-scm.org/D10723

changelogv2: use a dedicated on disk format for changelogv2 We drop two unused entry. This is mostly a proof of concept before starting to actually rework the format. Differential Revision: https://phab.mercurial-scm.org/D10667

revlog: do not call Rust code if the index is not compatible with it This will avoid hitting the TypeError we defined in the previous changesets. This is the simplest fix but not the most elegant. Ideally we would teach the Rust code to use any kind of revlog. However this is an adventure for another time. Differential Revision: https://phab.mercurial-scm.org/D10666

revlog: signal which revlog index are compatible with Rust Otherwise, Rust may treat python object like `cindex` object, leading to trouble. The new attribute is an integer because I expect we might need a flag field in the future. As a start we get the rust code to raise a clear TypeError. We will use the information in a smarter way in the next changesets. Differential Revision: https://phab.mercurial-scm.org/D10665

revlog: pass around the `rev` we deal with when packing/unpacking entry This will help code to compute some flag on the fly. Differential Revision: https://phab.mercurial-scm.org/D10664

changelogv2: use a dedicated version number If we want to change the format, we need a new version number. We start with that. Differential Revision: https://phab.mercurial-scm.org/D10663

changelogv2: `copies-side-data` now implies `changelogv2` To store information about file changes and copies only requires sidedata support for the changelog. So we only enables `changelogv2`. This is less impactful and should help us to ship the changeset centric copy tracing sooner. Differential Revision: https://phab.mercurial-scm.org/D10662

changelogv2: allow upgrade from and to this format Differential Revision: https://phab.mercurial-scm.org/D10661

changelogv2: introduce a "changelogv2" feature Right now, this means using revlogv2, but only for the changelog. We will have the format more unique in future changesets. Differential Revision: https://phab.mercurial-scm.org/D10660

revlogv2: drop the code related to inlined revlogv2 Tevlogv2 will not use inlined revlog. As a result we can drop the associated code. Differential Revision: https://phab.mercurial-scm.org/D10659

revlogv2: no longer attempt to use inline for new revlog revlogv2 steered away for the inline feature. Differential Revision: https://phab.mercurial-scm.org/D10658

sidedata: use the "feature" to detect that sidedata are enabled This is more versatile than relying on the revlogv2 requirements. Differential Revision: https://phab.mercurial-scm.org/D10657

revlog: compress sidedata when doing "post-pull" sidedata update All path writing sidedata are now using compression (when appropriate). Differential Revision: https://phab.mercurial-scm.org/D10656

revlog: compress sidedata in `_writeentry` When appropriate we store the sidedata in a compressed form. Differential Revision: https://phab.mercurial-scm.org/D10655

revlog: apply compression mode while reading sidedata Right now, we only store things as PLAIN, but we are ready for some compression now. Future changeset will start using some. Differential Revision: https://phab.mercurial-scm.org/D10654

revlog: introduce a compression mode for sidedata in the revlog index We will use this for compression for the sidedata payload. Differential Revision: https://phab.mercurial-scm.org/D10653

revlog: implement a "default compression" mode The revlog docker is now storing a default compression engine. When a chunk use that compression, a dedicated mode is used in the revlog entry and we can directly route it to the right decompressor. We should probably make PLAIN and DEFAULT mode the only available mode for revlogv2, but this is something for later. Differential Revision: https://phab.mercurial-scm.org/D10652