errors: let each Abort subclass define its error code It's more flexible to have the error codes defined on the error types themselves. That way extensions can easily set their own exit code. It also means that we can reduce a bit of duplication betwen `scmutil.callcatch()` and `chgserver.chgcmdserver.validate()`. Differential Revision: https://phab.mercurial-scm.org/D10735

rust: Fix "panic message is not a string literal" warnings These deprecation warnings would not become errors until we actively port crates to the (not yet released) Rust 2021 edition, but fixing them anyway reduces console output noise. Differential Revision: https://phab.mercurial-scm.org/D10743

rust: Add type annotation to fix inference on Rust Nightly When compiling with Rust Nightly, the im-rs crate silently makes use of the experimental language feature for trait impl specialization. This apperently changes public its APIs in subtle ways such that type inference of some user code can fail where it succeeds when specialization is disabled. This made Mercurial’s Rust unit tests have compilation errors on Nightly. I have not managed to find the exactl root cause, but I wrote down my findings so far at https://github.com/bodil/im-rs/issues/188 This adds type annotation to make unit tests rely less on type inference and work around the issue. Differential Revision: https://phab.mercurial-scm.org/D10742

recover: only apply last journal record per file (issue6423) This got broken in 2019 when the size check was introduced. It is most noticable when dealing with transactions that involve an inline to non-inline revlog storage transaction. It wasn't seen as much at the time because the in-memory journal actually de-duplicated the entry implicity, but since 63edc384d3b7 the on-disk journal is used for rollback as well as recover. Differential Revision: https://phab.mercurial-scm.org/D10726

revlog: update data file record before index rename When migrating from inline to non-inline data storage, the data file is recorded initially as zero sized so that it is removed on failure. But the record has to be updated before the index is renamed, otherwise data is lost on rollback. Differential Revision: https://phab.mercurial-scm.org/D10725

revlog: fix index computation during inline->non-inline transition The computation in 63edc384d3b7 failed to factor in the index entries themselve as revlog.start() doesn't count them. Found by Valtenin Gatienbaron with a more precise test case from me. Differential Revision: https://phab.mercurial-scm.org/D10724

dirstate-v2: Change the on-disk format to be tree-shaped Nodes are stored not only for tracked files but also for their ancestor directories. A node has "pointers" (byte count from the start of the file) to its direct child nodes. Everything can be accessed with zero copy. Differential Revision: https://phab.mercurial-scm.org/D10722

dirstate-tree: Extract into a method sorting children of a given node A later changset will use this in another place. This is an associated function (that Python would call static method) instead of a free function so it doesn’t need to be imported separately. It’s on `Node` rather than `ChildNodes` because the latter is a type alias to an external type (`HashMap`) so that would require an extension trait which needs to be imported separately. Differential Revision: https://phab.mercurial-scm.org/D10721

dirstate-v2: Add a variant of some tests, that uses the new format With this, the new format receives some testing every time someone runs tests with Rust extensions enabled, including on CI. Differential Revision: https://phab.mercurial-scm.org/D10720

dirstate-v2: Change the on-disk format when the requirement is enabled For now, the format is the same except with an additional marker at the start. This marker is redundant: for existing repositories it is `.hg/requires` that determines which format to use. For new repositories, it is the new `format.exp-dirstate-v2` config. There is no upgrade or downgrade so far. Most of the changes are about plumbing a boolean through layers of APIs to indicate which format should be used. Differential Revision: https://phab.mercurial-scm.org/D10719

dirstate-v2: Update the expected output of some tests for new requirement Fix most test failures (except in test-narrow-debugrebuilddirstate.t and test-upgrade-repo.t) caused by the new entry in config or in .hg/requires when running `run-tests.py --extra-config-opt format.exp-dirstate-v2=1` There is no CI so far for this configuration. Differential Revision: https://phab.mercurial-scm.org/D10718

tests: More cleanly separate expected hexdump output There are more lines that differ than are in common, and dirstate-v2 will complicate that further. Differential Revision: https://phab.mercurial-scm.org/D10717

dirstate-v2: Add a new experimental `exp-dirstate-v2` repository requirement This requirement is added to `.hg/requires` when creating a new repository if Rust extensions are enabled and the `format.exp-dirstate-v2` config is set. Nothing yet changes based on this requirement, but its mere presence affects some tests (for example if they print `.hg/requires`). The next two changesets update tests’ expected outputs accordingly. There is no CI so far that enables this configuration. Differential Revision: https://phab.mercurial-scm.org/D10716

urlutil: provide some information about "bad url" when processing `pushurl` It appears pushurl only support `<proto>://` entries. This is not obvious and can lead to obscure error. We make the error less obscure as a start.. Differential Revision: https://phab.mercurial-scm.org/D10455

revset: document the `outgoing` behavior if the path resolve to multiple urls Differential Revision: https://phab.mercurial-scm.org/D10454

multi-urls: document the feature We have a feature lets make it visible to people. Differential Revision: https://phab.mercurial-scm.org/D10453

multi-urls: add a boolean suboption that unlock path specification as list When this option is set, a list of patch can be specifed as value for `[paths]` entries. For the command who support it, this behave the same as providing multiple destination of the command line. Differential Revision: https://phab.mercurial-scm.org/D10452

urlutil: move url "fixing" at the time of `ui.paths` initialization Doing such fixing at the time is simpler and will be necessary to deal with urls list in a sane manner. It also reduce the size of fix-config which is always better. I wish we could get ride of the hackish way to pass the root around, I suspect that the `root` variable could be stored as part of the config value, along side the source. However getting to the end of this `root` business is a far too large detours to make now. The test change to `tests/test-hgrc.t` and `test-config.t` are expectied since we are not longer altering the config itself, but the way it is interpreted when building path. This seems more correct. I also added a couple of test call to `test-config.t` and `test-globalopts.t` to clarify that the expanding process is properly happening a the right time. Differential Revision: https://phab.mercurial-scm.org/D10451

rewriteutil: add pointer to help text when rewrite would cause divergence The evolve extension's version of the hint has this pointer. I missed it when I moved it to core. Differential Revision: https://phab.mercurial-scm.org/D10723

changelogv2: use a dedicated on disk format for changelogv2 We drop two unused entry. This is mostly a proof of concept before starting to actually rework the format. Differential Revision: https://phab.mercurial-scm.org/D10667

revlog: do not call Rust code if the index is not compatible with it This will avoid hitting the TypeError we defined in the previous changesets. This is the simplest fix but not the most elegant. Ideally we would teach the Rust code to use any kind of revlog. However this is an adventure for another time. Differential Revision: https://phab.mercurial-scm.org/D10666

revlog: signal which revlog index are compatible with Rust Otherwise, Rust may treat python object like `cindex` object, leading to trouble. The new attribute is an integer because I expect we might need a flag field in the future. As a start we get the rust code to raise a clear TypeError. We will use the information in a smarter way in the next changesets. Differential Revision: https://phab.mercurial-scm.org/D10665

revlog: pass around the `rev` we deal with when packing/unpacking entry This will help code to compute some flag on the fly. Differential Revision: https://phab.mercurial-scm.org/D10664

changelogv2: use a dedicated version number If we want to change the format, we need a new version number. We start with that. Differential Revision: https://phab.mercurial-scm.org/D10663

changelogv2: `copies-side-data` now implies `changelogv2` To store information about file changes and copies only requires sidedata support for the changelog. So we only enables `changelogv2`. This is less impactful and should help us to ship the changeset centric copy tracing sooner. Differential Revision: https://phab.mercurial-scm.org/D10662

changelogv2: allow upgrade from and to this format Differential Revision: https://phab.mercurial-scm.org/D10661

changelogv2: introduce a "changelogv2" feature Right now, this means using revlogv2, but only for the changelog. We will have the format more unique in future changesets. Differential Revision: https://phab.mercurial-scm.org/D10660

revlogv2: drop the code related to inlined revlogv2 Tevlogv2 will not use inlined revlog. As a result we can drop the associated code. Differential Revision: https://phab.mercurial-scm.org/D10659

revlogv2: no longer attempt to use inline for new revlog revlogv2 steered away for the inline feature. Differential Revision: https://phab.mercurial-scm.org/D10658

sidedata: use the "feature" to detect that sidedata are enabled This is more versatile than relying on the revlogv2 requirements. Differential Revision: https://phab.mercurial-scm.org/D10657

revlog: compress sidedata when doing "post-pull" sidedata update All path writing sidedata are now using compression (when appropriate). Differential Revision: https://phab.mercurial-scm.org/D10656

revlog: compress sidedata in `_writeentry` When appropriate we store the sidedata in a compressed form. Differential Revision: https://phab.mercurial-scm.org/D10655

revlog: apply compression mode while reading sidedata Right now, we only store things as PLAIN, but we are ready for some compression now. Future changeset will start using some. Differential Revision: https://phab.mercurial-scm.org/D10654

revlog: introduce a compression mode for sidedata in the revlog index We will use this for compression for the sidedata payload. Differential Revision: https://phab.mercurial-scm.org/D10653

revlog: implement a "default compression" mode The revlog docker is now storing a default compression engine. When a chunk use that compression, a dedicated mode is used in the revlog entry and we can directly route it to the right decompressor. We should probably make PLAIN and DEFAULT mode the only available mode for revlogv2, but this is something for later. Differential Revision: https://phab.mercurial-scm.org/D10652

revlog: add a `_get_decompressor` method This logic is non-trivial and we will need to reuse it. Differential Revision: https://phab.mercurial-scm.org/D10651

revlog: introduce a plain compression mode That mode is simple it means the chunk contains uncompressed data and can be used directly. Differential Revision: https://phab.mercurial-scm.org/D10650

revlog: use an intermediate variable for `data` in `_chunk` This will help us to dispatch this to the right decompressor. Differential Revision: https://phab.mercurial-scm.org/D10649

revlogv2: preserve the compression mode on disk The value is still the same and still not treated, but now it exists one disk. Differential Revision: https://phab.mercurial-scm.org/D10648

revlog: use `format_version` to dispatch different version in index_get This is semantically more correct. Differential Revision: https://phab.mercurial-scm.org/D10647

revlog: add a "data compression mode" entry in the index tuple That will make it possible to keep track of compression information in the revlog index, opening the way to more efficient revision restoration (in native code, but the python usage is already defeating performance work). We start with adding a new entry to the index tuple, using a value matching the current behavior. We will introduce storage and other value in later changesets. Differential Revision: https://phab.mercurial-scm.org/D10646

revlog: fix the `null_item` attribute for V0 This is not a proper index tuple. Differential Revision: https://phab.mercurial-scm.org/D10645

revlog: create a create `null_item` attribute for V0 This value is currently wrong, but at least it is now -obviously- wrong. Differential Revision: https://phab.mercurial-scm.org/D10644

revlog: improve documentation of the entry tuple The code in revlog, and outside revlog directly use the index's entry tuple, with direct integer indexing. This is a voluntary trade off to obtains better performance from the Python code at the expense of the developers sanity. Let's at least have a clear and central documentation about what this tuple is about. Differential Revision: https://phab.mercurial-scm.org/D10643

revlog: simplify entry update logic in `rewrite_sidedata` We don't need to rebuild the full entry. So lets just gather the data we need to pass to `replace_sidedata_info`. Differential Revision: https://phab.mercurial-scm.org/D10642

revlog: simplify the replace_sidedata_info code We already know how to serialize and de-serialize and enty. So lets just do that and modify the entry tuple directly. This avoid having to duplicated binary operation in complicated code. Differential Revision: https://phab.mercurial-scm.org/D10641

revlog: use `rev` instead of `i` in replace_sidedata_info This is clearer. Differential Revision: https://phab.mercurial-scm.org/D10640

revlogv2: also keep track for the size of the "data" file This is useful to make sure we always start writing at the right location, without effort. Differential Revision: https://phab.mercurial-scm.org/D10632

revlogv2: track pending write in the docket and expose it to hooks The docket is now able to write pending data. We could have used a distinct intermediate files, however keeping everything in the same file will make it simpler to keep track of the various involved files if necessary. However it might prove more complicated for streaming clone. This will be dealt with later. Note that we lifted the stderr redirection in the test since we no longer suffer from "unkown working directory parent" message. Differential Revision: https://phab.mercurial-scm.org/D10631

revlog: move the `trypending` logic from the `changelog` to the `revlog` We move the -reading- logic for the pending's '.a' suffixed index within the revlog class. This is motivated by the fact the logic could be simpler and cleaner if directly handled by the revlog docket. Before we can do so, we need to teach the revlog code about reading "pending" changes. To be honest, we already needed some special casing of the `.a` postfix, so this does not adds much complexity. The logic around -writing- the special '00changelog.i.a' remains in the `changelog` class. Note that the revlog-v2 logic no longer use this logic. The only remaining user of the `postfix` argument is the `censor` logic. We could probably also make the revlog full aware of it (most of the code is already implemented in revlog anyway) and get rid of the `postfix` argument and logic. However this is an adventure for another time. Since we have more information, we add more, paranoid, Programming error in case we detect such "pending reader" trying to do a read (which does not happens anyways). Differential Revision: https://phab.mercurial-scm.org/D10630

revlogv2: delay the update of the changelog docket to transaction end This prevent external reader to see the transaction content before it is commited. However this also prevent the hooks to see the transaction content. We will fix this in later changesets. We have to temporarily suppress the error output of the command ran during the transaction as they sometimes get confused about unknown working directory and sometimes issue message on std-err in unspecified order. Differential Revision: https://phab.mercurial-scm.org/D10629

revlogv2: track current index size in the docket This help use to fix transaction safety on repos. See next changesets for details. Differential Revision: https://phab.mercurial-scm.org/D10628

revlogv2: also test transactionality of revlog v2 Without surprise, this is broken :-) Now that it is tested, we can fix it. Differential Revision: https://phab.mercurial-scm.org/D10627

revlog: add a new test file focussed on testing transactionally issue What we test here is currently also covered by `test-hooks.t`. However having our own test file allow for more focussed/deep testing and to cover more variant when relevant. So we create a new test file. Differential Revision: https://phab.mercurial-scm.org/D10626

revlogv2: store version information in the docket only Having it duplicated in the index was both useless and a risk of discrepancy. Differential Revision: https://phab.mercurial-scm.org/D10625

revlogv2: introduce a very basic docket file This is the first stone toward using a docket file in revlogv2. Right now the docket is very basic and only store the version number (which is -also- stored into the index file…) and the other files have fixed name. This new implementation break transactionally… but they are no test checking transactionally for revlogv2… So I take this as an opportunity to start small. They are no usage of revlogv2 outside of tests anyway. The docket keeps the `.i` naming used by previous version index to preserve a unique entry point. We could decide to use a different name and look it up first, or to fully rework this in a future "store" version. However that does not seems necessary right now. We will re-introduces transactionality (and associated testing…) in a later changesets. A long list of TODOs have been added to the relevant comment. Differential Revision: https://phab.mercurial-scm.org/D10624