Martin von Zweigbergk <martinvonz@google.com> [Tue, 18 May 2021 17:15:49 -0700] rev 47300
errors: let each Abort subclass define its error code
It's more flexible to have the error codes defined on the error types
themselves. That way extensions can easily set their own exit code. It
also means that we can reduce a bit of duplication betwen
`scmutil.callcatch()` and `chgserver.chgcmdserver.validate()`.
Differential Revision: https://phab.mercurial-scm.org/D10735
Simon Sapin <simon.sapin@octobus.net> [Wed, 19 May 2021 15:10:49 +0200] rev 47299
rust: Fix "panic message is not a string literal" warnings
These deprecation warnings would not become errors until we actively port
crates to the (not yet released) Rust 2021 edition, but fixing them anyway
reduces console output noise.
Differential Revision: https://phab.mercurial-scm.org/D10743
Simon Sapin <simon.sapin@octobus.net> [Wed, 19 May 2021 15:08:27 +0200] rev 47298
rust: Add type annotation to fix inference on Rust Nightly
When compiling with Rust Nightly, the im-rs crate silently makes use of the
experimental language feature for trait impl specialization. This apperently
changes public its APIs in subtle ways such that type inference of some user
code can fail where it succeeds when specialization is disabled.
This made Mercurial’s Rust unit tests have compilation errors on Nightly.
I have not managed to find the exactl root cause, but I wrote down my findings
so far at https://github.com/bodil/im-rs/issues/188
This adds type annotation to make unit tests rely less on type inference
and work around the issue.
Differential Revision: https://phab.mercurial-scm.org/D10742
Joerg Sonnenberger <joerg@bec.de> [Wed, 19 May 2021 13:45:34 +0200] rev 47297
recover: only apply last journal record per file (issue6423)
This got broken in 2019 when the size check was introduced. It is most
noticable when dealing with transactions that involve an inline to
non-inline revlog storage transaction. It wasn't seen as much at the
time because the in-memory journal actually de-duplicated the entry
implicity, but since 63edc384d3b7 the on-disk journal is used for
rollback as well as recover.
Differential Revision: https://phab.mercurial-scm.org/D10726
Joerg Sonnenberger <joerg@bec.de> [Wed, 19 May 2021 13:46:19 +0200] rev 47296
revlog: update data file record before index rename
When migrating from inline to non-inline data storage, the data file is
recorded initially as zero sized so that it is removed on failure. But
the record has to be updated before the index is renamed, otherwise
data is lost on rollback.
Differential Revision: https://phab.mercurial-scm.org/D10725
Joerg Sonnenberger <joerg@bec.de> [Tue, 18 May 2021 02:35:27 +0200] rev 47295
revlog: fix index computation during inline->non-inline transition
The computation in 63edc384d3b7 failed to factor in the index entries
themselve as revlog.start() doesn't count them. Found by Valtenin
Gatienbaron with a more precise test case from me.
Differential Revision: https://phab.mercurial-scm.org/D10724
Simon Sapin <simon.sapin@octobus.net> [Wed, 19 May 2021 13:15:00 +0200] rev 47294
dirstate-v2: Change the on-disk format to be tree-shaped
Nodes are stored not only for tracked files but also for their ancestor
directories. A node has "pointers" (byte count from the start of the file)
to its direct child nodes. Everything can be accessed with zero copy.
Differential Revision: https://phab.mercurial-scm.org/D10722
Simon Sapin <simon.sapin@octobus.net> [Wed, 19 May 2021 13:15:00 +0200] rev 47293
dirstate-tree: Extract into a method sorting children of a given node
A later changset will use this in another place.
This is an associated function (that Python would call static method)
instead of a free function so it doesn’t need to be imported separately.
It’s on `Node` rather than `ChildNodes` because the latter is a type alias
to an external type (`HashMap`) so that would require an extension trait
which needs to be imported separately.
Differential Revision: https://phab.mercurial-scm.org/D10721
Simon Sapin <simon.sapin@octobus.net> [Wed, 19 May 2021 13:15:00 +0200] rev 47292
dirstate-v2: Add a variant of some tests, that uses the new format
With this, the new format receives some testing every time someone runs tests
with Rust extensions enabled, including on CI.
Differential Revision: https://phab.mercurial-scm.org/D10720
Simon Sapin <simon.sapin@octobus.net> [Wed, 19 May 2021 13:15:00 +0200] rev 47291
dirstate-v2: Change the on-disk format when the requirement is enabled
For now, the format is the same except with an additional marker at the start.
This marker is redundant: for existing repositories it is `.hg/requires` that
determines which format to use. For new repositories, it is the new
`format.exp-dirstate-v2` config. There is no upgrade or downgrade so far.
Most of the changes are about plumbing a boolean through layers of APIs to
indicate which format should be used.
Differential Revision: https://phab.mercurial-scm.org/D10719
Simon Sapin <simon.sapin@octobus.net> [Wed, 19 May 2021 13:15:00 +0200] rev 47290
dirstate-v2: Update the expected output of some tests for new requirement
Fix most test failures (except in test-narrow-debugrebuilddirstate.t and
test-upgrade-repo.t) caused by the new entry in config or in .hg/requires
when running `run-tests.py --extra-config-opt format.exp-dirstate-v2=1`
There is no CI so far for this configuration.
Differential Revision: https://phab.mercurial-scm.org/D10718
Simon Sapin <simon.sapin@octobus.net> [Wed, 19 May 2021 13:14:59 +0200] rev 47289
tests: More cleanly separate expected hexdump output
There are more lines that differ than are in common, and dirstate-v2
will complicate that further.
Differential Revision: https://phab.mercurial-scm.org/D10717
Simon Sapin <simon.sapin@octobus.net> [Wed, 19 May 2021 13:14:59 +0200] rev 47288
dirstate-v2: Add a new experimental `exp-dirstate-v2` repository requirement
This requirement is added to `.hg/requires` when creating a new repository
if Rust extensions are enabled and the `format.exp-dirstate-v2` config is set.
Nothing yet changes based on this requirement, but its mere presence affects
some tests (for example if they print `.hg/requires`). The next two changesets
update tests’ expected outputs accordingly.
There is no CI so far that enables this configuration.
Differential Revision: https://phab.mercurial-scm.org/D10716
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 16 Apr 2021 00:16:43 +0200] rev 47287
urlutil: provide some information about "bad url" when processing `pushurl`
It appears pushurl only support `<proto>://` entries. This is not obvious and
can lead to obscure error. We make the error less obscure as a start..
Differential Revision: https://phab.mercurial-scm.org/D10455
Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 11 Apr 2021 19:19:34 +0200] rev 47286
revset: document the `outgoing` behavior if the path resolve to multiple urls
Differential Revision: https://phab.mercurial-scm.org/D10454
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 16 Apr 2021 00:19:04 +0200] rev 47285
multi-urls: document the feature
We have a feature lets make it visible to people.
Differential Revision: https://phab.mercurial-scm.org/D10453
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 16 Apr 2021 00:16:47 +0200] rev 47284
multi-urls: add a boolean suboption that unlock path specification as list
When this option is set, a list of patch can be specifed as value for `[paths]`
entries. For the command who support it, this behave the same as providing
multiple destination of the command line.
Differential Revision: https://phab.mercurial-scm.org/D10452
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 15 Apr 2021 20:13:29 +0200] rev 47283
urlutil: move url "fixing" at the time of `ui.paths` initialization
Doing such fixing at the time is simpler and will be necessary to deal with
urls list in a sane manner. It also reduce the size of fix-config which is
always better.
I wish we could get ride of the hackish way to pass the root around, I suspect
that the `root` variable could be stored as part of the config value, along side
the source. However getting to the end of this `root` business is a far too
large detours to make now.
The test change to `tests/test-hgrc.t` and `test-config.t` are expectied since
we are not longer altering the config itself, but the way it is interpreted when
building path. This seems more correct.
I also added a couple of test call to `test-config.t` and `test-globalopts.t` to
clarify that the expanding process is properly happening a the right time.
Differential Revision: https://phab.mercurial-scm.org/D10451
Martin von Zweigbergk <martinvonz@google.com> [Mon, 17 May 2021 15:15:44 -0700] rev 47282
rewriteutil: add pointer to help text when rewrite would cause divergence
The evolve extension's version of the hint has this pointer. I missed
it when I moved it to core.
Differential Revision: https://phab.mercurial-scm.org/D10723
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 04 May 2021 11:20:10 +0200] rev 47281
changelogv2: use a dedicated on disk format for changelogv2
We drop two unused entry. This is mostly a proof of concept before starting to
actually rework the format.
Differential Revision: https://phab.mercurial-scm.org/D10667
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 04 May 2021 14:18:06 +0200] rev 47280
revlog: do not call Rust code if the index is not compatible with it
This will avoid hitting the TypeError we defined in the previous changesets.
This is the simplest fix but not the most elegant.
Ideally we would teach the Rust code to use any kind of revlog. However this is
an adventure for another time.
Differential Revision: https://phab.mercurial-scm.org/D10666
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 04 May 2021 14:16:26 +0200] rev 47279
revlog: signal which revlog index are compatible with Rust
Otherwise, Rust may treat python object like `cindex` object, leading to
trouble. The new attribute is an integer because I expect we might need a flag
field in the future.
As a start we get the rust code to raise a clear TypeError. We will use the
information in a smarter way in the next changesets.
Differential Revision: https://phab.mercurial-scm.org/D10665
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 04 May 2021 11:19:48 +0200] rev 47278
revlog: pass around the `rev` we deal with when packing/unpacking entry
This will help code to compute some flag on the fly.
Differential Revision: https://phab.mercurial-scm.org/D10664
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 29 Mar 2021 20:46:56 +0200] rev 47277
changelogv2: use a dedicated version number
If we want to change the format, we need a new version number. We start with
that.
Differential Revision: https://phab.mercurial-scm.org/D10663
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 04 May 2021 03:05:52 +0200] rev 47276
changelogv2: `copies-side-data` now implies `changelogv2`
To store information about file changes and copies only requires sidedata
support for the changelog. So we only enables `changelogv2`. This is less
impactful and should help us to ship the changeset centric copy tracing sooner.
Differential Revision: https://phab.mercurial-scm.org/D10662
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 04 May 2021 04:32:09 +0200] rev 47275
changelogv2: allow upgrade from and to this format
Differential Revision: https://phab.mercurial-scm.org/D10661
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 29 Mar 2021 22:40:54 +0200] rev 47274
changelogv2: introduce a "changelogv2" feature
Right now, this means using revlogv2, but only for the changelog. We will have
the format more unique in future changesets.
Differential Revision: https://phab.mercurial-scm.org/D10660
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 04 May 2021 05:18:57 +0200] rev 47273
revlogv2: drop the code related to inlined revlogv2
Tevlogv2 will not use inlined revlog. As a result we can drop the associated
code.
Differential Revision: https://phab.mercurial-scm.org/D10659
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 04 May 2021 02:55:56 +0200] rev 47272
revlogv2: no longer attempt to use inline for new revlog
revlogv2 steered away for the inline feature.
Differential Revision: https://phab.mercurial-scm.org/D10658
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 04 May 2021 02:52:24 +0200] rev 47271
sidedata: use the "feature" to detect that sidedata are enabled
This is more versatile than relying on the revlogv2 requirements.
Differential Revision: https://phab.mercurial-scm.org/D10657
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 23:40:05 +0200] rev 47270
revlog: compress sidedata when doing "post-pull" sidedata update
All path writing sidedata are now using compression (when appropriate).
Differential Revision: https://phab.mercurial-scm.org/D10656
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 23:14:48 +0200] rev 47269
revlog: compress sidedata in `_writeentry`
When appropriate we store the sidedata in a compressed form.
Differential Revision: https://phab.mercurial-scm.org/D10655
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 22:48:31 +0200] rev 47268
revlog: apply compression mode while reading sidedata
Right now, we only store things as PLAIN, but we are ready for some compression
now. Future changeset will start using some.
Differential Revision: https://phab.mercurial-scm.org/D10654
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 21:34:02 +0200] rev 47267
revlog: introduce a compression mode for sidedata in the revlog index
We will use this for compression for the sidedata payload.
Differential Revision: https://phab.mercurial-scm.org/D10653
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 21:13:24 +0200] rev 47266
revlog: implement a "default compression" mode
The revlog docker is now storing a default compression engine. When a chunk use
that compression, a dedicated mode is used in the revlog entry and we can
directly route it to the right decompressor.
We should probably make PLAIN and DEFAULT mode the only available mode for
revlogv2, but this is something for later.
Differential Revision: https://phab.mercurial-scm.org/D10652
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 21:04:55 +0200] rev 47265
revlog: add a `_get_decompressor` method
This logic is non-trivial and we will need to reuse it.
Differential Revision: https://phab.mercurial-scm.org/D10651
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 19:46:25 +0200] rev 47264
revlog: introduce a plain compression mode
That mode is simple it means the chunk contains uncompressed data and can be
used directly.
Differential Revision: https://phab.mercurial-scm.org/D10650
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 20:22:34 +0200] rev 47263
revlog: use an intermediate variable for `data` in `_chunk`
This will help us to dispatch this to the right decompressor.
Differential Revision: https://phab.mercurial-scm.org/D10649
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 18:41:51 +0200] rev 47262
revlogv2: preserve the compression mode on disk
The value is still the same and still not treated, but now it exists one disk.
Differential Revision: https://phab.mercurial-scm.org/D10648
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 04 May 2021 01:49:37 +0200] rev 47261
revlog: use `format_version` to dispatch different version in index_get
This is semantically more correct.
Differential Revision: https://phab.mercurial-scm.org/D10647
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 18:19:16 +0200] rev 47260
revlog: add a "data compression mode" entry in the index tuple
That will make it possible to keep track of compression information in the
revlog index, opening the way to more efficient revision restoration (in native
code, but the python usage is already defeating performance work).
We start with adding a new entry to the index tuple, using a value matching the
current behavior. We will introduce storage and other value in later changesets.
Differential Revision: https://phab.mercurial-scm.org/D10646
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 04 May 2021 01:15:03 +0200] rev 47259
revlog: fix the `null_item` attribute for V0
This is not a proper index tuple.
Differential Revision: https://phab.mercurial-scm.org/D10645
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 04 May 2021 01:13:53 +0200] rev 47258
revlog: create a create `null_item` attribute for V0
This value is currently wrong, but at least it is now -obviously- wrong.
Differential Revision: https://phab.mercurial-scm.org/D10644
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 16:52:38 +0200] rev 47257
revlog: improve documentation of the entry tuple
The code in revlog, and outside revlog directly use the index's entry tuple,
with direct integer indexing. This is a voluntary trade off to obtains better
performance from the Python code at the expense of the developers sanity.
Let's at least have a clear and central documentation about what this tuple is
about.
Differential Revision: https://phab.mercurial-scm.org/D10643
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 23:45:05 +0200] rev 47256
revlog: simplify entry update logic in `rewrite_sidedata`
We don't need to rebuild the full entry. So lets just gather the data we need
to pass to `replace_sidedata_info`.
Differential Revision: https://phab.mercurial-scm.org/D10642
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 04 May 2021 00:02:48 +0200] rev 47255
revlog: simplify the replace_sidedata_info code
We already know how to serialize and de-serialize and enty. So lets just do that
and modify the entry tuple directly.
This avoid having to duplicated binary operation in complicated code.
Differential Revision: https://phab.mercurial-scm.org/D10641
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 23:59:15 +0200] rev 47254
revlog: use `rev` instead of `i` in replace_sidedata_info
This is clearer.
Differential Revision: https://phab.mercurial-scm.org/D10640
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:35:35 +0200] rev 47253
revlogv2: also keep track for the size of the "data" file
This is useful to make sure we always start writing at the right location,
without effort.
Differential Revision: https://phab.mercurial-scm.org/D10632
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:35:25 +0200] rev 47252
revlogv2: track pending write in the docket and expose it to hooks
The docket is now able to write pending data. We could have used a distinct
intermediate files, however keeping everything in the same file will make it
simpler to keep track of the various involved files if necessary.
However it might prove more complicated for streaming clone. This will be dealt
with later.
Note that we lifted the stderr redirection in the test since we no longer suffer
from "unkown working directory parent" message.
Differential Revision: https://phab.mercurial-scm.org/D10631
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:35:14 +0200] rev 47251
revlog: move the `trypending` logic from the `changelog` to the `revlog`
We move the -reading- logic for the pending's '.a' suffixed index within the
revlog class. This is motivated by the fact the logic could be simpler and
cleaner if directly handled by the revlog docket. Before we can do so, we need
to teach the revlog code about reading "pending" changes. To be honest, we
already needed some special casing of the `.a` postfix, so this does not adds
much complexity.
The logic around -writing- the special '00changelog.i.a' remains in the
`changelog` class. Note that the revlog-v2 logic no longer use this logic.
The only remaining user of the `postfix` argument is the `censor` logic. We
could probably also make the revlog full aware of it (most of the code is
already implemented in revlog anyway) and get rid of the `postfix` argument and
logic. However this is an adventure for another time.
Since we have more information, we add more, paranoid, Programming error in case
we detect such "pending reader" trying to do a read (which does not happens
anyways).
Differential Revision: https://phab.mercurial-scm.org/D10630
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:35:02 +0200] rev 47250
revlogv2: delay the update of the changelog docket to transaction end
This prevent external reader to see the transaction content before it is
commited. However this also prevent the hooks to see the transaction content.
We will fix this in later changesets.
We have to temporarily suppress the error output of the command ran during the
transaction as they sometimes get confused about unknown working directory and
sometimes issue message on std-err in unspecified order.
Differential Revision: https://phab.mercurial-scm.org/D10629
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:34:52 +0200] rev 47249
revlogv2: track current index size in the docket
This help use to fix transaction safety on repos. See next changesets for details.
Differential Revision: https://phab.mercurial-scm.org/D10628
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:34:41 +0200] rev 47248
revlogv2: also test transactionality of revlog v2
Without surprise, this is broken :-)
Now that it is tested, we can fix it.
Differential Revision: https://phab.mercurial-scm.org/D10627
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:34:31 +0200] rev 47247
revlog: add a new test file focussed on testing transactionally issue
What we test here is currently also covered by `test-hooks.t`. However having
our own test file allow for more focussed/deep testing and to cover more variant
when relevant. So we create a new test file.
Differential Revision: https://phab.mercurial-scm.org/D10626
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:34:21 +0200] rev 47246
revlogv2: store version information in the docket only
Having it duplicated in the index was both useless and a risk of discrepancy.
Differential Revision: https://phab.mercurial-scm.org/D10625
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:34:11 +0200] rev 47245
revlogv2: introduce a very basic docket file
This is the first stone toward using a docket file in revlogv2. Right now the
docket is very basic and only store the version number (which is -also- stored
into the index file…) and the other files have fixed name. This new
implementation break transactionally… but they are no test checking
transactionally for revlogv2… So I take this as an opportunity to start small.
They are no usage of revlogv2 outside of tests anyway.
The docket keeps the `.i` naming used by previous version index to preserve a
unique entry point. We could decide to use a different name and look it up
first, or to fully rework this in a future "store" version. However that does
not seems necessary right now.
We will re-introduces transactionality (and associated testing…) in a later
changesets.
A long list of TODOs have been added to the relevant comment.
Differential Revision: https://phab.mercurial-scm.org/D10624