Matt Harbison <matt_harbison@yahoo.com> [Thu, 20 May 2021 19:59:03 -0400] rev 47325
tests: partially fix test-transaction-rollback-on-revlog-split.t on Windows
For whatever reason, the extra space causes different output on Windows:
$ printf '% 20d' '1' | xxd -
0000000: 2020 2020 2020 2020 2020 2020 2020 2020
0000010: 2020 2020 31 1
vs WSL:
/mnt/c/Users/Matt/hg/tests$ printf '% 20d' '1' | xxd -
00000000: 2020 2020 2020 2020 2020 2020 2020 2020
00000010: 2020 2031
Without the space, they both agree the last word is 2031. There are other
failures on Windows that I don't understand that are introduced by 46b828b85eb7
and 18415fc918a1.
Differential Revision: https://phab.mercurial-scm.org/D10763
Matt Harbison <matt_harbison@yahoo.com> [Thu, 20 May 2021 19:30:56 -0400] rev 47324
tests: fix test-transaction-safety.t on Windows
Mostly this is about `C:\dir\sub` style paths being turned into `C:dirsub` by
MSYS, and not being about to execute shell scripts in hooks directly. See also
a12819559ccb::9e3979a25bfe.
Differential Revision: https://phab.mercurial-scm.org/D10762
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 19 May 2021 19:57:55 +0200] rev 47323
docket: move the uid logic in the `revlogutils.docket` module
We want to use it for revlog-v2 (& Co), it seems more logical to have the logic
lives inside the `docket` file than the `nodemap` file.
Differential Revision: https://phab.mercurial-scm.org/D10755
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 19 May 2021 17:12:06 +0200] rev 47322
persistent-nodemap: use the intended uuid size
I overlooked this in the initial implementation. The format already supporting
any uuid size so, nothing horrible should happens.
Differential Revision: https://phab.mercurial-scm.org/D10754
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 19 May 2021 17:19:46 +0200] rev 47321
persistent-nodemap: add a way to make the picked uid predictable
This will make the test less volatile and therefore easier to maintain. I am
setting this is for the persistent nodemap docket first, but my target is to use
similar logic for the revlogv2 (and co) test.
For persisent nodemap, they are too many different in how each test variants (with rust, without rust, etc) needs new uid for this be really useful (right now). However it will be directly useful for the revlogv2 logic.
Differential Revision: https://phab.mercurial-scm.org/D10753
Matt Harbison <matt_harbison@yahoo.com> [Thu, 06 May 2021 19:39:19 -0400] rev 47320
run-tests: disable color in 'auto' mode on Windows if stdout is redirected
Differential Revision: https://phab.mercurial-scm.org/D10761
Matt Harbison <matt_harbison@yahoo.com> [Thu, 06 May 2021 18:52:08 -0400] rev 47319
run-tests: enable color on Windows
In setting up the CI for Windows on heptapod, I noticed it was complaining about
color not being enabled because pygments wasn't installed- even though it was.
I had initially disabled color on Windows when using Windows 7, because that
didn't understand ANSI color codes and made a mess of the output. But now that
it's been unsupported for over a year, I don't think we should care about it
either.
It's admittedly a hack to depend on Mercurial proper to enable color support in
the terminal, but I didn't feel like duplicating that code. I'm under the
impression that 3rd party stuff is supposed to use this runner in the Mercurial
repo instead of using their own copy, so I think it's safe to assume the
Mercurial code is available. If it's not, it won't break anything.
Differential Revision: https://phab.mercurial-scm.org/D10760
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> [Sun, 16 May 2021 10:57:14 -0400] rev 47318
transaction: trivial refactoring
Differential Revision: https://phab.mercurial-scm.org/D10714
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> [Tue, 18 May 2021 23:36:21 -0400] rev 47317
test: remove some unnecessary dependency on repo format
Differential Revision: https://phab.mercurial-scm.org/D10757
Raphaël Gomès <rgomes@octobus.net> [Fri, 21 May 2021 17:37:53 +0200] rev 47316
hg-cpython: fix new occuring TypeError
dd339191f2dc introduced a minor refactoring of error types that highlighted
a fragile error creation mechanism that was in place in the Rust `MixedIndex`.
`PyErr::from_instance` also accepts a Python class, as long as it's an
`Exception` class. Before the aforementioned commit, we never ran into a case
where this duck-typing mechanism failed. We rectify this behavior by doing the
instantiation ourselves.
Differential Revision: https://phab.mercurial-scm.org/D10764
Martin von Zweigbergk <martinvonz@google.com> [Thu, 20 May 2021 08:15:57 -0700] rev 47315
errors: make exit codes class variables instead
Kyle pointed out to me that we can simply make the exit codes class
variables. Python provides some magic for making them accessible as
instance variables.
This also makes it easier to let subclasses of existing errors
override the exit codes by letting them simply define their own values
as class variables. That means that there's no need to pass them into
the superclass's constructor arguments, so the superclass doesn't need
to expose the them as arguments. (Making a subclass set a different
exit code for a subclass of `StorageError` was actually the goal with
my recent series.)
Differential Revision: https://phab.mercurial-scm.org/D10758
Charles Chamberlain <cchamberlain@janestreet.com> [Tue, 18 May 2021 13:08:42 -0400] rev 47314
narrow: widen when files are excluded by sparse and not included by narrow
In a repo where some directories are included by narrow and the complement are
excluded by sparse, it was previously impossible to widen either because trying
to widen narrow would complain that the requested files are outside the sparse
checkout and trying to widen sparse would complain that the requested files are
outside the narrow checkout.
This changes the `hg tracked --addinclude` command to only actually update any
newly accessible files in the dirstate if they are also accessible via sparse.
Differential Revision: https://phab.mercurial-scm.org/D10734
Augie Fackler <augie@google.com> [Tue, 18 May 2021 13:34:06 -0400] rev 47313
hghave: make error output easier to diagnose
I had a typo that meant the new bash check was throwing an exception
(due to a missing argument), but it was very hard to diagnose without
this change.
Differential Revision: https://phab.mercurial-scm.org/D10733
Augie Fackler <augie@google.com> [Tue, 18 May 2021 13:24:41 -0400] rev 47312
tests: add req on bash for test-transaction-rollback-on-sigpipe (issue6429)
I think we could work around this by rewriting the helper scripts in
Python, but I don't want to deal with that now and this should prevent
failures due to a lack of bash.
Differential Revision: https://phab.mercurial-scm.org/D10732
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 17 May 2021 15:24:46 +0200] rev 47311
updatecaches: deprecate the `full` argument
Now that all users were migrated, we can use deprecate the old way. This would
give potential extensions code a heads up on the API change.
Differential Revision: https://phab.mercurial-scm.org/D10731
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 17 May 2021 15:42:18 +0200] rev 47310
updatecaches: use the `caches` argument instead of a special `full` value
After a clone we want to update most cachem, but not exactly all of them. We
can now cleanly express this.
Differential Revision: https://phab.mercurial-scm.org/D10730
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 17 May 2021 15:27:29 +0200] rev 47309
updatecaches: use the caches argument in `hg debugupdatecaches`
This is the new way.
Differential Revision: https://phab.mercurial-scm.org/D10729
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 17 May 2021 14:45:16 +0200] rev 47308
updatecaches: adds a `caches` parameters to `repo.updatecaches`
It will superseed the `full` parameters (and its `post-clone` variant from
stable). Various caller will be updated in the rest of this series.
Differential Revision: https://phab.mercurial-scm.org/D10728
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 17 May 2021 14:41:09 +0200] rev 47307
updatecaches: introduce a set of constants to control which are updated
Passing around a set of constant to select what need warming will be cleaner
and more flexible. We did not changed the API yet, as this changes is already
large enough. In the rest of the rest we will change more code to actually use
this constants (or more realistically pre-defined set of constant directly)
Differential Revision: https://phab.mercurial-scm.org/D10727
Martin von Zweigbergk <martinvonz@google.com> [Tue, 18 May 2021 21:50:09 -0700] rev 47306
errors: make StorageError subclass Error, attaching an exit code to it
Differential Revision: https://phab.mercurial-scm.org/D10741
Martin von Zweigbergk <martinvonz@google.com> [Tue, 18 May 2021 19:33:09 -0700] rev 47305
revlog: avoid raising no-arg RevlogError for internal flow control
I'm about to make RevlogError require a `message` argument and this
code was failing. This patch refactors it to not raise an exception
for intra-function flow control.
Differential Revision: https://phab.mercurial-scm.org/D10740
Martin von Zweigbergk <martinvonz@google.com> [Tue, 18 May 2021 21:45:59 -0700] rev 47304
errors: catch the new Error class in scmutil and chgserver
Differential Revision: https://phab.mercurial-scm.org/D10739
Martin von Zweigbergk <martinvonz@google.com> [Tue, 18 May 2021 21:32:12 -0700] rev 47303
errors: create superclass for Abort exception
I'd like to let extensions subclass `StorageError` to define a custom
exit code. However, `StorageError` does not extend `Abort` (which is
where the exit code currently lives), and it seems that it's not
supposed to either (`StorageError` seems to be for lower-level errors
and `Abort` is for command-level errors). This patch therefore
extracts all the code from `Abort` into a new `Error` class, which
I'll soon make `StorageError` also extend.
Differential Revision: https://phab.mercurial-scm.org/D10738
Martin von Zweigbergk <martinvonz@google.com> [Tue, 18 May 2021 22:07:16 -0700] rev 47302
errors: make InterventionRequired subclass Abort
The docstring for `Abort` says that it's for errors raised by commands
and `InterventionRequired` is definitely something raised by commands,
so it seems that it should be an `Abort`. This patch makes it so. It
adds a `coarse_exit_code` (in addition to the already existing
`detailed_exit_code`) to `Abort` to achieve that, since
`InterventionRequired` should result in a special exit code even when
the `ui.detailed-exit-code` config is not set.
Differential Revision: https://phab.mercurial-scm.org/D10737
Martin von Zweigbergk <martinvonz@google.com> [Tue, 18 May 2021 21:58:12 -0700] rev 47301
errors: move Abort earlier, so more exceptions can subclass it
I'd like to make at least `InterventionRequired` subclass `Abort` and
Python requires the superclass to be defined before the subtype.
Differential Revision: https://phab.mercurial-scm.org/D10736
Martin von Zweigbergk <martinvonz@google.com> [Tue, 18 May 2021 17:15:49 -0700] rev 47300
errors: let each Abort subclass define its error code
It's more flexible to have the error codes defined on the error types
themselves. That way extensions can easily set their own exit code. It
also means that we can reduce a bit of duplication betwen
`scmutil.callcatch()` and `chgserver.chgcmdserver.validate()`.
Differential Revision: https://phab.mercurial-scm.org/D10735
Simon Sapin <simon.sapin@octobus.net> [Wed, 19 May 2021 15:10:49 +0200] rev 47299
rust: Fix "panic message is not a string literal" warnings
These deprecation warnings would not become errors until we actively port
crates to the (not yet released) Rust 2021 edition, but fixing them anyway
reduces console output noise.
Differential Revision: https://phab.mercurial-scm.org/D10743
Simon Sapin <simon.sapin@octobus.net> [Wed, 19 May 2021 15:08:27 +0200] rev 47298
rust: Add type annotation to fix inference on Rust Nightly
When compiling with Rust Nightly, the im-rs crate silently makes use of the
experimental language feature for trait impl specialization. This apperently
changes public its APIs in subtle ways such that type inference of some user
code can fail where it succeeds when specialization is disabled.
This made Mercurial’s Rust unit tests have compilation errors on Nightly.
I have not managed to find the exactl root cause, but I wrote down my findings
so far at https://github.com/bodil/im-rs/issues/188
This adds type annotation to make unit tests rely less on type inference
and work around the issue.
Differential Revision: https://phab.mercurial-scm.org/D10742
Joerg Sonnenberger <joerg@bec.de> [Wed, 19 May 2021 13:45:34 +0200] rev 47297
recover: only apply last journal record per file (issue6423)
This got broken in 2019 when the size check was introduced. It is most
noticable when dealing with transactions that involve an inline to
non-inline revlog storage transaction. It wasn't seen as much at the
time because the in-memory journal actually de-duplicated the entry
implicity, but since 63edc384d3b7 the on-disk journal is used for
rollback as well as recover.
Differential Revision: https://phab.mercurial-scm.org/D10726
Joerg Sonnenberger <joerg@bec.de> [Wed, 19 May 2021 13:46:19 +0200] rev 47296
revlog: update data file record before index rename
When migrating from inline to non-inline data storage, the data file is
recorded initially as zero sized so that it is removed on failure. But
the record has to be updated before the index is renamed, otherwise
data is lost on rollback.
Differential Revision: https://phab.mercurial-scm.org/D10725
Joerg Sonnenberger <joerg@bec.de> [Tue, 18 May 2021 02:35:27 +0200] rev 47295
revlog: fix index computation during inline->non-inline transition
The computation in 63edc384d3b7 failed to factor in the index entries
themselve as revlog.start() doesn't count them. Found by Valtenin
Gatienbaron with a more precise test case from me.
Differential Revision: https://phab.mercurial-scm.org/D10724
Simon Sapin <simon.sapin@octobus.net> [Wed, 19 May 2021 13:15:00 +0200] rev 47294
dirstate-v2: Change the on-disk format to be tree-shaped
Nodes are stored not only for tracked files but also for their ancestor
directories. A node has "pointers" (byte count from the start of the file)
to its direct child nodes. Everything can be accessed with zero copy.
Differential Revision: https://phab.mercurial-scm.org/D10722
Simon Sapin <simon.sapin@octobus.net> [Wed, 19 May 2021 13:15:00 +0200] rev 47293
dirstate-tree: Extract into a method sorting children of a given node
A later changset will use this in another place.
This is an associated function (that Python would call static method)
instead of a free function so it doesn’t need to be imported separately.
It’s on `Node` rather than `ChildNodes` because the latter is a type alias
to an external type (`HashMap`) so that would require an extension trait
which needs to be imported separately.
Differential Revision: https://phab.mercurial-scm.org/D10721
Simon Sapin <simon.sapin@octobus.net> [Wed, 19 May 2021 13:15:00 +0200] rev 47292
dirstate-v2: Add a variant of some tests, that uses the new format
With this, the new format receives some testing every time someone runs tests
with Rust extensions enabled, including on CI.
Differential Revision: https://phab.mercurial-scm.org/D10720
Simon Sapin <simon.sapin@octobus.net> [Wed, 19 May 2021 13:15:00 +0200] rev 47291
dirstate-v2: Change the on-disk format when the requirement is enabled
For now, the format is the same except with an additional marker at the start.
This marker is redundant: for existing repositories it is `.hg/requires` that
determines which format to use. For new repositories, it is the new
`format.exp-dirstate-v2` config. There is no upgrade or downgrade so far.
Most of the changes are about plumbing a boolean through layers of APIs to
indicate which format should be used.
Differential Revision: https://phab.mercurial-scm.org/D10719
Simon Sapin <simon.sapin@octobus.net> [Wed, 19 May 2021 13:15:00 +0200] rev 47290
dirstate-v2: Update the expected output of some tests for new requirement
Fix most test failures (except in test-narrow-debugrebuilddirstate.t and
test-upgrade-repo.t) caused by the new entry in config or in .hg/requires
when running `run-tests.py --extra-config-opt format.exp-dirstate-v2=1`
There is no CI so far for this configuration.
Differential Revision: https://phab.mercurial-scm.org/D10718
Simon Sapin <simon.sapin@octobus.net> [Wed, 19 May 2021 13:14:59 +0200] rev 47289
tests: More cleanly separate expected hexdump output
There are more lines that differ than are in common, and dirstate-v2
will complicate that further.
Differential Revision: https://phab.mercurial-scm.org/D10717
Simon Sapin <simon.sapin@octobus.net> [Wed, 19 May 2021 13:14:59 +0200] rev 47288
dirstate-v2: Add a new experimental `exp-dirstate-v2` repository requirement
This requirement is added to `.hg/requires` when creating a new repository
if Rust extensions are enabled and the `format.exp-dirstate-v2` config is set.
Nothing yet changes based on this requirement, but its mere presence affects
some tests (for example if they print `.hg/requires`). The next two changesets
update tests’ expected outputs accordingly.
There is no CI so far that enables this configuration.
Differential Revision: https://phab.mercurial-scm.org/D10716
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 16 Apr 2021 00:16:43 +0200] rev 47287
urlutil: provide some information about "bad url" when processing `pushurl`
It appears pushurl only support `<proto>://` entries. This is not obvious and
can lead to obscure error. We make the error less obscure as a start..
Differential Revision: https://phab.mercurial-scm.org/D10455
Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 11 Apr 2021 19:19:34 +0200] rev 47286
revset: document the `outgoing` behavior if the path resolve to multiple urls
Differential Revision: https://phab.mercurial-scm.org/D10454
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 16 Apr 2021 00:19:04 +0200] rev 47285
multi-urls: document the feature
We have a feature lets make it visible to people.
Differential Revision: https://phab.mercurial-scm.org/D10453
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 16 Apr 2021 00:16:47 +0200] rev 47284
multi-urls: add a boolean suboption that unlock path specification as list
When this option is set, a list of patch can be specifed as value for `[paths]`
entries. For the command who support it, this behave the same as providing
multiple destination of the command line.
Differential Revision: https://phab.mercurial-scm.org/D10452
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 15 Apr 2021 20:13:29 +0200] rev 47283
urlutil: move url "fixing" at the time of `ui.paths` initialization
Doing such fixing at the time is simpler and will be necessary to deal with
urls list in a sane manner. It also reduce the size of fix-config which is
always better.
I wish we could get ride of the hackish way to pass the root around, I suspect
that the `root` variable could be stored as part of the config value, along side
the source. However getting to the end of this `root` business is a far too
large detours to make now.
The test change to `tests/test-hgrc.t` and `test-config.t` are expectied since
we are not longer altering the config itself, but the way it is interpreted when
building path. This seems more correct.
I also added a couple of test call to `test-config.t` and `test-globalopts.t` to
clarify that the expanding process is properly happening a the right time.
Differential Revision: https://phab.mercurial-scm.org/D10451
Martin von Zweigbergk <martinvonz@google.com> [Mon, 17 May 2021 15:15:44 -0700] rev 47282
rewriteutil: add pointer to help text when rewrite would cause divergence
The evolve extension's version of the hint has this pointer. I missed
it when I moved it to core.
Differential Revision: https://phab.mercurial-scm.org/D10723
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 04 May 2021 11:20:10 +0200] rev 47281
changelogv2: use a dedicated on disk format for changelogv2
We drop two unused entry. This is mostly a proof of concept before starting to
actually rework the format.
Differential Revision: https://phab.mercurial-scm.org/D10667
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 04 May 2021 14:18:06 +0200] rev 47280
revlog: do not call Rust code if the index is not compatible with it
This will avoid hitting the TypeError we defined in the previous changesets.
This is the simplest fix but not the most elegant.
Ideally we would teach the Rust code to use any kind of revlog. However this is
an adventure for another time.
Differential Revision: https://phab.mercurial-scm.org/D10666
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 04 May 2021 14:16:26 +0200] rev 47279
revlog: signal which revlog index are compatible with Rust
Otherwise, Rust may treat python object like `cindex` object, leading to
trouble. The new attribute is an integer because I expect we might need a flag
field in the future.
As a start we get the rust code to raise a clear TypeError. We will use the
information in a smarter way in the next changesets.
Differential Revision: https://phab.mercurial-scm.org/D10665
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 04 May 2021 11:19:48 +0200] rev 47278
revlog: pass around the `rev` we deal with when packing/unpacking entry
This will help code to compute some flag on the fly.
Differential Revision: https://phab.mercurial-scm.org/D10664
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 29 Mar 2021 20:46:56 +0200] rev 47277
changelogv2: use a dedicated version number
If we want to change the format, we need a new version number. We start with
that.
Differential Revision: https://phab.mercurial-scm.org/D10663
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 04 May 2021 03:05:52 +0200] rev 47276
changelogv2: `copies-side-data` now implies `changelogv2`
To store information about file changes and copies only requires sidedata
support for the changelog. So we only enables `changelogv2`. This is less
impactful and should help us to ship the changeset centric copy tracing sooner.
Differential Revision: https://phab.mercurial-scm.org/D10662
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 04 May 2021 04:32:09 +0200] rev 47275
changelogv2: allow upgrade from and to this format
Differential Revision: https://phab.mercurial-scm.org/D10661
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 29 Mar 2021 22:40:54 +0200] rev 47274
changelogv2: introduce a "changelogv2" feature
Right now, this means using revlogv2, but only for the changelog. We will have
the format more unique in future changesets.
Differential Revision: https://phab.mercurial-scm.org/D10660
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 04 May 2021 05:18:57 +0200] rev 47273
revlogv2: drop the code related to inlined revlogv2
Tevlogv2 will not use inlined revlog. As a result we can drop the associated
code.
Differential Revision: https://phab.mercurial-scm.org/D10659
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 04 May 2021 02:55:56 +0200] rev 47272
revlogv2: no longer attempt to use inline for new revlog
revlogv2 steered away for the inline feature.
Differential Revision: https://phab.mercurial-scm.org/D10658
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 04 May 2021 02:52:24 +0200] rev 47271
sidedata: use the "feature" to detect that sidedata are enabled
This is more versatile than relying on the revlogv2 requirements.
Differential Revision: https://phab.mercurial-scm.org/D10657
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 23:40:05 +0200] rev 47270
revlog: compress sidedata when doing "post-pull" sidedata update
All path writing sidedata are now using compression (when appropriate).
Differential Revision: https://phab.mercurial-scm.org/D10656
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 23:14:48 +0200] rev 47269
revlog: compress sidedata in `_writeentry`
When appropriate we store the sidedata in a compressed form.
Differential Revision: https://phab.mercurial-scm.org/D10655
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 22:48:31 +0200] rev 47268
revlog: apply compression mode while reading sidedata
Right now, we only store things as PLAIN, but we are ready for some compression
now. Future changeset will start using some.
Differential Revision: https://phab.mercurial-scm.org/D10654
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 21:34:02 +0200] rev 47267
revlog: introduce a compression mode for sidedata in the revlog index
We will use this for compression for the sidedata payload.
Differential Revision: https://phab.mercurial-scm.org/D10653
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 21:13:24 +0200] rev 47266
revlog: implement a "default compression" mode
The revlog docker is now storing a default compression engine. When a chunk use
that compression, a dedicated mode is used in the revlog entry and we can
directly route it to the right decompressor.
We should probably make PLAIN and DEFAULT mode the only available mode for
revlogv2, but this is something for later.
Differential Revision: https://phab.mercurial-scm.org/D10652
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 21:04:55 +0200] rev 47265
revlog: add a `_get_decompressor` method
This logic is non-trivial and we will need to reuse it.
Differential Revision: https://phab.mercurial-scm.org/D10651
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 19:46:25 +0200] rev 47264
revlog: introduce a plain compression mode
That mode is simple it means the chunk contains uncompressed data and can be
used directly.
Differential Revision: https://phab.mercurial-scm.org/D10650
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 20:22:34 +0200] rev 47263
revlog: use an intermediate variable for `data` in `_chunk`
This will help us to dispatch this to the right decompressor.
Differential Revision: https://phab.mercurial-scm.org/D10649
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 18:41:51 +0200] rev 47262
revlogv2: preserve the compression mode on disk
The value is still the same and still not treated, but now it exists one disk.
Differential Revision: https://phab.mercurial-scm.org/D10648
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 04 May 2021 01:49:37 +0200] rev 47261
revlog: use `format_version` to dispatch different version in index_get
This is semantically more correct.
Differential Revision: https://phab.mercurial-scm.org/D10647
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 18:19:16 +0200] rev 47260
revlog: add a "data compression mode" entry in the index tuple
That will make it possible to keep track of compression information in the
revlog index, opening the way to more efficient revision restoration (in native
code, but the python usage is already defeating performance work).
We start with adding a new entry to the index tuple, using a value matching the
current behavior. We will introduce storage and other value in later changesets.
Differential Revision: https://phab.mercurial-scm.org/D10646
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 04 May 2021 01:15:03 +0200] rev 47259
revlog: fix the `null_item` attribute for V0
This is not a proper index tuple.
Differential Revision: https://phab.mercurial-scm.org/D10645
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 04 May 2021 01:13:53 +0200] rev 47258
revlog: create a create `null_item` attribute for V0
This value is currently wrong, but at least it is now -obviously- wrong.
Differential Revision: https://phab.mercurial-scm.org/D10644
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 16:52:38 +0200] rev 47257
revlog: improve documentation of the entry tuple
The code in revlog, and outside revlog directly use the index's entry tuple,
with direct integer indexing. This is a voluntary trade off to obtains better
performance from the Python code at the expense of the developers sanity.
Let's at least have a clear and central documentation about what this tuple is
about.
Differential Revision: https://phab.mercurial-scm.org/D10643
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 23:45:05 +0200] rev 47256
revlog: simplify entry update logic in `rewrite_sidedata`
We don't need to rebuild the full entry. So lets just gather the data we need
to pass to `replace_sidedata_info`.
Differential Revision: https://phab.mercurial-scm.org/D10642
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 04 May 2021 00:02:48 +0200] rev 47255
revlog: simplify the replace_sidedata_info code
We already know how to serialize and de-serialize and enty. So lets just do that
and modify the entry tuple directly.
This avoid having to duplicated binary operation in complicated code.
Differential Revision: https://phab.mercurial-scm.org/D10641
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 23:59:15 +0200] rev 47254
revlog: use `rev` instead of `i` in replace_sidedata_info
This is clearer.
Differential Revision: https://phab.mercurial-scm.org/D10640
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:35:35 +0200] rev 47253
revlogv2: also keep track for the size of the "data" file
This is useful to make sure we always start writing at the right location,
without effort.
Differential Revision: https://phab.mercurial-scm.org/D10632
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:35:25 +0200] rev 47252
revlogv2: track pending write in the docket and expose it to hooks
The docket is now able to write pending data. We could have used a distinct
intermediate files, however keeping everything in the same file will make it
simpler to keep track of the various involved files if necessary.
However it might prove more complicated for streaming clone. This will be dealt
with later.
Note that we lifted the stderr redirection in the test since we no longer suffer
from "unkown working directory parent" message.
Differential Revision: https://phab.mercurial-scm.org/D10631
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:35:14 +0200] rev 47251
revlog: move the `trypending` logic from the `changelog` to the `revlog`
We move the -reading- logic for the pending's '.a' suffixed index within the
revlog class. This is motivated by the fact the logic could be simpler and
cleaner if directly handled by the revlog docket. Before we can do so, we need
to teach the revlog code about reading "pending" changes. To be honest, we
already needed some special casing of the `.a` postfix, so this does not adds
much complexity.
The logic around -writing- the special '00changelog.i.a' remains in the
`changelog` class. Note that the revlog-v2 logic no longer use this logic.
The only remaining user of the `postfix` argument is the `censor` logic. We
could probably also make the revlog full aware of it (most of the code is
already implemented in revlog anyway) and get rid of the `postfix` argument and
logic. However this is an adventure for another time.
Since we have more information, we add more, paranoid, Programming error in case
we detect such "pending reader" trying to do a read (which does not happens
anyways).
Differential Revision: https://phab.mercurial-scm.org/D10630
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:35:02 +0200] rev 47250
revlogv2: delay the update of the changelog docket to transaction end
This prevent external reader to see the transaction content before it is
commited. However this also prevent the hooks to see the transaction content.
We will fix this in later changesets.
We have to temporarily suppress the error output of the command ran during the
transaction as they sometimes get confused about unknown working directory and
sometimes issue message on std-err in unspecified order.
Differential Revision: https://phab.mercurial-scm.org/D10629
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:34:52 +0200] rev 47249
revlogv2: track current index size in the docket
This help use to fix transaction safety on repos. See next changesets for details.
Differential Revision: https://phab.mercurial-scm.org/D10628
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:34:41 +0200] rev 47248
revlogv2: also test transactionality of revlog v2
Without surprise, this is broken :-)
Now that it is tested, we can fix it.
Differential Revision: https://phab.mercurial-scm.org/D10627
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:34:31 +0200] rev 47247
revlog: add a new test file focussed on testing transactionally issue
What we test here is currently also covered by `test-hooks.t`. However having
our own test file allow for more focussed/deep testing and to cover more variant
when relevant. So we create a new test file.
Differential Revision: https://phab.mercurial-scm.org/D10626
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:34:21 +0200] rev 47246
revlogv2: store version information in the docket only
Having it duplicated in the index was both useless and a risk of discrepancy.
Differential Revision: https://phab.mercurial-scm.org/D10625
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:34:11 +0200] rev 47245
revlogv2: introduce a very basic docket file
This is the first stone toward using a docket file in revlogv2. Right now the
docket is very basic and only store the version number (which is -also- stored
into the index file…) and the other files have fixed name. This new
implementation break transactionally… but they are no test checking
transactionally for revlogv2… So I take this as an opportunity to start small.
They are no usage of revlogv2 outside of tests anyway.
The docket keeps the `.i` naming used by previous version index to preserve a
unique entry point. We could decide to use a different name and look it up
first, or to fully rework this in a future "store" version. However that does
not seems necessary right now.
We will re-introduces transactionality (and associated testing…) in a later
changesets.
A long list of TODOs have been added to the relevant comment.
Differential Revision: https://phab.mercurial-scm.org/D10624
Raphaël Gomès <rgomes@octobus.net> [Mon, 17 May 2021 15:05:24 +0200] rev 47244
branching: merge stable into default
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:34:01 +0200] rev 47243
revlogv2: mark revlogv2 as requires a full upgrade
This was overlooked previously.
Differential Revision: https://phab.mercurial-scm.org/D10623
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:30:46 +0200] rev 47242
revlog: unify flag processing when loading index
The new code use a simple declaration to do centralised processing. This is
clearer, shorter and less error prone. This will be especially useful as we plan
to add a fourth format: changelog-v2.
Differential Revision: https://phab.mercurial-scm.org/D10622
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:30:35 +0200] rev 47241
revlog: unify checks for supported flag
The new code use a simple declaration to do centralised checking. This is
clearer, shorter and less error prone. This will be especially useful as we plan
to add a fourth format: changelog-v2.
Differential Revision: https://phab.mercurial-scm.org/D10621
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:30:24 +0200] rev 47240
side-data: drop the associated config and requirements
This is no longer and extensions of revlog v1, but a part of revlog v2. We no
longer needs independent config and requirement management for that.
Differential Revision: https://phab.mercurial-scm.org/D10620
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:30:14 +0200] rev 47239
revlog: fix capitalisation of an error
We don't start error message with capital letters.
Differential Revision: https://phab.mercurial-scm.org/D10619
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:30:03 +0200] rev 47238
test: no longer directly use the sidedata config and requirements
They are on their way out, so we simply use revlogv2 in these cases.
Some test does not requires explicite request for the revlogv2 format since the
upgrade to a side-data-capable format is impled by
`exp-use-copies-side-data-changeset`.
Differential Revision: https://phab.mercurial-scm.org/D10618
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:29:52 +0200] rev 47237
sidedata: add a 'side-data' repository feature and use it
Most code don't really care how sidedata support is added, but it needs to know
if it is present. To achieve this. we use the `repo.features` attributes with a
new dedicated features".
Having such centralised information is more robust and will help us to remove the
sidedata requirements in a later changesets.
Differential Revision: https://phab.mercurial-scm.org/D10617
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:29:41 +0200] rev 47236
statichttp: add the missing `features` attribute
This make statichttp closer to the interface it is "supposed" to follow.
Differential Revision: https://phab.mercurial-scm.org/D10616
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:29:30 +0200] rev 47235
sidedata: use revlogv2 requirement in the test helper
We are about to drop the requirement dedicated to sidedata (since revlogv2 will
support them unconditionally and previous version will not.).
To prepare this more we adapt the test code to not use that requirements.
Differential Revision: https://phab.mercurial-scm.org/D10615
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:29:19 +0200] rev 47234
upgrade: display sidedata before upgrade
This is a small gratuitous change that help making the test simpler to
understand on its own.
Differential Revision: https://phab.mercurial-scm.org/D10614
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:29:09 +0200] rev 47233
revlog: determine sidedata support based on the revlog version
Revlog version "2" support sidedata, previous version does not. So lets make the
initialization simpler.
Differential Revision: https://phab.mercurial-scm.org/D10613
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:28:58 +0200] rev 47232
requirements: no longer drop `generaldelta` requirement with revlogv2
A repository could use a mix of revlogv1 and revlogv2, making the requirements
still necessary. Overall we should move away from the "requirements" file being
used a way to configure the repository and stick to it "what do you need to
access this repository". However this is a wider work for another time.
In addition the logic we just dropped was confusing the `hg debugformat`
command, breaking the upgrade code and inconsistent (eg: `sparse-revlog` is also
implied by `revlogv2`).
Finally, multiple other config option would imply the use of the `revlogv2`
requirements, without drop the `generaldelta` one, leading to more
inconsistency.
Differential Revision: https://phab.mercurial-scm.org/D10612
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:28:47 +0200] rev 47231
config: drop the `format.exp-revlogv2.2` option
It is oddly named and not used at all. The one used by the code is still
`experimental.revlogv2`. So we drop that one option for consistency. We move
associate documentation to the actual option.
Differential Revision: https://phab.mercurial-scm.org/D10611
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:28:36 +0200] rev 47230
revlog: only use the `_indexfp` method for read operation
This will avoid "other" code to not overlook `_writing` usage. We introduces
private method dedicated to writing to make use the right option are always
used.
Differential Revision: https://phab.mercurial-scm.org/D10610
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:28:26 +0200] rev 47229
revlog: use `_writing` in `rewrite_sidedata`
Special cases aren't special enough to break the rules.
Differential Revision: https://phab.mercurial-scm.org/D10609
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:28:15 +0200] rev 47228
revlog: open files in 'r+' instead of 'a+'
The code doing actual writing is already doing the necessary seeking, so we
could safely use 'r+'. This make the file objecs usable in more situation, like
updating the sidedata information during pull.
revlog: forcibly move the file cursor at the right location before writing
This is a paranoid change in case the changelog computation moved the cursors
under our feets.
This is not known to happens right now.
Differential Revision: https://phab.mercurial-scm.org/D10608
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:28:05 +0200] rev 47227
revlog: pass a transaction object to `rewrite_sidedata`
The `_writing` context need one, so we update the function signature before considering using `_writing` in rewrite_sidedata.
Differential Revision: https://phab.mercurial-scm.org/D10607
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:27:53 +0200] rev 47226
revlog: rename variable in `rewrite_sidedata` to match other code
Let's call the index file object and `ifh` and the data file object `dfh` as the
rest of the revlog code. This will make future change clearer.
Differential Revision: https://phab.mercurial-scm.org/D10606
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:27:42 +0200] rev 47225
revlog: introduce a mandatory `_writing` context to update revlog content
Before this change, various revlog methods where managing the opening and
closing of the revlog files manually and passing the file descriptor alors the
call path. To simplify the tracking of the write operation by a future docket,
we need something more organised. As a result, we introduce a `revlog._writing`
context manager that will wrap each revlog update operation. The file
descriptor are kept in the existing `revlog._writinghandles` parameter that
was already used by the `addgroup` logic.
All this change is internal to the revlog only, the "public" interface is not
affected. The `addrevision` and `addgroup` logic are still responsible for
setup up this context. However this new context give us multiple benefits:
* all writer use a same, unified, logic,
* this context is programmatically enforced,
* each write "session" as a clearly identified start and end.
The post-pull sidedata update logic is still doing writing by end and will be
adjusted in a later changesets.
This change affect the concurrency checker test, because register the state of
the file in the transaction sooner in `addrevision` (about as early as what
`addgroup` would do), so the abort is rollbacking the other commit. I don't want
to weaken the current main logic.
Differential Revision: https://phab.mercurial-scm.org/D10605
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:27:31 +0200] rev 47224
revlog: preindent some code in _enforceinlinesize
Indenting this beforehand will make a future changeset much simpler.
Differential Revision: https://phab.mercurial-scm.org/D10604
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:27:20 +0200] rev 47223
revlog: preindent some code in addgroup
Indenting this beforehand will make a future changeset much simpler.
Differential Revision: https://phab.mercurial-scm.org/D10603
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 06 May 2021 09:27:22 -0700] rev 47222
pyoxidizer: use Python 3.9 (BC)
Newer versions of PyOxidizer use Python 3.9 by default. We
previously pinned the version to 3.8 to facilitate porting to
a new PyOxidizer version and diffing results.
Now that the porting work is complete, let's bump Python
to Python 3.9.
This will effectively change our Windows Inno and WiX Python 3
installers from Python 3.8 to 3.9.
.. bc::
Windows .msi and .exe installers now use Python 3.9 instead of
Python 3.8.
Differential Revision: https://phab.mercurial-scm.org/D10689
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 06 May 2021 16:11:13 -0700] rev 47221
packaging: use PyOxidizer for producing WiX MSI installer
We recently taught our in-tree PyOxidizer configuration file to
produce MSI installers with WiX using PyOxidizer's built-in
support for doing so.
This commit changes our WiX + PyOxidizer installer generation
code to use this functionality.
After this change, all the Python packaging code is doing is the
following:
* Building HTML documentation
* Making gettext available to the build process.
* Munging CLI arguments to variables for the `pyoxidizer` execution.
* Invoking `pyoxidizer build`.
* Copying the produced `.msi` to the `dist/` directory.
Applying this stack on stable and rebuilding the 5.8 MSI installer
produced the following differences from the official 5.8 installer:
* .exe and .pyd files aren't byte identical (this is expected).
* Various .dist-info/ directories have different names due to older
versions of PyOxidizer being buggy and not properly normalizing
package names. (The new behavior is correct.)
* Various *.dist-info/RECORD files are different due to content
divergence of files (this is expected).
* The python38.dll differs due to newer PyOxidizer shipping a newer
version of Python 3.8.
* We now ship python3.dll because PyOxidizer now includes this file
by default.
* The vcruntime140.dll differs because newer PyOxidizer installs a newer
version. We also now ship a vcruntime140_1.dll because newer versions
of the redistributable ship 2 files now.
The WiX GUIDs and IDs of installed files have likely changed as a
result of PyOxidizer's different mechanism for generating those
identifiers. This means that an upgrade install of the MSI will
replace files instead of doing an incremental update. This is
likely harmless and we've incurred this kind of breakage before.
As far as I can tell, the new PyOxidizer-built MSI is functionally
equivalent to the old method. Once we drop support for Python 2.7
MSI installers, we can delete the WiX code from the repository.
This commit temporarily drops support for extra `.wxs` files. We
raise an exception instead of silently not using them, which I think
is appropriate. We should be able to add support back in by injecting
state into pyoxidizer.bzl via `--var`. I just didn't want to expend
cognitive load to think about the solution as part of this series.
Differential Revision: https://phab.mercurial-scm.org/D10688
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 06 May 2021 16:07:01 -0700] rev 47220
packaging: extract invocation of pyoxidizer to own function
I'll be refactoring how the WiX installer creation calls into
pyoxidizer and will need a lower level function for facilitating
that.
The new `run_pyoxidizer()` builds our execution environment
(with gettext available) and invokes `pyoxidizer`.
Differential Revision: https://phab.mercurial-scm.org/D10687
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 06 May 2021 16:06:20 -0700] rev 47219
packaging: rename run_pyoxidizer()
I'm going to split this function up in a future commit and I'll
want the name "run_pyoxidizer()" for a lower-level function for
invoking `pyoxidizer`.
Differential Revision: https://phab.mercurial-scm.org/D10686
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 06 May 2021 08:37:40 -0700] rev 47218
packaging: move documentation HTML building to own function
This is part of some light refactoring to enable us to use
PyOxidizer for WiX MSI installer generation.
Differential Revision: https://phab.mercurial-scm.org/D10685
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 06 May 2021 16:04:24 -0700] rev 47217
pyoxidizer: support code signing
Newer versions of PyOxidizer feature built-in support for
code signing. You simply declare a code signer in the Starlark
configuration file, activate it for automatic signing, and
PyOxidizer will add code signatures to signable files as it
encounters them.
This commit teaches our Starlark configuration file to enable
automatic code signing. But only on Windows for the moment, as our
immediate goal is to overhaul the Windows packaging.
The feature is opt-in: you must pass variables to PyOxidizer's
build context via `pyoxidizer build --var` or
`pyoxidizer build --var-env` to activate code signing.
Differential Revision: https://phab.mercurial-scm.org/D10684
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 06 May 2021 16:03:43 -0700] rev 47216
pyoxidizer: support producing MSI installers
Newer versions of PyOxidizer have support for building WiX MSI
installers "natively." Essentially, you can script the definition
of your WiX installer via Starlark and PyOxidizer can invoke
WiX tools to produce the installer.
This commit teaches our PyOxidizer config file to produce MSI
installers similarly to how
`contrib/packaging/packging.py wix` would do it.
We had to make a very minor change to `mercurial.wxs` to reflect
different paths depending on who builds. This is because when
PyOxidizer builds WiX installers, it does so from an isolated
directory, not Mercurial's source directory. We simply copy the
files into the build environment so they are accessible.
After this change, running `pyoxidizer build msi` produces a nearly
identical install layout to what the previous method produces.
When I applied this series on top of the 5.8 tag, here is the
list of differences and explanations:
* docs/*.html files are missing from the new installer because the
Python build environment doesn't have docutils.
* .pyd and .exe files differ, likely because I'm using a different
Visual Studio toolchain on my local computer than the official build
environment.
* Various .dist-info/ directories have different names. This is
because older versions of PyOxidizer had buggy behavior and weren't
properly normalizing package names in .dist-info/ directories. e.g.
we went from `cached-property-1.5.2.dist-info` to
`cached_property-1.5.2.dist-info`.
* Translations (.mo files) may be missing if gettext isn't in %Path%.
This is because the packaging.py code installs gettext and ensures
it can be found.
* Some *.dist-info/RECORD files vary due to SHA-256 content digest
divergence due to build environment differences. (This should be
harmless.)
* The new install layout ships a python3.dll because newer versions
of PyOxidizer ship this file.
* The new install layout has a different vcruntime140.dll and also a
vcruntime140_1.dll because newer versions of PyOxidizer ship a
newer version of the Visual C++ Redistributable Runtime.
The new PyOxidizer functionality is not yet integrated with
packaging.py. This will come in a subsequent commit. So for now, the
new functionality introduced here is unused.
Differential Revision: https://phab.mercurial-scm.org/D10683
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 06 May 2021 16:16:21 -0700] rev 47215
pyoxidizer: use allocator_backend instead of raw_allocator
The name of this attribute changed in PyOxidizer 0.11.
Differential Revision: https://phab.mercurial-scm.org/D10682
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 06 May 2021 15:58:37 -0700] rev 47214
pyoxidizer: simplify targets
The split targets existed to enable the use of a non-default
distribution flavor on Windows. Modern versions of PyOxidizer
use the "standalone_dynamic" distribution flavor by default.
So our split brain workaround is no longer needed.
Here, we unify the targets. We also remove an unreferenced
target function to create a resources file.
Differential Revision: https://phab.mercurial-scm.org/D10681
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 06 May 2021 15:56:04 -0700] rev 47213
contrib: install PyOxidizer 0.16.0
PyOxidizer now provides MSI installers and pre-built Linux binaries.
So we install that way. This significantly reduces the time to bootstrap
a new machine in automation, as building PyOxidizer from source on a low
core count machine takes several minutes.
This change temporarily breaks the ability of the automated environment
to use the in-repo pyoxidizer.bzl configuration file, as there are
backwards-incompatible changes with the upgrade that need to be
reflected. We'll handle those in subsequent commits.
Differential Revision: https://phab.mercurial-scm.org/D10680
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 06 May 2021 16:13:33 -0700] rev 47212
pyoxidizer: pin Python to 3.8
Newer versions of PyOxidizer use version 3.9 by default. As part of
upgrading PyOxidizer, we want to pin the version at 3.8 so we can
compare differences more easily.
Differential Revision: https://phab.mercurial-scm.org/D10679
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 06 May 2021 16:00:44 -0700] rev 47211
pyoxidizer: remove some boilerplate in file
This isn't used for anything and can safely be removed.
Differential Revision: https://phab.mercurial-scm.org/D10678
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 06 May 2021 15:45:04 -0700] rev 47210
contrib: upgrade Rust in Windows automation
I'm not sure why we don't install the minimum required Rust version
here like we do for Linux. Whatever: that's unrelated to wanting to
stay modern.
Differential Revision: https://phab.mercurial-scm.org/D10677
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 06 May 2021 15:44:29 -0700] rev 47209
contrib: update Python versions in Windows automation
Let's keep our Python versions modern.
Differential Revision: https://phab.mercurial-scm.org/D10676
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 06 May 2021 15:41:52 -0700] rev 47208
automation: update rust in Linux environment
Our minimum supported Rust is 1.41.1 per rust/README.rst. We also
bump the modern Rust version to latest stable to stay current.
Differential Revision: https://phab.mercurial-scm.org/D10675
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 06 May 2021 15:40:47 -0700] rev 47207
automation: install latest Python versions in Linux environment
Let's keep our Linux environment up to date.
Differential Revision: https://phab.mercurial-scm.org/D10674
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 06 May 2021 17:46:57 -0700] rev 47206
automation: create Python 3.5 variant of requirements.txt
The automation environment is refusing to build with the previous
file because some dependencies won't install on Python 3.5.
I couldn't find an easy way to salvage the situation with a single
requirements.txt file. So, I decided to introduce a variant
for Python 3.5.
As part of this, we update packages to latest versions.
(I do question why we are still supporting Python 3.5...)
Differential Revision: https://phab.mercurial-scm.org/D10690
Karthikeyan Singaravelan <tir.karthi@gmail.com> [Sat, 15 May 2021 09:45:10 +0000] rev 47205
hgweb: Fix deprecation warning in Python 3.10 (issue6520)
Differential Revision: https://phab.mercurial-scm.org/D10711
Simon Sapin <simon.sapin@octobus.net> [Mon, 10 May 2021 21:59:13 +0200] rev 47204
dirstate-tree: Remove newly-empty nodes after removing a `DirstateEntry`
This is actually necessary to make `DirstateMap::has_dir` correct, since it
assumes that a node without a `DirstateEntry` has at least one descedant node
with a `DirstateEntry`.
This bug would become apparent when a later changeset persists tree nodes
on disk in the "dirstate-v2" format.
Differential Revision: https://phab.mercurial-scm.org/D10706
Simon Sapin <simon.sapin@octobus.net> [Mon, 10 May 2021 21:31:05 +0200] rev 47203
dirstate-tree: Refactor DirstateMap::drop_file to be recursive
It should behave the same as before. This will enable the next changeset
to run code on the way "down" (in order to removing newly-empty nodes).
Differential Revision: https://phab.mercurial-scm.org/D10705
Martin von Zweigbergk <martinvonz@google.com> [Tue, 11 May 2021 12:22:26 -0700] rev 47202
templates: fix `revset('parents()') % ...` in amend message template
I don't understand why, but putting `revset('parents()') % {desc}` in
the commit message template for amend resulted in a crash because
`memctx.hex()` did `hex(self.node())` and its node was None. This
patch fixes that.
Martin von Zweigbergk <martinvonz@google.com>
Differential Revision: https://phab.mercurial-scm.org/D10707
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 15 Apr 2021 19:54:03 +0200] rev 47201
path: teach the `hg path` command to display boolean sub-option
The next changeset introduce one.
Differential Revision: https://phab.mercurial-scm.org/D10450
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 16 Apr 2021 01:18:28 +0200] rev 47200
parselist: move the function from config to stringutil
We move the function in a lower level module to avoid cycle. It moves next to
`parsebool` who had to migrate for the same reasons.
Differential Revision: https://phab.mercurial-scm.org/D10449
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 16 Apr 2021 02:14:21 +0200] rev 47199
bookmark: use `list_paths` to access path definition
The content of the `[paths]` config section is receiving transformation that
make it hard to recognise whats the actual name, and the next changeset will
make it worse. So we use the official API for this instead.
Differential Revision: https://phab.mercurial-scm.org/D10448
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 15 Apr 2021 16:58:20 +0200] rev 47198
urlutil: make `paths` class old list of `path`
We move from a `{name → path}` mapping to a `{name → [path]}` mapping. And
update all user code accordingly. For now, all the list contains exactly one
element, but we are now in a good place to make the config understand a list of
url.
Differential Revision: https://phab.mercurial-scm.org/D10447
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 15 Apr 2021 17:15:43 +0200] rev 47197
urlutil: extract `chain_path` in a function
This will no longer modify `path` inplace so it does not make much sense as a method.
Differential Revision: https://phab.mercurial-scm.org/D10446
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 15 Apr 2021 17:12:25 +0200] rev 47196
urlutil: add a `copy` method to `path
This will be useful when inheriting from multiple path at the same time.
Differential Revision: https://phab.mercurial-scm.org/D10445
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 15 Apr 2021 12:33:05 +0200] rev 47195
template: add a `paths` field to all entry in peerurls
This make it possible to display multiple path per name in the near future.
Differential Revision: https://phab.mercurial-scm.org/D10444
Simon Sapin <simon.sapin@octobus.net> [Tue, 11 May 2021 18:10:59 +0200] rev 47194
status: Add some more tests
Trying to improve coverage for various non-ovbious scenarios
Differential Revision: https://phab.mercurial-scm.org/D10704
Matt Harbison <matt_harbison@yahoo.com> [Sat, 08 May 2021 00:21:31 -0400] rev 47193
run-tests: use the same python version for shebang lines on Windows
The latest py3 is used if the minor number isn't specified. After running the
script to install all of the build dependencies, that moved the default from 3.8
to 3.9 on the CI system. That in turn caused a bunch of tests to be skipped
that were running prior, even when the test runner was invoked with `py -3.8`.
While we should almost always use the latest version, we really shouldn't make
it hard to test different versions or allow things to randomly break in subtle
ways like that.
Differential Revision: https://phab.mercurial-scm.org/D10702
Karthikeyan Singaravelan <tir.karthi@gmail.com> [Fri, 14 May 2021 10:01:29 +0000] rev 47192
vfs: Fix deprecation warning in Python 3.10 (issue6520)
Differential Revision: https://phab.mercurial-scm.org/D10710
Raphaël Gomès <rgomes@octobus.net> [Wed, 12 May 2021 10:24:17 +0200] rev 47191
contrib: fix typo
I forgot to fix it in flight, this commit will do fine.
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:27:09 +0200] rev 47190
revlog: rename `indexdata` to entry_data
Same reasoning as the previous changeset, we might not be looking at index data
here.
Differential Revision: https://phab.mercurial-scm.org/D10602
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:26:59 +0200] rev 47189
revlog: use "entry_point" phrasing for loading the revlog
The main entry for the revlog will not necessary be the index, but a small
"docket". So we change the variable names and we move the initialisation of the
index_file and data_file after that first entry point have been read.
Differential Revision: https://phab.mercurial-scm.org/D10601
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:26:48 +0200] rev 47188
revlog: directly use self._format_flags when loading index
The shorthand variable does not bring much, so we drop it to simplify the code.
Differential Revision: https://phab.mercurial-scm.org/D10600
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:26:37 +0200] rev 47187
revlog: directly use self._format_version when loading index
The shorthand variable does not bring much, so we drop it to simplify the code.
Differential Revision: https://phab.mercurial-scm.org/D10599
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:26:27 +0200] rev 47186
revlog: use `_format_flags` to access flags instead of `header`
It seems better to reuse the variable we carefully extracted
This also open the way to more flexible way to retrieve these flags.
Differential Revision: https://phab.mercurial-scm.org/D10598
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:26:17 +0200] rev 47185
revlog: rename `newversionflags` to `new_header`
This make it consistent with the previous changeset.
Differential Revision: https://phab.mercurial-scm.org/D10597
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:26:06 +0200] rev 47184
revlog: rename `versionflags` to header
The variable is named as such because it contains "version" information and
"flags" information. However you mostly needs to know the code to understand the
name which is not great. The fact that this is the very first four bytes in all
revlog seems more relevant, so we rename the variable "header".
Differential Revision: https://phab.mercurial-scm.org/D10596
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:25:56 +0200] rev 47183
revlog: move index reading logic in a dedicated method
They are multiple motivation to do it:
* The logic is complicated enough to deserver its own method.
* We will need to reuse this once we put a docket in use.
* This split the actual reading from the processing of the read data better.
Differential Revision: https://phab.mercurial-scm.org/D10595
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:25:45 +0200] rev 47182
revlog: define the actual index and datafile at loading time
This is just code movement, to make the code closer to where we actually use it
and where it will be defined in the future.
Differential Revision: https://phab.mercurial-scm.org/D10594
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:25:34 +0200] rev 47181
revlog: simplify a conditionnal in _enforceinlinesize
This is a gratuitous change to make things a bit easier to read.
Differential Revision: https://phab.mercurial-scm.org/D10593
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:25:23 +0200] rev 47180
revlog: drop `flush` parameter from `_peek_iscensored`
This is not used anywhere.
Differential Revision: https://phab.mercurial-scm.org/D10592
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:25:12 +0200] rev 47179
revlog: fix error message when data are missing
The error message the message was not using the requested offset, but the
adjusted offset to that read more data for improved caching. This resulted in
confusing error message.
Differential Revision: https://phab.mercurial-scm.org/D10591
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:25:01 +0200] rev 47178
revlog: rename `nodemap_file` to `_nodemap_file`
Same reasoning as for `indexfile and datafile`, lets hide these implementation
details.
Differential Revision: https://phab.mercurial-scm.org/D10590
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:24:50 +0200] rev 47177
revlog: use revlog.display_id in error related to bad revisions
Differential Revision: https://phab.mercurial-scm.org/D10589
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:24:40 +0200] rev 47176
revlog: use revlog.display_id in "revision too big" errors
Differential Revision: https://phab.mercurial-scm.org/D10588
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:24:29 +0200] rev 47175
revlog: use revlog.display_id in censor related errors
Differential Revision: https://phab.mercurial-scm.org/D10587
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:24:18 +0200] rev 47174
revlog: use revlog.display_id in integrity error
Differential Revision: https://phab.mercurial-scm.org/D10586
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:24:08 +0200] rev 47173
revlog: use revlog.display_id in ambiguity errors
Differential Revision: https://phab.mercurial-scm.org/D10585
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:23:58 +0200] rev 47172
revlog: use revlog.display_id for corruption error
Differential Revision: https://phab.mercurial-scm.org/D10584
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:23:48 +0200] rev 47171
revlog: use revlog.display_id in format related errors
Differential Revision: https://phab.mercurial-scm.org/D10583
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:23:37 +0200] rev 47170
revlog: use revlog.display_id in narrow error message
Differential Revision: https://phab.mercurial-scm.org/D10582
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:23:27 +0200] rev 47169
revlog: use revlog.display_id in LookupError
Differential Revision: https://phab.mercurial-scm.org/D10581
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:23:17 +0200] rev 47168
revlog: use revlog.display_id for FilteredLookupError
Differential Revision: https://phab.mercurial-scm.org/D10580
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:23:07 +0200] rev 47167
revlog: introduce a `display_id` property
We currently using the "index file" to identify a revlog in error output. Since
we are about to make the "index file" location more volatile, we need something
better. We move to use the "radix", as it is close to what we currently use.
We could probably do better, as pointed out in the comment, however that would
be a quite detour from my current goal.
Differential Revision: https://phab.mercurial-scm.org/D10579
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:22:57 +0200] rev 47166
revlog: also use radix when computing nodemap data file
We have a radix, lets use it!
Differential Revision: https://phab.mercurial-scm.org/D10578
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:22:47 +0200] rev 47165
revlog: stop usage of `_indexfile` to computing nodemap path
We now have the radix explicitely lets use the radix explicitely
Differential Revision: https://phab.mercurial-scm.org/D10577
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:22:36 +0200] rev 47164
revlog: use a "radix" to address revlog
Instead of pointing to the index directly and to derive the other file from
that, we directly provide the radix and let the revlog determine the associated
file path internally. This is more robust and will give us more flexibility for
picking this file name in the future.
Differential Revision: https://phab.mercurial-scm.org/D10576
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:22:26 +0200] rev 47163
revlog: rename `datafile` to `datafile`
We want to make the actual location of the datafile and location more of an
implementation details than what is is currently. In that process, we make the
attribute private.
Differential Revision: https://phab.mercurial-scm.org/D10575
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:22:16 +0200] rev 47162
revlog: rename `indexfile` to `_indexfile`
We want to make the actual location of the indexfile and location more of an
implementation details than what is is currently. In that process, we make the
attribute private.
Differential Revision: https://phab.mercurial-scm.org/D10574
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:22:06 +0200] rev 47161
filelog: drop `indexfile` from `filelog`
Since `filelog` objects are not revlog (no really, they are not…) we drop the
revlog specific attribute. We need to directly access the underlying revlog in a
couple of place that already assume that we have a revlog here.
This is motivated by future change to that revlog attribute.
Differential Revision: https://phab.mercurial-scm.org/D10573
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:21:56 +0200] rev 47160
manifest: drop the `indexfile` from `manifestrevlog`
Since `manifestrevlog` object are not revlog (no really, they are not…) we drop
the revlog specific attribute. We need to directly access the underlying revlog
in a couple of place that already assume that we have a revlog here.
This is motivated by future change to that revlog attribute.
Differential Revision: https://phab.mercurial-scm.org/D10572
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:21:46 +0200] rev 47159
revlog: deal with special "postfix" explicitely
revlog usually use a straight forward '.i' and '.d' naming except for two cases
"in-transaction" changelog, and censoring. Our goal is to let the revlog code
deal with the internal of the file naming itself. To do so, we need to start
dealing with these postfix explicitly.
Differential Revision: https://phab.mercurial-scm.org/D10571
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:21:35 +0200] rev 47158
revlog: split the option initialisation in its own method
The part of the code is huge, keeping it separated will keep the `_loadindex`
method simpler and help keeping logic well insulated.
Differential Revision: https://phab.mercurial-scm.org/D10570
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:21:25 +0200] rev 47157
revlog: always "append" full size tuple
Same reasoning as the previous patch.
Differential Revision: https://phab.mercurial-scm.org/D10569
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:21:15 +0200] rev 47156
revlog: make the index always return the same tuple
It is simpler to manage the diferrence in on disk format in the internal index
code itself and lets the rest of the code always handle the same object.
This will become even more important when the data we store will be entirely
different (for example the changelog does not need the "linkrev" field.
We start with item reading, we will deal with item writing in the next
changesets.
Differential Revision: https://phab.mercurial-scm.org/D10568
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:21:05 +0200] rev 47155
revlog: introduce an explicit `format_version` member in the index struct
This will allow for cleaner check than assuming each version has a different
size. Unsurprisingly I am planning to use this to introduce more format variant.
Differential Revision: https://phab.mercurial-scm.org/D10567
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:20:55 +0200] rev 47154
revlog: rename `hdrsize` to `entry_size` in the C code
This is the size of and index entry, so lets make it clearer.
Differential Revision: https://phab.mercurial-scm.org/D10566
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:20:45 +0200] rev 47153
revlog: split the `version` attribute into its two components
The `revlog.version` attribute contained an integer coding 2 different informations:
* the revlog version number
* a bit field defining some specific feature of the revlog
We now explicitly store the two components independently. This avoid exposing
the implementation details all around the code and prepare for future revlog
version that would encode the information in a different way.
In the process we drop the `version` attribute from the interface. It was
flagged for removal when that interface was created.
Differential Revision: https://phab.mercurial-scm.org/D10565
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:20:35 +0200] rev 47152
verify: pass a revlog to `_checkrevlog` in `_verifymanifest`
Since `manifestrevlog` is not a `revlog`, we are passing strange thing to
`_checkrevlog`. We fix this to avoid breakage during future change.
Differential Revision: https://phab.mercurial-scm.org/D10564
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:20:25 +0200] rev 47151
revlog: replace flag check related to generaldelta with attribute check
Same logic as the previous changesets.
Differential Revision: https://phab.mercurial-scm.org/D10563
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:19:09 +0200] rev 47150
revlog: replace REVLOGV2 check related to sidedata with `hassidedata` checks
This is more flexible and semantically more correct. The associated revlog's
attribute exist since 827cb4fe62a3, so well we start linking sidedata to
revlogv2.
Differential Revision: https://phab.mercurial-scm.org/D10562
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:19:05 +0200] rev 47149
revlog: explicitely pass the "indexfile" parameter
Most of this was already done when introducing the `target` parameter, but some
remained. Having "indexfile" passed explicitely will help us to change the way
we address a revlog later in the stack. With the introduction of more generic
`docket`, the entry point will not necessarly be `xxx.i` file, and the actual
index files will have a variable name.
Differential Revision: https://phab.mercurial-scm.org/D10561
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:18:58 +0200] rev 47148
revlog: highlight current incompatibility in `rewrite_sidedata`
See comment for details. We will need to fix the test coverage when this
incompatibility is lifted.
Differential Revision: https://phab.mercurial-scm.org/D10544
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:18:48 +0200] rev 47147
revlog: adjust rewrite_sidedata code to not delete existing revlog content
The "w+" file mode is deleting all the content of the opened file. Which is bad…
This is not caught by the test because the test only check for a full, initial
pull where not pre-existing content exists. So we need to fix our test coverage
here.
However they are another issue that prevent "incremental" pull to work here. See
next changeset for details.
Differential Revision: https://phab.mercurial-scm.org/D10543
Simon Sapin <simon.sapin@octobus.net> [Fri, 07 May 2021 17:33:47 +0200] rev 47146
status: Add tests for some more edge cases
* Size-preserving file contents modification
* Filtering output to a deleted or removed file
Differential Revision: https://phab.mercurial-scm.org/D10701
Simon Sapin <simon.sapin@octobus.net> [Fri, 07 May 2021 16:44:36 +0200] rev 47145
status: Extend issue 6483 test to exclude patterns
With `hg status -X`, not just include pattern with `hg status -I`
Differential Revision: https://phab.mercurial-scm.org/D10700
Simon Sapin <simon.sapin@octobus.net> [Fri, 07 May 2021 16:41:07 +0200] rev 47144
dirstate-tree: Add a test showing that issue 6335 is fixed
… when using the new status algorithm and the tree-based dirstate.
The previous algorithm still has this bug.
Differential Revision: https://phab.mercurial-scm.org/D10699
Simon Sapin <simon.sapin@octobus.net> [Mon, 03 May 2021 20:04:19 +0200] rev 47143
dirstate-tree: Add a dirstate-v1-tree variant of some tests
The `dirstate-v1` variant has the previous behavior.
`dirstate-v1-tree` uses the same format on disk, but uses the new
`DirstateMap` with a tree data structure and the new `status` algorithm.
These were untested so far.
Differential Revision: https://phab.mercurial-scm.org/D10698
Matt Harbison <matt_harbison@yahoo.com> [Fri, 07 May 2021 22:06:25 -0400] rev 47142
merge with stable
Martin von Zweigbergk <martinvonz@google.com> [Fri, 07 May 2021 08:38:17 -0700] rev 47141
rename: add hint about --at-rev if source file doesn't exist
It's quite common that users want to record copy (rename) information
after committing the working copy changes (i.e. an added and a deleted
file). When they try `hg mv [--after] <src> <dst>`, that just fails
because the source file doesn't exist. It seems helpful if we can
point them to `--at-rev=.` in this case.
Differential Revision: https://phab.mercurial-scm.org/D10697
Simon Sapin <simon.sapin@octobus.net> [Fri, 30 Apr 2021 20:21:56 +0200] rev 47140
dirstate-tree: Borrow paths from the "on disk" bytes
Use std::borrow::Cow to avoid some memory allocations and copying.
Differential Revision: https://phab.mercurial-scm.org/D10560
Simon Sapin <simon.sapin@octobus.net> [Fri, 30 Apr 2021 19:33:04 +0200] rev 47139
dirstate-tree: Borrow copy source paths from the "on disk" bytes
Use std::borrow::Cow to avoid some memory allocations and copying.
These particular allocations are not visible when profiling (as many files
in a typical repo don’t have a copy source). This change is "warm up"
for doing the same with paths of files themselves, which is more involved
since those paths are used as `HashMap` keys. This gets of the way the
addition of a lifetime parameter to several types.
Differential Revision: https://phab.mercurial-scm.org/D10559
Simon Sapin <simon.sapin@octobus.net> [Fri, 30 Apr 2021 19:57:46 +0200] rev 47138
rust: Use `&HgPath` instead of `&HgPathBuf` in may APIs
Getting the former (through `Deref`) is almost the only useful thing one can
do with the latter anyway. With this changes, API become more flexible for the
"provider" of these paths which may store something else that Deref’s to HgPath,
such as `std::borrow::Cow<HgPath>`. Using `Cow` can help reduce memory alloactions
and copying.
Differential Revision: https://phab.mercurial-scm.org/D10558
Simon Sapin <simon.sapin@octobus.net> [Fri, 30 Apr 2021 18:24:54 +0200] rev 47137
dirstate-tree: Make `DirstateMap` borrow from a bytes buffer
… that has the contents of the `.hg/dirstate` file.
This only applies to the tree-based flavor of `DirstateMap`.
For now only the entire `&[u8]` slice is stored, so this is not useful yet.
Adding a lifetime parameter to the `DirstateMap` struct (in hg-core) makes
Python bindings non-trivial because we keep that struct in a Python object
that has a dynamic lifetime tied to Python’s reference-counting and GC.
As long as we keep the `PyBytes` that owns the borrowed bytes buffer next to
the borrowing struct, the buffer will live long enough for the borrows to stay
valid. However this relationship cannot be expressed in safe Rust code in a
way that would statisfy they borrow-checker. We use `unsafe` code to erase
that lifetime parameter, and encapsulate it in a safe abstraction similar to
the owning-ref crate: https://docs.rs/owning_ref/
Differential Revision: https://phab.mercurial-scm.org/D10557
Simon Sapin <simon.sapin@octobus.net> [Fri, 30 Apr 2021 18:13:31 +0200] rev 47136
rust: Read dirstate from disk in DirstateMap constructor
Before this changeset, Python code first creates an empty `DirstateMap` Rust
object, then immediately calls its `read` method with a byte string of the
contents of the `.hg/dirstate` file.
This makes that byte string available to the constructor of `DirstateMap`
in the hg-cpython crate. This is a first step towards enabling parts of
`DirstateMap` in the hg-core crate to borrow from this buffer without copying.
Differential Revision: https://phab.mercurial-scm.org/D10556
Simon Sapin <simon.sapin@octobus.net> [Fri, 30 Apr 2021 15:40:11 +0200] rev 47135
rust: Remove handling of `parents` in `DirstateMap`
The Python wrapper class `dirstatemap` can take care of it.
This removes the need to have both `_rustmap` and `_inner_rustmap`.
Differential Revision: https://phab.mercurial-scm.org/D10555
Simon Sapin <simon.sapin@octobus.net> [Fri, 30 Apr 2021 14:22:14 +0200] rev 47134
dirstate-tree: Fold "tracked descendants" counter update in main walk
For the purpose of implementing `has_tracked_dir` (which means "has tracked
descendants) without an expensive sub-tree traversal, we maintaing a counter
of tracked descendants on each "directory" node of the tree-shaped dirstate.
Before this changeset, mutating or inserting a node at a given path would
involve:
* Walking the tree from root through ancestors to find the node or the spot
where to insert it
* Looking at the previous node if any to decide what counter update is needed
* Performing any node mutation
* Walking the tree *again* to update counters in ancestor nodes
When profiling `hg status` on a large repo, this second walk takes times
while loading a the dirstate from disk.
It turns out we have enough information to decide before he first tree walk
what counter update is needed. This changeset merges the two walks, gaining
~10% of the total time for `hg update` (in the same hyperfine benchmark as
the previous changeset).
---
Profiling was done by compiling with this `.cargo/config`:
[profile.release]
debug = true
then running with:
py-spy record -r 500 -n -o /tmp/hg.json --format speedscope -- \
./hg status -R $REPO --config experimental.dirstate-tree.in-memory=1
then visualizing the recorded JSON file in https://www.speedscope.app/
Differential Revision: https://phab.mercurial-scm.org/D10554