Raphaël Gomès <rgomes@octobus.net> [Wed, 21 Apr 2021 10:58:21 +0200] rev 47382
bzr: only support bzr conversion on Python3
Python 2 support will be dropped from Mercurial before anyone needs to do a
bzr -> hg conversion on Python 2 again. Bazaar tests were broken with the new
lib anyway, which we'll get into in the next patches.
Differential Revision: https://phab.mercurial-scm.org/D10512
Martin von Zweigbergk <martinvonz@google.com> [Tue, 01 Jun 2021 22:38:29 -0700] rev 47381
rewriteutil: add pointer to help text when rewrite would cause divergence
The evolve extension's version of the hint has this pointer. I missed
it when I moved it to core. This is pretty much the same as
D10723. Turns out I missed two places :(
Differential Revision: https://phab.mercurial-scm.org/D10832
Simon Sapin <simon.sapin@octobus.net> [Tue, 25 May 2021 09:20:30 +0200] rev 47380
rhg: Add support for dirstate-v2
Differential Revision: https://phab.mercurial-scm.org/D10804
Simon Sapin <simon.sapin@octobus.net> [Wed, 26 May 2021 11:53:37 +0200] rev 47379
rhg: Remove some intermediate Vecs in `rhg files`
Instead of calling `parse_dirstate` which then calls `parse_dirstate_entries`,
call the latter directly in order to skip some intermediate steps.
Differential Revision: https://phab.mercurial-scm.org/D10803
Martin von Zweigbergk <martinvonz@google.com> [Tue, 01 Jun 2021 15:19:08 -0700] rev 47378
merge: make applyupdates() not mutate mresult argument
We have an extension at work that overrides `merge.applyupdates()` to
make it skip some writes and instead instruct the virtual filesystem
we use to get a different version. That override doesn't work
correctly when doing `hg co -m` and there's a modified file in the
dirstate that's deleted in the destination. That's because
`applyupdates()` mutates its `mresult` argument and our extension had
passed in a modified copied of `mresult` to the overridden function,
which resulted in the mutation not having any effect. This patch fixes
that by letting the caller (i.e. `merge._update()`) update `mresult`
with the extra actions instead. Besides fixing our internal extension,
that seems cleaner to me anyway (better to not mutate `mresult` only
in some cases and we can skip some of the logic if we're not going to
update the dirstate anyway).
Differential Revision: https://phab.mercurial-scm.org/D10830
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 03 Jun 2021 16:12:03 +0200] rev 47377
index: use an explicit constant for INDEX_HEADER format and use it for docket
This avoid leaking python-3.6 compatibility details too much.
Differential Revision: https://phab.mercurial-scm.org/D10831
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 01 Jun 2021 09:18:27 +0200] rev 47376
verify: use some intermediate variables instead of a multi-liner
This is shorter and easier to read as the indentation remains the same.
Differential Revision: https://phab.mercurial-scm.org/D10822
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 01 Jun 2021 09:18:20 +0200] rev 47375
verify: use some intermediate variables instead of a multi-liner
This is shorter and easier to read as the indentation remains the same.
We extract the long message in a module level constant for clarity.
Differential Revision: https://phab.mercurial-scm.org/D10821
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 01 Jun 2021 09:14:58 +0200] rev 47374
verify: use some intermediate variables instead of a multi-liner
This is shorter and easier to read as the indentation remains the same.
Differential Revision: https://phab.mercurial-scm.org/D10820
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 01 Jun 2021 09:13:33 +0200] rev 47373
verify: use some intermediate variables instead of a multi-liner
This is shorter and easier to read as the indentation remains the same.
We extract the long message in a module level constant for clarity.
Differential Revision: https://phab.mercurial-scm.org/D10819
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 01 Jun 2021 09:11:54 +0200] rev 47372
verify: use some intermediate variables instead of a multi-liner
This is shorter and easier to read as the indentation remains the same.
Differential Revision: https://phab.mercurial-scm.org/D10818
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 01 Jun 2021 09:11:07 +0200] rev 47371
verify: use some intermediate variables instead of a multi-liner
This is shorter and easier to read as the indentation remains the same.
Differential Revision: https://phab.mercurial-scm.org/D10817
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 01 Jun 2021 09:11:01 +0200] rev 47370
verify: use some intermediate variables instead of a multi-liner
This is shorter and easier to read as the indentation remains the same.
Differential Revision: https://phab.mercurial-scm.org/D10816
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 01 Jun 2021 09:09:01 +0200] rev 47369
verify: use some intermediate variables instead of a multi-liner
This is shorter and easier to read as the indentation remains the same.
We extract the long message in a module level constant for clarity.
Differential Revision: https://phab.mercurial-scm.org/D10815
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 01 Jun 2021 09:06:33 +0200] rev 47368
verify: use some intermediate variables instead of a multi-liner
This is shorter and easier to read as the indentation remains the same.
Differential Revision: https://phab.mercurial-scm.org/D10814
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 01 Jun 2021 09:05:53 +0200] rev 47367
verify: use some intermediate variables instead of a multi-liner
This is shorter and easier to read as the indentation remains the same.
Differential Revision: https://phab.mercurial-scm.org/D10813
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 01 Jun 2021 09:05:23 +0200] rev 47366
verify: use some intermediate variables instead of a multi-liner
This is shorter and easier to read as the indentation remains the same.
Differential Revision: https://phab.mercurial-scm.org/D10812
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 01 Jun 2021 09:03:48 +0200] rev 47365
verify: use some intermediate variables instead of a multi-liner
This is shorter and easier to read as the indentation remains the same.
Differential Revision: https://phab.mercurial-scm.org/D10811
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 01 Jun 2021 09:03:24 +0200] rev 47364
verify: use some intermediate variables instead of a multi-liner
This is shorter and easier to read as the indentation remains the same.
We extract the long message in a module level constant for clarity.
Differential Revision: https://phab.mercurial-scm.org/D10810
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 01 Jun 2021 09:01:23 +0200] rev 47363
verify: use some intermediate variables instead of a multi-liner
This is shorter and easier to read as the indentation remains the same.
Differential Revision: https://phab.mercurial-scm.org/D10809
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 01 Jun 2021 08:59:50 +0200] rev 47362
verify: use some intermediate variables instead of a multi-liner
This is shorter and easier to read as the indentation remains the same.
Differential Revision: https://phab.mercurial-scm.org/D10808
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 01 Jun 2021 08:59:07 +0200] rev 47361
verify: use some intermediate variables instead of a multi-liner
This is shorter and easier to read as the indentation remains the same.
Differential Revision: https://phab.mercurial-scm.org/D10807
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 01 Jun 2021 08:58:53 +0200] rev 47360
verify: expand a one liner into explicit commands
The result is not longer, but it is more edible.
Differential Revision: https://phab.mercurial-scm.org/D10806
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 01 Jun 2021 08:54:53 +0200] rev 47359
verify: align a comment line
This argument description is back with it comrade.
Differential Revision: https://phab.mercurial-scm.org/D10805
Simon Sapin <simon.sapin@octobus.net> [Tue, 01 Jun 2021 16:55:59 +0200] rev 47358
dirstate-v2: Drop parent directory cache when removing a dirstate node
The premise of the directory cache is that the dirstate contains child nodes
for every entry that `read_dir` would return. When removing nodes, that may
not be the case anymore so the cache should be invalidated.
Differential Revision: https://phab.mercurial-scm.org/D10829
Simon Sapin <simon.sapin@octobus.net> [Mon, 31 May 2021 19:54:41 +0200] rev 47357
dirstate-v2: Add --dirs to debugdirstate command
`hg debugdirstate --dirs` also shows information stored in the dirstate
(for `read_dir` caching) about directories.
Differential Revision: https://phab.mercurial-scm.org/D10828
Simon Sapin <simon.sapin@octobus.net> [Mon, 31 May 2021 18:35:44 +0200] rev 47356
dirstate-v2: Write .hg/dirstate back to disk on directory cache changes
Differential Revision: https://phab.mercurial-scm.org/D10827
Simon Sapin <simon.sapin@octobus.net> [Fri, 28 May 2021 11:48:59 +0200] rev 47355
dirstate-v2: Skip readdir in status based on directory mtime
When calling `read_dir` during `status` and the directory is found to be
eligible for caching (see code comments), write the directory’s mtime to the
dirstate. The presence of a directory mtime in the dirstate is meaningful
and indicates eligibility.
When an eligible directory mtime is found in the dirstate and `stat()` shows
that the mtime has not changed, `status` can skip calling `read_dir` again
and instead rely on the names of child nodes in the dirstate tree.
The `tempfile` crate is used to create a temporary file in order to use its
modification time as "current time" with the same truncation as other files
and directories would have in their own modification time.
Differential Revision: https://phab.mercurial-scm.org/D10826
Simon Sapin <simon.sapin@octobus.net> [Thu, 27 May 2021 18:40:54 +0200] rev 47354
dirstate-v2: Allow tree nodes without an entry to store a timestamp
Timestamps are stored on 96 bits:
* 64 bits for the signed number of seconds since the Unix epoch
* 32 bits for the nanoseconds in the `0 <= ns < 1_000_000_000` range
For now timestamps are not used or set yet.
Differential Revision: https://phab.mercurial-scm.org/D10825
Simon Sapin <simon.sapin@octobus.net> [Fri, 28 May 2021 20:07:27 +0200] rev 47353
dirstate-tree: Change status() results to not borrow DirstateMap
The `status` function takes a `&'tree mut DirstateMap<'on_disk>` parameter.
`'on_disk` borrows a read-only byte buffer with the contents of the
`.hg/dirstate` file. `DirstateMap` internally uses represents file paths as
`std::borrow::Cow<'on_disk, HgPath>`, which borrows the byte buffer when
possible and allocates an owned string if not, such as for files added to the
dirstate after it was loaded from disk.
Previously the return type of of `status` has a `'tree` lifetime, meaning it
could borrow all paths from the `DirstateMap`. With this changeset, that
lifetime is changed to `'on_disk` meaning that only paths from the byte buffer
can be borrowed, and paths allocated by `DirstateMap` must be copied.
Usually most paths are in the byte buffer, and most paths are not part of the
return value of `status`, so the number of extra copies should be small.
This change will enable `status` to mutate the `DirstateMap` after it has
finished constructing its return value. Previously such mutation would be
prevented by possible on-going borrows.
Differential Revision: https://phab.mercurial-scm.org/D10824
Simon Sapin <simon.sapin@octobus.net> [Fri, 28 May 2021 12:16:14 +0200] rev 47352
dirstate-tree: Fix status algorithm with unreadable directory
When reading a directory fails such as because of insufficient permissions,
it should be treated as empty by status instead of skipped entirely.
Differential Revision: https://phab.mercurial-scm.org/D10823
Martin von Zweigbergk <martinvonz@google.com> [Tue, 25 May 2021 16:46:32 -0700] rev 47351
docket: make compatible with py3.6, where Struct.format is bytes
Differential Revision: https://phab.mercurial-scm.org/D10770
Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 30 May 2021 22:12:48 +0200] rev 47350
revlogv2: make sure bundling pick a compatible bundle format
Before this change, revlog-v2 repository where bundled using the incompatible
"v1" format.
Differential Revision: https://phab.mercurial-scm.org/D10802
Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 30 May 2021 20:42:51 +0200] rev 47349
censor: do not process sidedata of censored revision while bundling
The revision is censored, we should ignore it.
Differential Revision: https://phab.mercurial-scm.org/D10801
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 28 May 2021 20:00:27 +0200] rev 47348
changegroup: fix deltachunk API to be consistent from one class to another
Depending of the subclass the 8th index of `chunkdata` items was either a
sidedata dict of a proto_flags integer. We have not fixed the inconsistency and
we already return fixed "delta" items from `deltaiter`.
Differential Revision: https://phab.mercurial-scm.org/D10778
Augie Fackler <augie@google.com> [Thu, 27 May 2021 12:10:59 -0400] rev 47347
fuzz: add hg to sys.path when constructing mpatch corpus
Differential Revision: https://phab.mercurial-scm.org/D10777
Simon Sapin <simon.sapin@octobus.net> [Wed, 19 May 2021 16:18:16 +0200] rev 47346
dirstate-tree: Skip readdir() in `hg status -mard`
When running the status algorithm in a mode where we don’t list unknown
or ignored files, all we care about are files that are listed in the dirstate.
We can there for skip making expensive calls to readdir() to list the contents
of filesystem directories, and instead only run stat() to get the filesystem
state of files listed in the dirstate. (This state may be an error for files
that don’t exist anymore on the filesystem.)
On 16 CPU threads, this reduces the time spent in the `status()` function for
`hg status -mard` on an old snapshot of mozilla-central from ~70ms to ~50ms.
Differential Revision: https://phab.mercurial-scm.org/D10752
Simon Sapin <simon.sapin@octobus.net> [Wed, 19 May 2021 13:15:00 +0200] rev 47345
dirstate-v2: Parse the dirstate lazily, with copy-on-write nodes
TODO: more description
Differential Revision: https://phab.mercurial-scm.org/D10751
Simon Sapin <simon.sapin@octobus.net> [Wed, 19 May 2021 13:15:00 +0200] rev 47344
dirstate-v2: Make the dirstate bytes buffer available in more places
Differential Revision: https://phab.mercurial-scm.org/D10750
Simon Sapin <simon.sapin@octobus.net> [Wed, 19 May 2021 13:15:00 +0200] rev 47343
dirstate-v2: Make more APIs fallible, returning Result
When parsing becomes lazy, parse error will potentially happen in more places.
This propagates such errors to callers.
Differential Revision: https://phab.mercurial-scm.org/D10749
Simon Sapin <simon.sapin@octobus.net> [Wed, 19 May 2021 13:15:00 +0200] rev 47342
dirstate-v2: Add a zero-size error type for dirstate v2 parse errors
This error should only happen if Mercurial is buggy or the file is corrupted.
It indicates for example that:
* A part of the file refers to another part, and the byte offset or item count
would cause reading out of bounds, beyond the end of the file.
* The byte for an entry state has an invalid value
When parsing becomes lazy, many more functions will return a `Result` with
this error. Making it zero-size reduces the work that the `?` operator needs
to do to pass around the error value.
Differential Revision: https://phab.mercurial-scm.org/D10748
Simon Sapin <simon.sapin@octobus.net> [Wed, 19 May 2021 13:15:00 +0200] rev 47341
dirstate-tree: Add `NodeRef` and `ChildNodesRef` enums
They are used instead of `&Node` and `&ChildNodes` respectively.
The `ChildNodes` type alias also becomes a similar enum.
For now they only have one variant each, to be extended later.
Adding enums now forces various use sites go through new methods
instead of manipulating the underlying data structure directly.
Differential Revision: https://phab.mercurial-scm.org/D10747
Simon Sapin <simon.sapin@octobus.net> [Wed, 19 May 2021 13:15:00 +0200] rev 47340
rust: Return owned instead of borrowed DirstateEntry in DirstateMap APIs
This will enable the tree-based DirstateMap to not always have an actual
DirstateEntry in memory for all nodes, but construct it on demand.
Differential Revision: https://phab.mercurial-scm.org/D10746
Simon Sapin <simon.sapin@octobus.net> [Wed, 19 May 2021 13:15:00 +0200] rev 47339
dirstate-tree: Downgrade `&mut Node` to `&Node` in status and serialization
Mutable access is not used, and upcoming changes will make it more costly
(with copy-on-write nodes that can be read from disk representation)
Differential Revision: https://phab.mercurial-scm.org/D10745
Simon Sapin <simon.sapin@octobus.net> [Wed, 19 May 2021 13:15:00 +0200] rev 47338
dirstate-tree: Remove DirstateMap::iter_node_data_mut
In an upcoming changeset we want DirstateMap to be able to work directly
with nodes in their "on disk" representation, without always allocating
corresponding in-memory data structures. Nodes would have two possible
representations: one immutable "on disk" refering to the bytes buffer
of the contents of the .hg/dirstate file, and one mutable with HashMap
like the curren data structure.
These nodes would have copy-on-write semantics: when an immutable node
would need to be mutated, instead we allocate new mutable node for it and
its ancestors.
A mutable iterator of the entire tree would still be possible, but it would
become much more expensive since we’d need to allocate mutable nodes for
everything.
Instead, remove this iterator. It was only used to clear ambiguous mtimes
while serializing the `DirstateMap`. Instead clearing and serialization are
now two separate passes. Clearing first uses an immutable iterator to collect
the paths of nodes that need to be cleared, then accesses only those nodes
mutably.
Differential Revision: https://phab.mercurial-scm.org/D10744
Matt Harbison <matt_harbison@yahoo.com> [Fri, 28 May 2021 17:33:20 -0400] rev 47337
merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 26 May 2021 21:46:45 +0200] rev 47336
revlog: close the index file handle after the data one
This make sure the data file is flushed before the index. preventing the index
to reference unflushed data.
Differential Revision: https://phab.mercurial-scm.org/D10776
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 26 May 2021 21:35:51 +0200] rev 47335
revlog: simplify the try nesting in the `_writing` context
Lets use a single try, with conditional cleanup. This make is easier to add a
file handle dedicated to sidedata.
Differential Revision: https://phab.mercurial-scm.org/D10775
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 20 May 2021 21:54:21 +0200] rev 47334
revlogv2: add a `get_data` helper to grab the next piece of docket
This make the processing more compact but abstracting repetitive processing
away.
Differential Revision: https://phab.mercurial-scm.org/D10774
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 20 May 2021 21:48:53 +0200] rev 47333
revlogv2: simplify and clarify the processing of each entry
As we add more entries and some of them has non trivial processing it seems
useful to make the processing leaner and clearly separated to simplify futures
patches.
Differential Revision: https://phab.mercurial-scm.org/D10773
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 19 May 2021 16:55:36 +0200] rev 47332
revlogv2: use a unique filename for data
Having a unique data will allow for ambiguity less rewriting of revlog content,
something useful to clarify handling of some operation like censoring or
stripping.
Differential Revision: https://phab.mercurial-scm.org/D10772
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 18 May 2021 15:07:17 +0200] rev 47331
revlogv2: use a unique filename for index
Having a unique index will allow for ambiguity less rewriting of revlog content,
something useful to clarify handling of some operation like censoring or
stripping.
Differential Revision: https://phab.mercurial-scm.org/D10771
Simon Sapin <simon.sapin@octobus.net> [Sat, 22 May 2021 17:35:54 +0200] rev 47330
rhg: A missing .hg/dirstate file is not an error
Instead treat it as like an empty file
Differential Revision: https://phab.mercurial-scm.org/D10766
Simon Sapin <simon.sapin@octobus.net> [Sat, 22 May 2021 17:32:09 +0200] rev 47329
rhg: Sort `rhg status` output correctly
* The relative order of states is:
modified, added, removed, deleted, unknown, ignored, clean
* Files in the same state should be sorted by name, regardless of whether
or not the were in "unsure" state based on metadata alone.
Differential Revision: https://phab.mercurial-scm.org/D10765
Simon Sapin <simon.sapin@octobus.net> [Wed, 19 May 2021 18:35:43 +0200] rev 47328
dirstate-v2: Add `hg debugupgraderepo` command support
This command changes changes the file formats used inside an existing
repository to what they would be in a new repository with the current config.
For example:
hg debugupgraderepo --config format.exp-dirstate-v2=1 --run
hg debugupgraderepo --config format.exp-dirstate-v2=0 --run
If a repository has a dirstate in v1 format, the first command would upgrade it
to dirstate-v2. Conversely, if a repository has a dirstate in v2 format, the
second command would downgrade it to v1. (Both may also run some unrelated
upgrades.)
Since `format.exp-dirstate-v2` is currently disabled by default, not specifying
it in `--config` or any configuration file would result in the second command.
Differential Revision: https://phab.mercurial-scm.org/D10769
Simon Sapin <simon.sapin@octobus.net> [Fri, 21 May 2021 17:12:47 +0200] rev 47327
upgrade: Use `improvement` subclasses everywhere, not instances
This changes the source definition of optimizations to match that of formats:
a subclass with a decorator, instead of an instance passed to a function call.
Not having any instance removes the confusion between class attributes and
instance attributes, which were used interchangeably.
Differential Revision: https://phab.mercurial-scm.org/D10768
Matt Harbison <matt_harbison@yahoo.com> [Tue, 11 May 2021 01:05:38 -0400] rev 47326
exewrapper: avoid directly linking against python3X.dll
Subsequent code calls `LoadLibrary()` to attempt to load the DLL, but because of
this symbol reference, there is an attempt to load the DLL used during the build
prior to `_main()` running. This causes the whole process to fail if the DLL
isn't in the standard search path. That also means it will never load the DLL
for HackableMercurial. (Maybe we should get rid of that for py3, since you can
install python for a user without admin rights?)
This could also be resolved by calling `GetProcAddress()` on the symbol and
dereferencing it, but using the environment variable is consistent with the
*.bat file since fc8a5c9ecee0. (The environment variable persists after the
interpreter is initialized.)
Far more concerning is somehow I've gotten my system into a state where setting
the flag causes any output to the pager to be lost (as if it wasn't set at all)
in MSYS, cmd.exe, WSL, and PowerShell using py3.9.0, but the environment
variable works properly. I'm sure this flag worked on some versions of py3, so
I'm not sure what's going on here. This is might be related to init config
related changes in 3.8[1], since it works with 3.7.8, but fails with 3.8.1.
Somebody who understands encoding issues better than I do should give some
thought to if we need to make some changes to our encoding strategy on Windows
with py3.
With or without the flag/envvar, there is proper output if the command is
directly paged by piping to `more.com` (in any environment) or `less` (in MSYS
and WSL), or if paging is disabled with `--pager=no`. Legacy mode is required
though when Mercurial decides to spin up a pager.
[1] https://bugs.python.org/issue41941
Differential Revision: https://phab.mercurial-scm.org/D10756
Matt Harbison <matt_harbison@yahoo.com> [Thu, 20 May 2021 19:59:03 -0400] rev 47325
tests: partially fix test-transaction-rollback-on-revlog-split.t on Windows
For whatever reason, the extra space causes different output on Windows:
$ printf '% 20d' '1' | xxd -
0000000: 2020 2020 2020 2020 2020 2020 2020 2020
0000010: 2020 2020 31 1
vs WSL:
/mnt/c/Users/Matt/hg/tests$ printf '% 20d' '1' | xxd -
00000000: 2020 2020 2020 2020 2020 2020 2020 2020
00000010: 2020 2031
Without the space, they both agree the last word is 2031. There are other
failures on Windows that I don't understand that are introduced by 46b828b85eb7
and 18415fc918a1.
Differential Revision: https://phab.mercurial-scm.org/D10763
Matt Harbison <matt_harbison@yahoo.com> [Thu, 20 May 2021 19:30:56 -0400] rev 47324
tests: fix test-transaction-safety.t on Windows
Mostly this is about `C:\dir\sub` style paths being turned into `C:dirsub` by
MSYS, and not being about to execute shell scripts in hooks directly. See also
a12819559ccb::9e3979a25bfe.
Differential Revision: https://phab.mercurial-scm.org/D10762
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 19 May 2021 19:57:55 +0200] rev 47323
docket: move the uid logic in the `revlogutils.docket` module
We want to use it for revlog-v2 (& Co), it seems more logical to have the logic
lives inside the `docket` file than the `nodemap` file.
Differential Revision: https://phab.mercurial-scm.org/D10755
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 19 May 2021 17:12:06 +0200] rev 47322
persistent-nodemap: use the intended uuid size
I overlooked this in the initial implementation. The format already supporting
any uuid size so, nothing horrible should happens.
Differential Revision: https://phab.mercurial-scm.org/D10754
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 19 May 2021 17:19:46 +0200] rev 47321
persistent-nodemap: add a way to make the picked uid predictable
This will make the test less volatile and therefore easier to maintain. I am
setting this is for the persistent nodemap docket first, but my target is to use
similar logic for the revlogv2 (and co) test.
For persisent nodemap, they are too many different in how each test variants (with rust, without rust, etc) needs new uid for this be really useful (right now). However it will be directly useful for the revlogv2 logic.
Differential Revision: https://phab.mercurial-scm.org/D10753
Matt Harbison <matt_harbison@yahoo.com> [Thu, 06 May 2021 19:39:19 -0400] rev 47320
run-tests: disable color in 'auto' mode on Windows if stdout is redirected
Differential Revision: https://phab.mercurial-scm.org/D10761
Matt Harbison <matt_harbison@yahoo.com> [Thu, 06 May 2021 18:52:08 -0400] rev 47319
run-tests: enable color on Windows
In setting up the CI for Windows on heptapod, I noticed it was complaining about
color not being enabled because pygments wasn't installed- even though it was.
I had initially disabled color on Windows when using Windows 7, because that
didn't understand ANSI color codes and made a mess of the output. But now that
it's been unsupported for over a year, I don't think we should care about it
either.
It's admittedly a hack to depend on Mercurial proper to enable color support in
the terminal, but I didn't feel like duplicating that code. I'm under the
impression that 3rd party stuff is supposed to use this runner in the Mercurial
repo instead of using their own copy, so I think it's safe to assume the
Mercurial code is available. If it's not, it won't break anything.
Differential Revision: https://phab.mercurial-scm.org/D10760
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> [Sun, 16 May 2021 10:57:14 -0400] rev 47318
transaction: trivial refactoring
Differential Revision: https://phab.mercurial-scm.org/D10714
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> [Tue, 18 May 2021 23:36:21 -0400] rev 47317
test: remove some unnecessary dependency on repo format
Differential Revision: https://phab.mercurial-scm.org/D10757
Raphaël Gomès <rgomes@octobus.net> [Fri, 21 May 2021 17:37:53 +0200] rev 47316
hg-cpython: fix new occuring TypeError
dd339191f2dc introduced a minor refactoring of error types that highlighted
a fragile error creation mechanism that was in place in the Rust `MixedIndex`.
`PyErr::from_instance` also accepts a Python class, as long as it's an
`Exception` class. Before the aforementioned commit, we never ran into a case
where this duck-typing mechanism failed. We rectify this behavior by doing the
instantiation ourselves.
Differential Revision: https://phab.mercurial-scm.org/D10764
Martin von Zweigbergk <martinvonz@google.com> [Thu, 20 May 2021 08:15:57 -0700] rev 47315
errors: make exit codes class variables instead
Kyle pointed out to me that we can simply make the exit codes class
variables. Python provides some magic for making them accessible as
instance variables.
This also makes it easier to let subclasses of existing errors
override the exit codes by letting them simply define their own values
as class variables. That means that there's no need to pass them into
the superclass's constructor arguments, so the superclass doesn't need
to expose the them as arguments. (Making a subclass set a different
exit code for a subclass of `StorageError` was actually the goal with
my recent series.)
Differential Revision: https://phab.mercurial-scm.org/D10758
Charles Chamberlain <cchamberlain@janestreet.com> [Tue, 18 May 2021 13:08:42 -0400] rev 47314
narrow: widen when files are excluded by sparse and not included by narrow
In a repo where some directories are included by narrow and the complement are
excluded by sparse, it was previously impossible to widen either because trying
to widen narrow would complain that the requested files are outside the sparse
checkout and trying to widen sparse would complain that the requested files are
outside the narrow checkout.
This changes the `hg tracked --addinclude` command to only actually update any
newly accessible files in the dirstate if they are also accessible via sparse.
Differential Revision: https://phab.mercurial-scm.org/D10734
Augie Fackler <augie@google.com> [Tue, 18 May 2021 13:34:06 -0400] rev 47313
hghave: make error output easier to diagnose
I had a typo that meant the new bash check was throwing an exception
(due to a missing argument), but it was very hard to diagnose without
this change.
Differential Revision: https://phab.mercurial-scm.org/D10733
Augie Fackler <augie@google.com> [Tue, 18 May 2021 13:24:41 -0400] rev 47312
tests: add req on bash for test-transaction-rollback-on-sigpipe (issue6429)
I think we could work around this by rewriting the helper scripts in
Python, but I don't want to deal with that now and this should prevent
failures due to a lack of bash.
Differential Revision: https://phab.mercurial-scm.org/D10732
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 17 May 2021 15:24:46 +0200] rev 47311
updatecaches: deprecate the `full` argument
Now that all users were migrated, we can use deprecate the old way. This would
give potential extensions code a heads up on the API change.
Differential Revision: https://phab.mercurial-scm.org/D10731
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 17 May 2021 15:42:18 +0200] rev 47310
updatecaches: use the `caches` argument instead of a special `full` value
After a clone we want to update most cachem, but not exactly all of them. We
can now cleanly express this.
Differential Revision: https://phab.mercurial-scm.org/D10730
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 17 May 2021 15:27:29 +0200] rev 47309
updatecaches: use the caches argument in `hg debugupdatecaches`
This is the new way.
Differential Revision: https://phab.mercurial-scm.org/D10729
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 17 May 2021 14:45:16 +0200] rev 47308
updatecaches: adds a `caches` parameters to `repo.updatecaches`
It will superseed the `full` parameters (and its `post-clone` variant from
stable). Various caller will be updated in the rest of this series.
Differential Revision: https://phab.mercurial-scm.org/D10728
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 17 May 2021 14:41:09 +0200] rev 47307
updatecaches: introduce a set of constants to control which are updated
Passing around a set of constant to select what need warming will be cleaner
and more flexible. We did not changed the API yet, as this changes is already
large enough. In the rest of the rest we will change more code to actually use
this constants (or more realistically pre-defined set of constant directly)
Differential Revision: https://phab.mercurial-scm.org/D10727
Martin von Zweigbergk <martinvonz@google.com> [Tue, 18 May 2021 21:50:09 -0700] rev 47306
errors: make StorageError subclass Error, attaching an exit code to it
Differential Revision: https://phab.mercurial-scm.org/D10741
Martin von Zweigbergk <martinvonz@google.com> [Tue, 18 May 2021 19:33:09 -0700] rev 47305
revlog: avoid raising no-arg RevlogError for internal flow control
I'm about to make RevlogError require a `message` argument and this
code was failing. This patch refactors it to not raise an exception
for intra-function flow control.
Differential Revision: https://phab.mercurial-scm.org/D10740
Martin von Zweigbergk <martinvonz@google.com> [Tue, 18 May 2021 21:45:59 -0700] rev 47304
errors: catch the new Error class in scmutil and chgserver
Differential Revision: https://phab.mercurial-scm.org/D10739
Martin von Zweigbergk <martinvonz@google.com> [Tue, 18 May 2021 21:32:12 -0700] rev 47303
errors: create superclass for Abort exception
I'd like to let extensions subclass `StorageError` to define a custom
exit code. However, `StorageError` does not extend `Abort` (which is
where the exit code currently lives), and it seems that it's not
supposed to either (`StorageError` seems to be for lower-level errors
and `Abort` is for command-level errors). This patch therefore
extracts all the code from `Abort` into a new `Error` class, which
I'll soon make `StorageError` also extend.
Differential Revision: https://phab.mercurial-scm.org/D10738
Martin von Zweigbergk <martinvonz@google.com> [Tue, 18 May 2021 22:07:16 -0700] rev 47302
errors: make InterventionRequired subclass Abort
The docstring for `Abort` says that it's for errors raised by commands
and `InterventionRequired` is definitely something raised by commands,
so it seems that it should be an `Abort`. This patch makes it so. It
adds a `coarse_exit_code` (in addition to the already existing
`detailed_exit_code`) to `Abort` to achieve that, since
`InterventionRequired` should result in a special exit code even when
the `ui.detailed-exit-code` config is not set.
Differential Revision: https://phab.mercurial-scm.org/D10737
Martin von Zweigbergk <martinvonz@google.com> [Tue, 18 May 2021 21:58:12 -0700] rev 47301
errors: move Abort earlier, so more exceptions can subclass it
I'd like to make at least `InterventionRequired` subclass `Abort` and
Python requires the superclass to be defined before the subtype.
Differential Revision: https://phab.mercurial-scm.org/D10736
Martin von Zweigbergk <martinvonz@google.com> [Tue, 18 May 2021 17:15:49 -0700] rev 47300
errors: let each Abort subclass define its error code
It's more flexible to have the error codes defined on the error types
themselves. That way extensions can easily set their own exit code. It
also means that we can reduce a bit of duplication betwen
`scmutil.callcatch()` and `chgserver.chgcmdserver.validate()`.
Differential Revision: https://phab.mercurial-scm.org/D10735
Simon Sapin <simon.sapin@octobus.net> [Wed, 19 May 2021 15:10:49 +0200] rev 47299
rust: Fix "panic message is not a string literal" warnings
These deprecation warnings would not become errors until we actively port
crates to the (not yet released) Rust 2021 edition, but fixing them anyway
reduces console output noise.
Differential Revision: https://phab.mercurial-scm.org/D10743
Simon Sapin <simon.sapin@octobus.net> [Wed, 19 May 2021 15:08:27 +0200] rev 47298
rust: Add type annotation to fix inference on Rust Nightly
When compiling with Rust Nightly, the im-rs crate silently makes use of the
experimental language feature for trait impl specialization. This apperently
changes public its APIs in subtle ways such that type inference of some user
code can fail where it succeeds when specialization is disabled.
This made Mercurial’s Rust unit tests have compilation errors on Nightly.
I have not managed to find the exactl root cause, but I wrote down my findings
so far at https://github.com/bodil/im-rs/issues/188
This adds type annotation to make unit tests rely less on type inference
and work around the issue.
Differential Revision: https://phab.mercurial-scm.org/D10742
Joerg Sonnenberger <joerg@bec.de> [Wed, 19 May 2021 13:45:34 +0200] rev 47297
recover: only apply last journal record per file (issue6423)
This got broken in 2019 when the size check was introduced. It is most
noticable when dealing with transactions that involve an inline to
non-inline revlog storage transaction. It wasn't seen as much at the
time because the in-memory journal actually de-duplicated the entry
implicity, but since 63edc384d3b7 the on-disk journal is used for
rollback as well as recover.
Differential Revision: https://phab.mercurial-scm.org/D10726
Joerg Sonnenberger <joerg@bec.de> [Wed, 19 May 2021 13:46:19 +0200] rev 47296
revlog: update data file record before index rename
When migrating from inline to non-inline data storage, the data file is
recorded initially as zero sized so that it is removed on failure. But
the record has to be updated before the index is renamed, otherwise
data is lost on rollback.
Differential Revision: https://phab.mercurial-scm.org/D10725
Joerg Sonnenberger <joerg@bec.de> [Tue, 18 May 2021 02:35:27 +0200] rev 47295
revlog: fix index computation during inline->non-inline transition
The computation in 63edc384d3b7 failed to factor in the index entries
themselve as revlog.start() doesn't count them. Found by Valtenin
Gatienbaron with a more precise test case from me.
Differential Revision: https://phab.mercurial-scm.org/D10724
Simon Sapin <simon.sapin@octobus.net> [Wed, 19 May 2021 13:15:00 +0200] rev 47294
dirstate-v2: Change the on-disk format to be tree-shaped
Nodes are stored not only for tracked files but also for their ancestor
directories. A node has "pointers" (byte count from the start of the file)
to its direct child nodes. Everything can be accessed with zero copy.
Differential Revision: https://phab.mercurial-scm.org/D10722
Simon Sapin <simon.sapin@octobus.net> [Wed, 19 May 2021 13:15:00 +0200] rev 47293
dirstate-tree: Extract into a method sorting children of a given node
A later changset will use this in another place.
This is an associated function (that Python would call static method)
instead of a free function so it doesn’t need to be imported separately.
It’s on `Node` rather than `ChildNodes` because the latter is a type alias
to an external type (`HashMap`) so that would require an extension trait
which needs to be imported separately.
Differential Revision: https://phab.mercurial-scm.org/D10721
Simon Sapin <simon.sapin@octobus.net> [Wed, 19 May 2021 13:15:00 +0200] rev 47292
dirstate-v2: Add a variant of some tests, that uses the new format
With this, the new format receives some testing every time someone runs tests
with Rust extensions enabled, including on CI.
Differential Revision: https://phab.mercurial-scm.org/D10720
Simon Sapin <simon.sapin@octobus.net> [Wed, 19 May 2021 13:15:00 +0200] rev 47291
dirstate-v2: Change the on-disk format when the requirement is enabled
For now, the format is the same except with an additional marker at the start.
This marker is redundant: for existing repositories it is `.hg/requires` that
determines which format to use. For new repositories, it is the new
`format.exp-dirstate-v2` config. There is no upgrade or downgrade so far.
Most of the changes are about plumbing a boolean through layers of APIs to
indicate which format should be used.
Differential Revision: https://phab.mercurial-scm.org/D10719
Simon Sapin <simon.sapin@octobus.net> [Wed, 19 May 2021 13:15:00 +0200] rev 47290
dirstate-v2: Update the expected output of some tests for new requirement
Fix most test failures (except in test-narrow-debugrebuilddirstate.t and
test-upgrade-repo.t) caused by the new entry in config or in .hg/requires
when running `run-tests.py --extra-config-opt format.exp-dirstate-v2=1`
There is no CI so far for this configuration.
Differential Revision: https://phab.mercurial-scm.org/D10718
Simon Sapin <simon.sapin@octobus.net> [Wed, 19 May 2021 13:14:59 +0200] rev 47289
tests: More cleanly separate expected hexdump output
There are more lines that differ than are in common, and dirstate-v2
will complicate that further.
Differential Revision: https://phab.mercurial-scm.org/D10717
Simon Sapin <simon.sapin@octobus.net> [Wed, 19 May 2021 13:14:59 +0200] rev 47288
dirstate-v2: Add a new experimental `exp-dirstate-v2` repository requirement
This requirement is added to `.hg/requires` when creating a new repository
if Rust extensions are enabled and the `format.exp-dirstate-v2` config is set.
Nothing yet changes based on this requirement, but its mere presence affects
some tests (for example if they print `.hg/requires`). The next two changesets
update tests’ expected outputs accordingly.
There is no CI so far that enables this configuration.
Differential Revision: https://phab.mercurial-scm.org/D10716
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 16 Apr 2021 00:16:43 +0200] rev 47287
urlutil: provide some information about "bad url" when processing `pushurl`
It appears pushurl only support `<proto>://` entries. This is not obvious and
can lead to obscure error. We make the error less obscure as a start..
Differential Revision: https://phab.mercurial-scm.org/D10455
Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 11 Apr 2021 19:19:34 +0200] rev 47286
revset: document the `outgoing` behavior if the path resolve to multiple urls
Differential Revision: https://phab.mercurial-scm.org/D10454
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 16 Apr 2021 00:19:04 +0200] rev 47285
multi-urls: document the feature
We have a feature lets make it visible to people.
Differential Revision: https://phab.mercurial-scm.org/D10453
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 16 Apr 2021 00:16:47 +0200] rev 47284
multi-urls: add a boolean suboption that unlock path specification as list
When this option is set, a list of patch can be specifed as value for `[paths]`
entries. For the command who support it, this behave the same as providing
multiple destination of the command line.
Differential Revision: https://phab.mercurial-scm.org/D10452
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 15 Apr 2021 20:13:29 +0200] rev 47283
urlutil: move url "fixing" at the time of `ui.paths` initialization
Doing such fixing at the time is simpler and will be necessary to deal with
urls list in a sane manner. It also reduce the size of fix-config which is
always better.
I wish we could get ride of the hackish way to pass the root around, I suspect
that the `root` variable could be stored as part of the config value, along side
the source. However getting to the end of this `root` business is a far too
large detours to make now.
The test change to `tests/test-hgrc.t` and `test-config.t` are expectied since
we are not longer altering the config itself, but the way it is interpreted when
building path. This seems more correct.
I also added a couple of test call to `test-config.t` and `test-globalopts.t` to
clarify that the expanding process is properly happening a the right time.
Differential Revision: https://phab.mercurial-scm.org/D10451
Martin von Zweigbergk <martinvonz@google.com> [Mon, 17 May 2021 15:15:44 -0700] rev 47282
rewriteutil: add pointer to help text when rewrite would cause divergence
The evolve extension's version of the hint has this pointer. I missed
it when I moved it to core.
Differential Revision: https://phab.mercurial-scm.org/D10723
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 04 May 2021 11:20:10 +0200] rev 47281
changelogv2: use a dedicated on disk format for changelogv2
We drop two unused entry. This is mostly a proof of concept before starting to
actually rework the format.
Differential Revision: https://phab.mercurial-scm.org/D10667
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 04 May 2021 14:18:06 +0200] rev 47280
revlog: do not call Rust code if the index is not compatible with it
This will avoid hitting the TypeError we defined in the previous changesets.
This is the simplest fix but not the most elegant.
Ideally we would teach the Rust code to use any kind of revlog. However this is
an adventure for another time.
Differential Revision: https://phab.mercurial-scm.org/D10666
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 04 May 2021 14:16:26 +0200] rev 47279
revlog: signal which revlog index are compatible with Rust
Otherwise, Rust may treat python object like `cindex` object, leading to
trouble. The new attribute is an integer because I expect we might need a flag
field in the future.
As a start we get the rust code to raise a clear TypeError. We will use the
information in a smarter way in the next changesets.
Differential Revision: https://phab.mercurial-scm.org/D10665
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 04 May 2021 11:19:48 +0200] rev 47278
revlog: pass around the `rev` we deal with when packing/unpacking entry
This will help code to compute some flag on the fly.
Differential Revision: https://phab.mercurial-scm.org/D10664
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 29 Mar 2021 20:46:56 +0200] rev 47277
changelogv2: use a dedicated version number
If we want to change the format, we need a new version number. We start with
that.
Differential Revision: https://phab.mercurial-scm.org/D10663
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 04 May 2021 03:05:52 +0200] rev 47276
changelogv2: `copies-side-data` now implies `changelogv2`
To store information about file changes and copies only requires sidedata
support for the changelog. So we only enables `changelogv2`. This is less
impactful and should help us to ship the changeset centric copy tracing sooner.
Differential Revision: https://phab.mercurial-scm.org/D10662
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 04 May 2021 04:32:09 +0200] rev 47275
changelogv2: allow upgrade from and to this format
Differential Revision: https://phab.mercurial-scm.org/D10661
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 29 Mar 2021 22:40:54 +0200] rev 47274
changelogv2: introduce a "changelogv2" feature
Right now, this means using revlogv2, but only for the changelog. We will have
the format more unique in future changesets.
Differential Revision: https://phab.mercurial-scm.org/D10660
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 04 May 2021 05:18:57 +0200] rev 47273
revlogv2: drop the code related to inlined revlogv2
Tevlogv2 will not use inlined revlog. As a result we can drop the associated
code.
Differential Revision: https://phab.mercurial-scm.org/D10659
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 04 May 2021 02:55:56 +0200] rev 47272
revlogv2: no longer attempt to use inline for new revlog
revlogv2 steered away for the inline feature.
Differential Revision: https://phab.mercurial-scm.org/D10658
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 04 May 2021 02:52:24 +0200] rev 47271
sidedata: use the "feature" to detect that sidedata are enabled
This is more versatile than relying on the revlogv2 requirements.
Differential Revision: https://phab.mercurial-scm.org/D10657
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 23:40:05 +0200] rev 47270
revlog: compress sidedata when doing "post-pull" sidedata update
All path writing sidedata are now using compression (when appropriate).
Differential Revision: https://phab.mercurial-scm.org/D10656
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 23:14:48 +0200] rev 47269
revlog: compress sidedata in `_writeentry`
When appropriate we store the sidedata in a compressed form.
Differential Revision: https://phab.mercurial-scm.org/D10655
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 22:48:31 +0200] rev 47268
revlog: apply compression mode while reading sidedata
Right now, we only store things as PLAIN, but we are ready for some compression
now. Future changeset will start using some.
Differential Revision: https://phab.mercurial-scm.org/D10654
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 21:34:02 +0200] rev 47267
revlog: introduce a compression mode for sidedata in the revlog index
We will use this for compression for the sidedata payload.
Differential Revision: https://phab.mercurial-scm.org/D10653
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 21:13:24 +0200] rev 47266
revlog: implement a "default compression" mode
The revlog docker is now storing a default compression engine. When a chunk use
that compression, a dedicated mode is used in the revlog entry and we can
directly route it to the right decompressor.
We should probably make PLAIN and DEFAULT mode the only available mode for
revlogv2, but this is something for later.
Differential Revision: https://phab.mercurial-scm.org/D10652
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 21:04:55 +0200] rev 47265
revlog: add a `_get_decompressor` method
This logic is non-trivial and we will need to reuse it.
Differential Revision: https://phab.mercurial-scm.org/D10651
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 19:46:25 +0200] rev 47264
revlog: introduce a plain compression mode
That mode is simple it means the chunk contains uncompressed data and can be
used directly.
Differential Revision: https://phab.mercurial-scm.org/D10650
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 20:22:34 +0200] rev 47263
revlog: use an intermediate variable for `data` in `_chunk`
This will help us to dispatch this to the right decompressor.
Differential Revision: https://phab.mercurial-scm.org/D10649
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 18:41:51 +0200] rev 47262
revlogv2: preserve the compression mode on disk
The value is still the same and still not treated, but now it exists one disk.
Differential Revision: https://phab.mercurial-scm.org/D10648
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 04 May 2021 01:49:37 +0200] rev 47261
revlog: use `format_version` to dispatch different version in index_get
This is semantically more correct.
Differential Revision: https://phab.mercurial-scm.org/D10647
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 18:19:16 +0200] rev 47260
revlog: add a "data compression mode" entry in the index tuple
That will make it possible to keep track of compression information in the
revlog index, opening the way to more efficient revision restoration (in native
code, but the python usage is already defeating performance work).
We start with adding a new entry to the index tuple, using a value matching the
current behavior. We will introduce storage and other value in later changesets.
Differential Revision: https://phab.mercurial-scm.org/D10646
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 04 May 2021 01:15:03 +0200] rev 47259
revlog: fix the `null_item` attribute for V0
This is not a proper index tuple.
Differential Revision: https://phab.mercurial-scm.org/D10645
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 04 May 2021 01:13:53 +0200] rev 47258
revlog: create a create `null_item` attribute for V0
This value is currently wrong, but at least it is now -obviously- wrong.
Differential Revision: https://phab.mercurial-scm.org/D10644
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 16:52:38 +0200] rev 47257
revlog: improve documentation of the entry tuple
The code in revlog, and outside revlog directly use the index's entry tuple,
with direct integer indexing. This is a voluntary trade off to obtains better
performance from the Python code at the expense of the developers sanity.
Let's at least have a clear and central documentation about what this tuple is
about.
Differential Revision: https://phab.mercurial-scm.org/D10643
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 23:45:05 +0200] rev 47256
revlog: simplify entry update logic in `rewrite_sidedata`
We don't need to rebuild the full entry. So lets just gather the data we need
to pass to `replace_sidedata_info`.
Differential Revision: https://phab.mercurial-scm.org/D10642
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 04 May 2021 00:02:48 +0200] rev 47255
revlog: simplify the replace_sidedata_info code
We already know how to serialize and de-serialize and enty. So lets just do that
and modify the entry tuple directly.
This avoid having to duplicated binary operation in complicated code.
Differential Revision: https://phab.mercurial-scm.org/D10641
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 23:59:15 +0200] rev 47254
revlog: use `rev` instead of `i` in replace_sidedata_info
This is clearer.
Differential Revision: https://phab.mercurial-scm.org/D10640
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:35:35 +0200] rev 47253
revlogv2: also keep track for the size of the "data" file
This is useful to make sure we always start writing at the right location,
without effort.
Differential Revision: https://phab.mercurial-scm.org/D10632
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:35:25 +0200] rev 47252
revlogv2: track pending write in the docket and expose it to hooks
The docket is now able to write pending data. We could have used a distinct
intermediate files, however keeping everything in the same file will make it
simpler to keep track of the various involved files if necessary.
However it might prove more complicated for streaming clone. This will be dealt
with later.
Note that we lifted the stderr redirection in the test since we no longer suffer
from "unkown working directory parent" message.
Differential Revision: https://phab.mercurial-scm.org/D10631
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:35:14 +0200] rev 47251
revlog: move the `trypending` logic from the `changelog` to the `revlog`
We move the -reading- logic for the pending's '.a' suffixed index within the
revlog class. This is motivated by the fact the logic could be simpler and
cleaner if directly handled by the revlog docket. Before we can do so, we need
to teach the revlog code about reading "pending" changes. To be honest, we
already needed some special casing of the `.a` postfix, so this does not adds
much complexity.
The logic around -writing- the special '00changelog.i.a' remains in the
`changelog` class. Note that the revlog-v2 logic no longer use this logic.
The only remaining user of the `postfix` argument is the `censor` logic. We
could probably also make the revlog full aware of it (most of the code is
already implemented in revlog anyway) and get rid of the `postfix` argument and
logic. However this is an adventure for another time.
Since we have more information, we add more, paranoid, Programming error in case
we detect such "pending reader" trying to do a read (which does not happens
anyways).
Differential Revision: https://phab.mercurial-scm.org/D10630
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:35:02 +0200] rev 47250
revlogv2: delay the update of the changelog docket to transaction end
This prevent external reader to see the transaction content before it is
commited. However this also prevent the hooks to see the transaction content.
We will fix this in later changesets.
We have to temporarily suppress the error output of the command ran during the
transaction as they sometimes get confused about unknown working directory and
sometimes issue message on std-err in unspecified order.
Differential Revision: https://phab.mercurial-scm.org/D10629
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:34:52 +0200] rev 47249
revlogv2: track current index size in the docket
This help use to fix transaction safety on repos. See next changesets for details.
Differential Revision: https://phab.mercurial-scm.org/D10628
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:34:41 +0200] rev 47248
revlogv2: also test transactionality of revlog v2
Without surprise, this is broken :-)
Now that it is tested, we can fix it.
Differential Revision: https://phab.mercurial-scm.org/D10627
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:34:31 +0200] rev 47247
revlog: add a new test file focussed on testing transactionally issue
What we test here is currently also covered by `test-hooks.t`. However having
our own test file allow for more focussed/deep testing and to cover more variant
when relevant. So we create a new test file.
Differential Revision: https://phab.mercurial-scm.org/D10626
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:34:21 +0200] rev 47246
revlogv2: store version information in the docket only
Having it duplicated in the index was both useless and a risk of discrepancy.
Differential Revision: https://phab.mercurial-scm.org/D10625
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:34:11 +0200] rev 47245
revlogv2: introduce a very basic docket file
This is the first stone toward using a docket file in revlogv2. Right now the
docket is very basic and only store the version number (which is -also- stored
into the index file…) and the other files have fixed name. This new
implementation break transactionally… but they are no test checking
transactionally for revlogv2… So I take this as an opportunity to start small.
They are no usage of revlogv2 outside of tests anyway.
The docket keeps the `.i` naming used by previous version index to preserve a
unique entry point. We could decide to use a different name and look it up
first, or to fully rework this in a future "store" version. However that does
not seems necessary right now.
We will re-introduces transactionality (and associated testing…) in a later
changesets.
A long list of TODOs have been added to the relevant comment.
Differential Revision: https://phab.mercurial-scm.org/D10624
Raphaël Gomès <rgomes@octobus.net> [Mon, 17 May 2021 15:05:24 +0200] rev 47244
branching: merge stable into default
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:34:01 +0200] rev 47243
revlogv2: mark revlogv2 as requires a full upgrade
This was overlooked previously.
Differential Revision: https://phab.mercurial-scm.org/D10623
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:30:46 +0200] rev 47242
revlog: unify flag processing when loading index
The new code use a simple declaration to do centralised processing. This is
clearer, shorter and less error prone. This will be especially useful as we plan
to add a fourth format: changelog-v2.
Differential Revision: https://phab.mercurial-scm.org/D10622
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:30:35 +0200] rev 47241
revlog: unify checks for supported flag
The new code use a simple declaration to do centralised checking. This is
clearer, shorter and less error prone. This will be especially useful as we plan
to add a fourth format: changelog-v2.
Differential Revision: https://phab.mercurial-scm.org/D10621
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:30:24 +0200] rev 47240
side-data: drop the associated config and requirements
This is no longer and extensions of revlog v1, but a part of revlog v2. We no
longer needs independent config and requirement management for that.
Differential Revision: https://phab.mercurial-scm.org/D10620
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:30:14 +0200] rev 47239
revlog: fix capitalisation of an error
We don't start error message with capital letters.
Differential Revision: https://phab.mercurial-scm.org/D10619
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:30:03 +0200] rev 47238
test: no longer directly use the sidedata config and requirements
They are on their way out, so we simply use revlogv2 in these cases.
Some test does not requires explicite request for the revlogv2 format since the
upgrade to a side-data-capable format is impled by
`exp-use-copies-side-data-changeset`.
Differential Revision: https://phab.mercurial-scm.org/D10618
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:29:52 +0200] rev 47237
sidedata: add a 'side-data' repository feature and use it
Most code don't really care how sidedata support is added, but it needs to know
if it is present. To achieve this. we use the `repo.features` attributes with a
new dedicated features".
Having such centralised information is more robust and will help us to remove the
sidedata requirements in a later changesets.
Differential Revision: https://phab.mercurial-scm.org/D10617
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:29:41 +0200] rev 47236
statichttp: add the missing `features` attribute
This make statichttp closer to the interface it is "supposed" to follow.
Differential Revision: https://phab.mercurial-scm.org/D10616
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:29:30 +0200] rev 47235
sidedata: use revlogv2 requirement in the test helper
We are about to drop the requirement dedicated to sidedata (since revlogv2 will
support them unconditionally and previous version will not.).
To prepare this more we adapt the test code to not use that requirements.
Differential Revision: https://phab.mercurial-scm.org/D10615
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:29:19 +0200] rev 47234
upgrade: display sidedata before upgrade
This is a small gratuitous change that help making the test simpler to
understand on its own.
Differential Revision: https://phab.mercurial-scm.org/D10614
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:29:09 +0200] rev 47233
revlog: determine sidedata support based on the revlog version
Revlog version "2" support sidedata, previous version does not. So lets make the
initialization simpler.
Differential Revision: https://phab.mercurial-scm.org/D10613
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:28:58 +0200] rev 47232
requirements: no longer drop `generaldelta` requirement with revlogv2
A repository could use a mix of revlogv1 and revlogv2, making the requirements
still necessary. Overall we should move away from the "requirements" file being
used a way to configure the repository and stick to it "what do you need to
access this repository". However this is a wider work for another time.
In addition the logic we just dropped was confusing the `hg debugformat`
command, breaking the upgrade code and inconsistent (eg: `sparse-revlog` is also
implied by `revlogv2`).
Finally, multiple other config option would imply the use of the `revlogv2`
requirements, without drop the `generaldelta` one, leading to more
inconsistency.
Differential Revision: https://phab.mercurial-scm.org/D10612
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:28:47 +0200] rev 47231
config: drop the `format.exp-revlogv2.2` option
It is oddly named and not used at all. The one used by the code is still
`experimental.revlogv2`. So we drop that one option for consistency. We move
associate documentation to the actual option.
Differential Revision: https://phab.mercurial-scm.org/D10611
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:28:36 +0200] rev 47230
revlog: only use the `_indexfp` method for read operation
This will avoid "other" code to not overlook `_writing` usage. We introduces
private method dedicated to writing to make use the right option are always
used.
Differential Revision: https://phab.mercurial-scm.org/D10610
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:28:26 +0200] rev 47229
revlog: use `_writing` in `rewrite_sidedata`
Special cases aren't special enough to break the rules.
Differential Revision: https://phab.mercurial-scm.org/D10609
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:28:15 +0200] rev 47228
revlog: open files in 'r+' instead of 'a+'
The code doing actual writing is already doing the necessary seeking, so we
could safely use 'r+'. This make the file objecs usable in more situation, like
updating the sidedata information during pull.
revlog: forcibly move the file cursor at the right location before writing
This is a paranoid change in case the changelog computation moved the cursors
under our feets.
This is not known to happens right now.
Differential Revision: https://phab.mercurial-scm.org/D10608
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:28:05 +0200] rev 47227
revlog: pass a transaction object to `rewrite_sidedata`
The `_writing` context need one, so we update the function signature before considering using `_writing` in rewrite_sidedata.
Differential Revision: https://phab.mercurial-scm.org/D10607
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:27:53 +0200] rev 47226
revlog: rename variable in `rewrite_sidedata` to match other code
Let's call the index file object and `ifh` and the data file object `dfh` as the
rest of the revlog code. This will make future change clearer.
Differential Revision: https://phab.mercurial-scm.org/D10606
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:27:42 +0200] rev 47225
revlog: introduce a mandatory `_writing` context to update revlog content
Before this change, various revlog methods where managing the opening and
closing of the revlog files manually and passing the file descriptor alors the
call path. To simplify the tracking of the write operation by a future docket,
we need something more organised. As a result, we introduce a `revlog._writing`
context manager that will wrap each revlog update operation. The file
descriptor are kept in the existing `revlog._writinghandles` parameter that
was already used by the `addgroup` logic.
All this change is internal to the revlog only, the "public" interface is not
affected. The `addrevision` and `addgroup` logic are still responsible for
setup up this context. However this new context give us multiple benefits:
* all writer use a same, unified, logic,
* this context is programmatically enforced,
* each write "session" as a clearly identified start and end.
The post-pull sidedata update logic is still doing writing by end and will be
adjusted in a later changesets.
This change affect the concurrency checker test, because register the state of
the file in the transaction sooner in `addrevision` (about as early as what
`addgroup` would do), so the abort is rollbacking the other commit. I don't want
to weaken the current main logic.
Differential Revision: https://phab.mercurial-scm.org/D10605
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:27:31 +0200] rev 47224
revlog: preindent some code in _enforceinlinesize
Indenting this beforehand will make a future changeset much simpler.
Differential Revision: https://phab.mercurial-scm.org/D10604
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:27:20 +0200] rev 47223
revlog: preindent some code in addgroup
Indenting this beforehand will make a future changeset much simpler.
Differential Revision: https://phab.mercurial-scm.org/D10603
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 06 May 2021 09:27:22 -0700] rev 47222
pyoxidizer: use Python 3.9 (BC)
Newer versions of PyOxidizer use Python 3.9 by default. We
previously pinned the version to 3.8 to facilitate porting to
a new PyOxidizer version and diffing results.
Now that the porting work is complete, let's bump Python
to Python 3.9.
This will effectively change our Windows Inno and WiX Python 3
installers from Python 3.8 to 3.9.
.. bc::
Windows .msi and .exe installers now use Python 3.9 instead of
Python 3.8.
Differential Revision: https://phab.mercurial-scm.org/D10689
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 06 May 2021 16:11:13 -0700] rev 47221
packaging: use PyOxidizer for producing WiX MSI installer
We recently taught our in-tree PyOxidizer configuration file to
produce MSI installers with WiX using PyOxidizer's built-in
support for doing so.
This commit changes our WiX + PyOxidizer installer generation
code to use this functionality.
After this change, all the Python packaging code is doing is the
following:
* Building HTML documentation
* Making gettext available to the build process.
* Munging CLI arguments to variables for the `pyoxidizer` execution.
* Invoking `pyoxidizer build`.
* Copying the produced `.msi` to the `dist/` directory.
Applying this stack on stable and rebuilding the 5.8 MSI installer
produced the following differences from the official 5.8 installer:
* .exe and .pyd files aren't byte identical (this is expected).
* Various .dist-info/ directories have different names due to older
versions of PyOxidizer being buggy and not properly normalizing
package names. (The new behavior is correct.)
* Various *.dist-info/RECORD files are different due to content
divergence of files (this is expected).
* The python38.dll differs due to newer PyOxidizer shipping a newer
version of Python 3.8.
* We now ship python3.dll because PyOxidizer now includes this file
by default.
* The vcruntime140.dll differs because newer PyOxidizer installs a newer
version. We also now ship a vcruntime140_1.dll because newer versions
of the redistributable ship 2 files now.
The WiX GUIDs and IDs of installed files have likely changed as a
result of PyOxidizer's different mechanism for generating those
identifiers. This means that an upgrade install of the MSI will
replace files instead of doing an incremental update. This is
likely harmless and we've incurred this kind of breakage before.
As far as I can tell, the new PyOxidizer-built MSI is functionally
equivalent to the old method. Once we drop support for Python 2.7
MSI installers, we can delete the WiX code from the repository.
This commit temporarily drops support for extra `.wxs` files. We
raise an exception instead of silently not using them, which I think
is appropriate. We should be able to add support back in by injecting
state into pyoxidizer.bzl via `--var`. I just didn't want to expend
cognitive load to think about the solution as part of this series.
Differential Revision: https://phab.mercurial-scm.org/D10688
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 06 May 2021 16:07:01 -0700] rev 47220
packaging: extract invocation of pyoxidizer to own function
I'll be refactoring how the WiX installer creation calls into
pyoxidizer and will need a lower level function for facilitating
that.
The new `run_pyoxidizer()` builds our execution environment
(with gettext available) and invokes `pyoxidizer`.
Differential Revision: https://phab.mercurial-scm.org/D10687
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 06 May 2021 16:06:20 -0700] rev 47219
packaging: rename run_pyoxidizer()
I'm going to split this function up in a future commit and I'll
want the name "run_pyoxidizer()" for a lower-level function for
invoking `pyoxidizer`.
Differential Revision: https://phab.mercurial-scm.org/D10686
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 06 May 2021 08:37:40 -0700] rev 47218
packaging: move documentation HTML building to own function
This is part of some light refactoring to enable us to use
PyOxidizer for WiX MSI installer generation.
Differential Revision: https://phab.mercurial-scm.org/D10685
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 06 May 2021 16:04:24 -0700] rev 47217
pyoxidizer: support code signing
Newer versions of PyOxidizer feature built-in support for
code signing. You simply declare a code signer in the Starlark
configuration file, activate it for automatic signing, and
PyOxidizer will add code signatures to signable files as it
encounters them.
This commit teaches our Starlark configuration file to enable
automatic code signing. But only on Windows for the moment, as our
immediate goal is to overhaul the Windows packaging.
The feature is opt-in: you must pass variables to PyOxidizer's
build context via `pyoxidizer build --var` or
`pyoxidizer build --var-env` to activate code signing.
Differential Revision: https://phab.mercurial-scm.org/D10684
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 06 May 2021 16:03:43 -0700] rev 47216
pyoxidizer: support producing MSI installers
Newer versions of PyOxidizer have support for building WiX MSI
installers "natively." Essentially, you can script the definition
of your WiX installer via Starlark and PyOxidizer can invoke
WiX tools to produce the installer.
This commit teaches our PyOxidizer config file to produce MSI
installers similarly to how
`contrib/packaging/packging.py wix` would do it.
We had to make a very minor change to `mercurial.wxs` to reflect
different paths depending on who builds. This is because when
PyOxidizer builds WiX installers, it does so from an isolated
directory, not Mercurial's source directory. We simply copy the
files into the build environment so they are accessible.
After this change, running `pyoxidizer build msi` produces a nearly
identical install layout to what the previous method produces.
When I applied this series on top of the 5.8 tag, here is the
list of differences and explanations:
* docs/*.html files are missing from the new installer because the
Python build environment doesn't have docutils.
* .pyd and .exe files differ, likely because I'm using a different
Visual Studio toolchain on my local computer than the official build
environment.
* Various .dist-info/ directories have different names. This is
because older versions of PyOxidizer had buggy behavior and weren't
properly normalizing package names in .dist-info/ directories. e.g.
we went from `cached-property-1.5.2.dist-info` to
`cached_property-1.5.2.dist-info`.
* Translations (.mo files) may be missing if gettext isn't in %Path%.
This is because the packaging.py code installs gettext and ensures
it can be found.
* Some *.dist-info/RECORD files vary due to SHA-256 content digest
divergence due to build environment differences. (This should be
harmless.)
* The new install layout ships a python3.dll because newer versions
of PyOxidizer ship this file.
* The new install layout has a different vcruntime140.dll and also a
vcruntime140_1.dll because newer versions of PyOxidizer ship a
newer version of the Visual C++ Redistributable Runtime.
The new PyOxidizer functionality is not yet integrated with
packaging.py. This will come in a subsequent commit. So for now, the
new functionality introduced here is unused.
Differential Revision: https://phab.mercurial-scm.org/D10683
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 06 May 2021 16:16:21 -0700] rev 47215
pyoxidizer: use allocator_backend instead of raw_allocator
The name of this attribute changed in PyOxidizer 0.11.
Differential Revision: https://phab.mercurial-scm.org/D10682
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 06 May 2021 15:58:37 -0700] rev 47214
pyoxidizer: simplify targets
The split targets existed to enable the use of a non-default
distribution flavor on Windows. Modern versions of PyOxidizer
use the "standalone_dynamic" distribution flavor by default.
So our split brain workaround is no longer needed.
Here, we unify the targets. We also remove an unreferenced
target function to create a resources file.
Differential Revision: https://phab.mercurial-scm.org/D10681
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 06 May 2021 15:56:04 -0700] rev 47213
contrib: install PyOxidizer 0.16.0
PyOxidizer now provides MSI installers and pre-built Linux binaries.
So we install that way. This significantly reduces the time to bootstrap
a new machine in automation, as building PyOxidizer from source on a low
core count machine takes several minutes.
This change temporarily breaks the ability of the automated environment
to use the in-repo pyoxidizer.bzl configuration file, as there are
backwards-incompatible changes with the upgrade that need to be
reflected. We'll handle those in subsequent commits.
Differential Revision: https://phab.mercurial-scm.org/D10680
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 06 May 2021 16:13:33 -0700] rev 47212
pyoxidizer: pin Python to 3.8
Newer versions of PyOxidizer use version 3.9 by default. As part of
upgrading PyOxidizer, we want to pin the version at 3.8 so we can
compare differences more easily.
Differential Revision: https://phab.mercurial-scm.org/D10679
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 06 May 2021 16:00:44 -0700] rev 47211
pyoxidizer: remove some boilerplate in file
This isn't used for anything and can safely be removed.
Differential Revision: https://phab.mercurial-scm.org/D10678
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 06 May 2021 15:45:04 -0700] rev 47210
contrib: upgrade Rust in Windows automation
I'm not sure why we don't install the minimum required Rust version
here like we do for Linux. Whatever: that's unrelated to wanting to
stay modern.
Differential Revision: https://phab.mercurial-scm.org/D10677
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 06 May 2021 15:44:29 -0700] rev 47209
contrib: update Python versions in Windows automation
Let's keep our Python versions modern.
Differential Revision: https://phab.mercurial-scm.org/D10676
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 06 May 2021 15:41:52 -0700] rev 47208
automation: update rust in Linux environment
Our minimum supported Rust is 1.41.1 per rust/README.rst. We also
bump the modern Rust version to latest stable to stay current.
Differential Revision: https://phab.mercurial-scm.org/D10675
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 06 May 2021 15:40:47 -0700] rev 47207
automation: install latest Python versions in Linux environment
Let's keep our Linux environment up to date.
Differential Revision: https://phab.mercurial-scm.org/D10674
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 06 May 2021 17:46:57 -0700] rev 47206
automation: create Python 3.5 variant of requirements.txt
The automation environment is refusing to build with the previous
file because some dependencies won't install on Python 3.5.
I couldn't find an easy way to salvage the situation with a single
requirements.txt file. So, I decided to introduce a variant
for Python 3.5.
As part of this, we update packages to latest versions.
(I do question why we are still supporting Python 3.5...)
Differential Revision: https://phab.mercurial-scm.org/D10690
Karthikeyan Singaravelan <tir.karthi@gmail.com> [Sat, 15 May 2021 09:45:10 +0000] rev 47205
hgweb: Fix deprecation warning in Python 3.10 (issue6520)
Differential Revision: https://phab.mercurial-scm.org/D10711
Simon Sapin <simon.sapin@octobus.net> [Mon, 10 May 2021 21:59:13 +0200] rev 47204
dirstate-tree: Remove newly-empty nodes after removing a `DirstateEntry`
This is actually necessary to make `DirstateMap::has_dir` correct, since it
assumes that a node without a `DirstateEntry` has at least one descedant node
with a `DirstateEntry`.
This bug would become apparent when a later changeset persists tree nodes
on disk in the "dirstate-v2" format.
Differential Revision: https://phab.mercurial-scm.org/D10706
Simon Sapin <simon.sapin@octobus.net> [Mon, 10 May 2021 21:31:05 +0200] rev 47203
dirstate-tree: Refactor DirstateMap::drop_file to be recursive
It should behave the same as before. This will enable the next changeset
to run code on the way "down" (in order to removing newly-empty nodes).
Differential Revision: https://phab.mercurial-scm.org/D10705
Martin von Zweigbergk <martinvonz@google.com> [Tue, 11 May 2021 12:22:26 -0700] rev 47202
templates: fix `revset('parents()') % ...` in amend message template
I don't understand why, but putting `revset('parents()') % {desc}` in
the commit message template for amend resulted in a crash because
`memctx.hex()` did `hex(self.node())` and its node was None. This
patch fixes that.
Martin von Zweigbergk <martinvonz@google.com>
Differential Revision: https://phab.mercurial-scm.org/D10707
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 15 Apr 2021 19:54:03 +0200] rev 47201
path: teach the `hg path` command to display boolean sub-option
The next changeset introduce one.
Differential Revision: https://phab.mercurial-scm.org/D10450
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 16 Apr 2021 01:18:28 +0200] rev 47200
parselist: move the function from config to stringutil
We move the function in a lower level module to avoid cycle. It moves next to
`parsebool` who had to migrate for the same reasons.
Differential Revision: https://phab.mercurial-scm.org/D10449
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 16 Apr 2021 02:14:21 +0200] rev 47199
bookmark: use `list_paths` to access path definition
The content of the `[paths]` config section is receiving transformation that
make it hard to recognise whats the actual name, and the next changeset will
make it worse. So we use the official API for this instead.
Differential Revision: https://phab.mercurial-scm.org/D10448
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 15 Apr 2021 16:58:20 +0200] rev 47198
urlutil: make `paths` class old list of `path`
We move from a `{name → path}` mapping to a `{name → [path]}` mapping. And
update all user code accordingly. For now, all the list contains exactly one
element, but we are now in a good place to make the config understand a list of
url.
Differential Revision: https://phab.mercurial-scm.org/D10447
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 15 Apr 2021 17:15:43 +0200] rev 47197
urlutil: extract `chain_path` in a function
This will no longer modify `path` inplace so it does not make much sense as a method.
Differential Revision: https://phab.mercurial-scm.org/D10446
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 15 Apr 2021 17:12:25 +0200] rev 47196
urlutil: add a `copy` method to `path
This will be useful when inheriting from multiple path at the same time.
Differential Revision: https://phab.mercurial-scm.org/D10445
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 15 Apr 2021 12:33:05 +0200] rev 47195
template: add a `paths` field to all entry in peerurls
This make it possible to display multiple path per name in the near future.
Differential Revision: https://phab.mercurial-scm.org/D10444
Simon Sapin <simon.sapin@octobus.net> [Tue, 11 May 2021 18:10:59 +0200] rev 47194
status: Add some more tests
Trying to improve coverage for various non-ovbious scenarios
Differential Revision: https://phab.mercurial-scm.org/D10704
Matt Harbison <matt_harbison@yahoo.com> [Sat, 08 May 2021 00:21:31 -0400] rev 47193
run-tests: use the same python version for shebang lines on Windows
The latest py3 is used if the minor number isn't specified. After running the
script to install all of the build dependencies, that moved the default from 3.8
to 3.9 on the CI system. That in turn caused a bunch of tests to be skipped
that were running prior, even when the test runner was invoked with `py -3.8`.
While we should almost always use the latest version, we really shouldn't make
it hard to test different versions or allow things to randomly break in subtle
ways like that.
Differential Revision: https://phab.mercurial-scm.org/D10702
Karthikeyan Singaravelan <tir.karthi@gmail.com> [Fri, 14 May 2021 10:01:29 +0000] rev 47192
vfs: Fix deprecation warning in Python 3.10 (issue6520)
Differential Revision: https://phab.mercurial-scm.org/D10710
Raphaël Gomès <rgomes@octobus.net> [Wed, 12 May 2021 10:24:17 +0200] rev 47191
contrib: fix typo
I forgot to fix it in flight, this commit will do fine.
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:27:09 +0200] rev 47190
revlog: rename `indexdata` to entry_data
Same reasoning as the previous changeset, we might not be looking at index data
here.
Differential Revision: https://phab.mercurial-scm.org/D10602
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:26:59 +0200] rev 47189
revlog: use "entry_point" phrasing for loading the revlog
The main entry for the revlog will not necessary be the index, but a small
"docket". So we change the variable names and we move the initialisation of the
index_file and data_file after that first entry point have been read.
Differential Revision: https://phab.mercurial-scm.org/D10601
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:26:48 +0200] rev 47188
revlog: directly use self._format_flags when loading index
The shorthand variable does not bring much, so we drop it to simplify the code.
Differential Revision: https://phab.mercurial-scm.org/D10600
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:26:37 +0200] rev 47187
revlog: directly use self._format_version when loading index
The shorthand variable does not bring much, so we drop it to simplify the code.
Differential Revision: https://phab.mercurial-scm.org/D10599
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:26:27 +0200] rev 47186
revlog: use `_format_flags` to access flags instead of `header`
It seems better to reuse the variable we carefully extracted
This also open the way to more flexible way to retrieve these flags.
Differential Revision: https://phab.mercurial-scm.org/D10598
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:26:17 +0200] rev 47185
revlog: rename `newversionflags` to `new_header`
This make it consistent with the previous changeset.
Differential Revision: https://phab.mercurial-scm.org/D10597
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:26:06 +0200] rev 47184
revlog: rename `versionflags` to header
The variable is named as such because it contains "version" information and
"flags" information. However you mostly needs to know the code to understand the
name which is not great. The fact that this is the very first four bytes in all
revlog seems more relevant, so we rename the variable "header".
Differential Revision: https://phab.mercurial-scm.org/D10596
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:25:56 +0200] rev 47183
revlog: move index reading logic in a dedicated method
They are multiple motivation to do it:
* The logic is complicated enough to deserver its own method.
* We will need to reuse this once we put a docket in use.
* This split the actual reading from the processing of the read data better.
Differential Revision: https://phab.mercurial-scm.org/D10595
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:25:45 +0200] rev 47182
revlog: define the actual index and datafile at loading time
This is just code movement, to make the code closer to where we actually use it
and where it will be defined in the future.
Differential Revision: https://phab.mercurial-scm.org/D10594
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:25:34 +0200] rev 47181
revlog: simplify a conditionnal in _enforceinlinesize
This is a gratuitous change to make things a bit easier to read.
Differential Revision: https://phab.mercurial-scm.org/D10593
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:25:23 +0200] rev 47180
revlog: drop `flush` parameter from `_peek_iscensored`
This is not used anywhere.
Differential Revision: https://phab.mercurial-scm.org/D10592
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:25:12 +0200] rev 47179
revlog: fix error message when data are missing
The error message the message was not using the requested offset, but the
adjusted offset to that read more data for improved caching. This resulted in
confusing error message.
Differential Revision: https://phab.mercurial-scm.org/D10591
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:25:01 +0200] rev 47178
revlog: rename `nodemap_file` to `_nodemap_file`
Same reasoning as for `indexfile and datafile`, lets hide these implementation
details.
Differential Revision: https://phab.mercurial-scm.org/D10590
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:24:50 +0200] rev 47177
revlog: use revlog.display_id in error related to bad revisions
Differential Revision: https://phab.mercurial-scm.org/D10589
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:24:40 +0200] rev 47176
revlog: use revlog.display_id in "revision too big" errors
Differential Revision: https://phab.mercurial-scm.org/D10588
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:24:29 +0200] rev 47175
revlog: use revlog.display_id in censor related errors
Differential Revision: https://phab.mercurial-scm.org/D10587
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:24:18 +0200] rev 47174
revlog: use revlog.display_id in integrity error
Differential Revision: https://phab.mercurial-scm.org/D10586
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:24:08 +0200] rev 47173
revlog: use revlog.display_id in ambiguity errors
Differential Revision: https://phab.mercurial-scm.org/D10585
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:23:58 +0200] rev 47172
revlog: use revlog.display_id for corruption error
Differential Revision: https://phab.mercurial-scm.org/D10584
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:23:48 +0200] rev 47171
revlog: use revlog.display_id in format related errors
Differential Revision: https://phab.mercurial-scm.org/D10583
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:23:37 +0200] rev 47170
revlog: use revlog.display_id in narrow error message
Differential Revision: https://phab.mercurial-scm.org/D10582
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:23:27 +0200] rev 47169
revlog: use revlog.display_id in LookupError
Differential Revision: https://phab.mercurial-scm.org/D10581
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:23:17 +0200] rev 47168
revlog: use revlog.display_id for FilteredLookupError
Differential Revision: https://phab.mercurial-scm.org/D10580
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:23:07 +0200] rev 47167
revlog: introduce a `display_id` property
We currently using the "index file" to identify a revlog in error output. Since
we are about to make the "index file" location more volatile, we need something
better. We move to use the "radix", as it is close to what we currently use.
We could probably do better, as pointed out in the comment, however that would
be a quite detour from my current goal.
Differential Revision: https://phab.mercurial-scm.org/D10579
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:22:57 +0200] rev 47166
revlog: also use radix when computing nodemap data file
We have a radix, lets use it!
Differential Revision: https://phab.mercurial-scm.org/D10578
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:22:47 +0200] rev 47165
revlog: stop usage of `_indexfile` to computing nodemap path
We now have the radix explicitely lets use the radix explicitely
Differential Revision: https://phab.mercurial-scm.org/D10577
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:22:36 +0200] rev 47164
revlog: use a "radix" to address revlog
Instead of pointing to the index directly and to derive the other file from
that, we directly provide the radix and let the revlog determine the associated
file path internally. This is more robust and will give us more flexibility for
picking this file name in the future.
Differential Revision: https://phab.mercurial-scm.org/D10576
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:22:26 +0200] rev 47163
revlog: rename `datafile` to `datafile`
We want to make the actual location of the datafile and location more of an
implementation details than what is is currently. In that process, we make the
attribute private.
Differential Revision: https://phab.mercurial-scm.org/D10575
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:22:16 +0200] rev 47162
revlog: rename `indexfile` to `_indexfile`
We want to make the actual location of the indexfile and location more of an
implementation details than what is is currently. In that process, we make the
attribute private.
Differential Revision: https://phab.mercurial-scm.org/D10574
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:22:06 +0200] rev 47161
filelog: drop `indexfile` from `filelog`
Since `filelog` objects are not revlog (no really, they are not…) we drop the
revlog specific attribute. We need to directly access the underlying revlog in a
couple of place that already assume that we have a revlog here.
This is motivated by future change to that revlog attribute.
Differential Revision: https://phab.mercurial-scm.org/D10573
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:21:56 +0200] rev 47160
manifest: drop the `indexfile` from `manifestrevlog`
Since `manifestrevlog` object are not revlog (no really, they are not…) we drop
the revlog specific attribute. We need to directly access the underlying revlog
in a couple of place that already assume that we have a revlog here.
This is motivated by future change to that revlog attribute.
Differential Revision: https://phab.mercurial-scm.org/D10572
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:21:46 +0200] rev 47159
revlog: deal with special "postfix" explicitely
revlog usually use a straight forward '.i' and '.d' naming except for two cases
"in-transaction" changelog, and censoring. Our goal is to let the revlog code
deal with the internal of the file naming itself. To do so, we need to start
dealing with these postfix explicitly.
Differential Revision: https://phab.mercurial-scm.org/D10571
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:21:35 +0200] rev 47158
revlog: split the option initialisation in its own method
The part of the code is huge, keeping it separated will keep the `_loadindex`
method simpler and help keeping logic well insulated.
Differential Revision: https://phab.mercurial-scm.org/D10570
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:21:25 +0200] rev 47157
revlog: always "append" full size tuple
Same reasoning as the previous patch.
Differential Revision: https://phab.mercurial-scm.org/D10569
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:21:15 +0200] rev 47156
revlog: make the index always return the same tuple
It is simpler to manage the diferrence in on disk format in the internal index
code itself and lets the rest of the code always handle the same object.
This will become even more important when the data we store will be entirely
different (for example the changelog does not need the "linkrev" field.
We start with item reading, we will deal with item writing in the next
changesets.
Differential Revision: https://phab.mercurial-scm.org/D10568
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:21:05 +0200] rev 47155
revlog: introduce an explicit `format_version` member in the index struct
This will allow for cleaner check than assuming each version has a different
size. Unsurprisingly I am planning to use this to introduce more format variant.
Differential Revision: https://phab.mercurial-scm.org/D10567
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:20:55 +0200] rev 47154
revlog: rename `hdrsize` to `entry_size` in the C code
This is the size of and index entry, so lets make it clearer.
Differential Revision: https://phab.mercurial-scm.org/D10566
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:20:45 +0200] rev 47153
revlog: split the `version` attribute into its two components
The `revlog.version` attribute contained an integer coding 2 different informations:
* the revlog version number
* a bit field defining some specific feature of the revlog
We now explicitly store the two components independently. This avoid exposing
the implementation details all around the code and prepare for future revlog
version that would encode the information in a different way.
In the process we drop the `version` attribute from the interface. It was
flagged for removal when that interface was created.
Differential Revision: https://phab.mercurial-scm.org/D10565
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:20:35 +0200] rev 47152
verify: pass a revlog to `_checkrevlog` in `_verifymanifest`
Since `manifestrevlog` is not a `revlog`, we are passing strange thing to
`_checkrevlog`. We fix this to avoid breakage during future change.
Differential Revision: https://phab.mercurial-scm.org/D10564
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:20:25 +0200] rev 47151
revlog: replace flag check related to generaldelta with attribute check
Same logic as the previous changesets.
Differential Revision: https://phab.mercurial-scm.org/D10563
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:19:09 +0200] rev 47150
revlog: replace REVLOGV2 check related to sidedata with `hassidedata` checks
This is more flexible and semantically more correct. The associated revlog's
attribute exist since 827cb4fe62a3, so well we start linking sidedata to
revlogv2.
Differential Revision: https://phab.mercurial-scm.org/D10562
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:19:05 +0200] rev 47149
revlog: explicitely pass the "indexfile" parameter
Most of this was already done when introducing the `target` parameter, but some
remained. Having "indexfile" passed explicitely will help us to change the way
we address a revlog later in the stack. With the introduction of more generic
`docket`, the entry point will not necessarly be `xxx.i` file, and the actual
index files will have a variable name.
Differential Revision: https://phab.mercurial-scm.org/D10561
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:18:58 +0200] rev 47148
revlog: highlight current incompatibility in `rewrite_sidedata`
See comment for details. We will need to fix the test coverage when this
incompatibility is lifted.
Differential Revision: https://phab.mercurial-scm.org/D10544
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:18:48 +0200] rev 47147
revlog: adjust rewrite_sidedata code to not delete existing revlog content
The "w+" file mode is deleting all the content of the opened file. Which is bad…
This is not caught by the test because the test only check for a full, initial
pull where not pre-existing content exists. So we need to fix our test coverage
here.
However they are another issue that prevent "incremental" pull to work here. See
next changeset for details.
Differential Revision: https://phab.mercurial-scm.org/D10543
Simon Sapin <simon.sapin@octobus.net> [Fri, 07 May 2021 17:33:47 +0200] rev 47146
status: Add tests for some more edge cases
* Size-preserving file contents modification
* Filtering output to a deleted or removed file
Differential Revision: https://phab.mercurial-scm.org/D10701
Simon Sapin <simon.sapin@octobus.net> [Fri, 07 May 2021 16:44:36 +0200] rev 47145
status: Extend issue 6483 test to exclude patterns
With `hg status -X`, not just include pattern with `hg status -I`
Differential Revision: https://phab.mercurial-scm.org/D10700
Simon Sapin <simon.sapin@octobus.net> [Fri, 07 May 2021 16:41:07 +0200] rev 47144
dirstate-tree: Add a test showing that issue 6335 is fixed
… when using the new status algorithm and the tree-based dirstate.
The previous algorithm still has this bug.
Differential Revision: https://phab.mercurial-scm.org/D10699
Simon Sapin <simon.sapin@octobus.net> [Mon, 03 May 2021 20:04:19 +0200] rev 47143
dirstate-tree: Add a dirstate-v1-tree variant of some tests
The `dirstate-v1` variant has the previous behavior.
`dirstate-v1-tree` uses the same format on disk, but uses the new
`DirstateMap` with a tree data structure and the new `status` algorithm.
These were untested so far.
Differential Revision: https://phab.mercurial-scm.org/D10698