Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 30 May 2021 20:42:51 +0200] rev 47349
censor: do not process sidedata of censored revision while bundling
The revision is censored, we should ignore it.
Differential Revision: https://phab.mercurial-scm.org/D10801
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 28 May 2021 20:00:27 +0200] rev 47348
changegroup: fix deltachunk API to be consistent from one class to another
Depending of the subclass the 8th index of `chunkdata` items was either a
sidedata dict of a proto_flags integer. We have not fixed the inconsistency and
we already return fixed "delta" items from `deltaiter`.
Differential Revision: https://phab.mercurial-scm.org/D10778
Augie Fackler <augie@google.com> [Thu, 27 May 2021 12:10:59 -0400] rev 47347
fuzz: add hg to sys.path when constructing mpatch corpus
Differential Revision: https://phab.mercurial-scm.org/D10777
Simon Sapin <simon.sapin@octobus.net> [Wed, 19 May 2021 16:18:16 +0200] rev 47346
dirstate-tree: Skip readdir() in `hg status -mard`
When running the status algorithm in a mode where we don’t list unknown
or ignored files, all we care about are files that are listed in the dirstate.
We can there for skip making expensive calls to readdir() to list the contents
of filesystem directories, and instead only run stat() to get the filesystem
state of files listed in the dirstate. (This state may be an error for files
that don’t exist anymore on the filesystem.)
On 16 CPU threads, this reduces the time spent in the `status()` function for
`hg status -mard` on an old snapshot of mozilla-central from ~70ms to ~50ms.
Differential Revision: https://phab.mercurial-scm.org/D10752
Simon Sapin <simon.sapin@octobus.net> [Wed, 19 May 2021 13:15:00 +0200] rev 47345
dirstate-v2: Parse the dirstate lazily, with copy-on-write nodes
TODO: more description
Differential Revision: https://phab.mercurial-scm.org/D10751
Simon Sapin <simon.sapin@octobus.net> [Wed, 19 May 2021 13:15:00 +0200] rev 47344
dirstate-v2: Make the dirstate bytes buffer available in more places
Differential Revision: https://phab.mercurial-scm.org/D10750
Simon Sapin <simon.sapin@octobus.net> [Wed, 19 May 2021 13:15:00 +0200] rev 47343
dirstate-v2: Make more APIs fallible, returning Result
When parsing becomes lazy, parse error will potentially happen in more places.
This propagates such errors to callers.
Differential Revision: https://phab.mercurial-scm.org/D10749
Simon Sapin <simon.sapin@octobus.net> [Wed, 19 May 2021 13:15:00 +0200] rev 47342
dirstate-v2: Add a zero-size error type for dirstate v2 parse errors
This error should only happen if Mercurial is buggy or the file is corrupted.
It indicates for example that:
* A part of the file refers to another part, and the byte offset or item count
would cause reading out of bounds, beyond the end of the file.
* The byte for an entry state has an invalid value
When parsing becomes lazy, many more functions will return a `Result` with
this error. Making it zero-size reduces the work that the `?` operator needs
to do to pass around the error value.
Differential Revision: https://phab.mercurial-scm.org/D10748
Simon Sapin <simon.sapin@octobus.net> [Wed, 19 May 2021 13:15:00 +0200] rev 47341
dirstate-tree: Add `NodeRef` and `ChildNodesRef` enums
They are used instead of `&Node` and `&ChildNodes` respectively.
The `ChildNodes` type alias also becomes a similar enum.
For now they only have one variant each, to be extended later.
Adding enums now forces various use sites go through new methods
instead of manipulating the underlying data structure directly.
Differential Revision: https://phab.mercurial-scm.org/D10747
Simon Sapin <simon.sapin@octobus.net> [Wed, 19 May 2021 13:15:00 +0200] rev 47340
rust: Return owned instead of borrowed DirstateEntry in DirstateMap APIs
This will enable the tree-based DirstateMap to not always have an actual
DirstateEntry in memory for all nodes, but construct it on demand.
Differential Revision: https://phab.mercurial-scm.org/D10746
Simon Sapin <simon.sapin@octobus.net> [Wed, 19 May 2021 13:15:00 +0200] rev 47339
dirstate-tree: Downgrade `&mut Node` to `&Node` in status and serialization
Mutable access is not used, and upcoming changes will make it more costly
(with copy-on-write nodes that can be read from disk representation)
Differential Revision: https://phab.mercurial-scm.org/D10745
Simon Sapin <simon.sapin@octobus.net> [Wed, 19 May 2021 13:15:00 +0200] rev 47338
dirstate-tree: Remove DirstateMap::iter_node_data_mut
In an upcoming changeset we want DirstateMap to be able to work directly
with nodes in their "on disk" representation, without always allocating
corresponding in-memory data structures. Nodes would have two possible
representations: one immutable "on disk" refering to the bytes buffer
of the contents of the .hg/dirstate file, and one mutable with HashMap
like the curren data structure.
These nodes would have copy-on-write semantics: when an immutable node
would need to be mutated, instead we allocate new mutable node for it and
its ancestors.
A mutable iterator of the entire tree would still be possible, but it would
become much more expensive since we’d need to allocate mutable nodes for
everything.
Instead, remove this iterator. It was only used to clear ambiguous mtimes
while serializing the `DirstateMap`. Instead clearing and serialization are
now two separate passes. Clearing first uses an immutable iterator to collect
the paths of nodes that need to be cleared, then accesses only those nodes
mutably.
Differential Revision: https://phab.mercurial-scm.org/D10744
Matt Harbison <matt_harbison@yahoo.com> [Fri, 28 May 2021 17:33:20 -0400] rev 47337
merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 26 May 2021 21:46:45 +0200] rev 47336
revlog: close the index file handle after the data one
This make sure the data file is flushed before the index. preventing the index
to reference unflushed data.
Differential Revision: https://phab.mercurial-scm.org/D10776
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 26 May 2021 21:35:51 +0200] rev 47335
revlog: simplify the try nesting in the `_writing` context
Lets use a single try, with conditional cleanup. This make is easier to add a
file handle dedicated to sidedata.
Differential Revision: https://phab.mercurial-scm.org/D10775
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 20 May 2021 21:54:21 +0200] rev 47334
revlogv2: add a `get_data` helper to grab the next piece of docket
This make the processing more compact but abstracting repetitive processing
away.
Differential Revision: https://phab.mercurial-scm.org/D10774
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 20 May 2021 21:48:53 +0200] rev 47333
revlogv2: simplify and clarify the processing of each entry
As we add more entries and some of them has non trivial processing it seems
useful to make the processing leaner and clearly separated to simplify futures
patches.
Differential Revision: https://phab.mercurial-scm.org/D10773
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 19 May 2021 16:55:36 +0200] rev 47332
revlogv2: use a unique filename for data
Having a unique data will allow for ambiguity less rewriting of revlog content,
something useful to clarify handling of some operation like censoring or
stripping.
Differential Revision: https://phab.mercurial-scm.org/D10772
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 18 May 2021 15:07:17 +0200] rev 47331
revlogv2: use a unique filename for index
Having a unique index will allow for ambiguity less rewriting of revlog content,
something useful to clarify handling of some operation like censoring or
stripping.
Differential Revision: https://phab.mercurial-scm.org/D10771
Simon Sapin <simon.sapin@octobus.net> [Sat, 22 May 2021 17:35:54 +0200] rev 47330
rhg: A missing .hg/dirstate file is not an error
Instead treat it as like an empty file
Differential Revision: https://phab.mercurial-scm.org/D10766
Simon Sapin <simon.sapin@octobus.net> [Sat, 22 May 2021 17:32:09 +0200] rev 47329
rhg: Sort `rhg status` output correctly
* The relative order of states is:
modified, added, removed, deleted, unknown, ignored, clean
* Files in the same state should be sorted by name, regardless of whether
or not the were in "unsure" state based on metadata alone.
Differential Revision: https://phab.mercurial-scm.org/D10765
Simon Sapin <simon.sapin@octobus.net> [Wed, 19 May 2021 18:35:43 +0200] rev 47328
dirstate-v2: Add `hg debugupgraderepo` command support
This command changes changes the file formats used inside an existing
repository to what they would be in a new repository with the current config.
For example:
hg debugupgraderepo --config format.exp-dirstate-v2=1 --run
hg debugupgraderepo --config format.exp-dirstate-v2=0 --run
If a repository has a dirstate in v1 format, the first command would upgrade it
to dirstate-v2. Conversely, if a repository has a dirstate in v2 format, the
second command would downgrade it to v1. (Both may also run some unrelated
upgrades.)
Since `format.exp-dirstate-v2` is currently disabled by default, not specifying
it in `--config` or any configuration file would result in the second command.
Differential Revision: https://phab.mercurial-scm.org/D10769
Simon Sapin <simon.sapin@octobus.net> [Fri, 21 May 2021 17:12:47 +0200] rev 47327
upgrade: Use `improvement` subclasses everywhere, not instances
This changes the source definition of optimizations to match that of formats:
a subclass with a decorator, instead of an instance passed to a function call.
Not having any instance removes the confusion between class attributes and
instance attributes, which were used interchangeably.
Differential Revision: https://phab.mercurial-scm.org/D10768
Matt Harbison <matt_harbison@yahoo.com> [Tue, 11 May 2021 01:05:38 -0400] rev 47326
exewrapper: avoid directly linking against python3X.dll
Subsequent code calls `LoadLibrary()` to attempt to load the DLL, but because of
this symbol reference, there is an attempt to load the DLL used during the build
prior to `_main()` running. This causes the whole process to fail if the DLL
isn't in the standard search path. That also means it will never load the DLL
for HackableMercurial. (Maybe we should get rid of that for py3, since you can
install python for a user without admin rights?)
This could also be resolved by calling `GetProcAddress()` on the symbol and
dereferencing it, but using the environment variable is consistent with the
*.bat file since fc8a5c9ecee0. (The environment variable persists after the
interpreter is initialized.)
Far more concerning is somehow I've gotten my system into a state where setting
the flag causes any output to the pager to be lost (as if it wasn't set at all)
in MSYS, cmd.exe, WSL, and PowerShell using py3.9.0, but the environment
variable works properly. I'm sure this flag worked on some versions of py3, so
I'm not sure what's going on here. This is might be related to init config
related changes in 3.8[1], since it works with 3.7.8, but fails with 3.8.1.
Somebody who understands encoding issues better than I do should give some
thought to if we need to make some changes to our encoding strategy on Windows
with py3.
With or without the flag/envvar, there is proper output if the command is
directly paged by piping to `more.com` (in any environment) or `less` (in MSYS
and WSL), or if paging is disabled with `--pager=no`. Legacy mode is required
though when Mercurial decides to spin up a pager.
[1] https://bugs.python.org/issue41941
Differential Revision: https://phab.mercurial-scm.org/D10756
Matt Harbison <matt_harbison@yahoo.com> [Thu, 20 May 2021 19:59:03 -0400] rev 47325
tests: partially fix test-transaction-rollback-on-revlog-split.t on Windows
For whatever reason, the extra space causes different output on Windows:
$ printf '% 20d' '1' | xxd -
0000000: 2020 2020 2020 2020 2020 2020 2020 2020
0000010: 2020 2020 31 1
vs WSL:
/mnt/c/Users/Matt/hg/tests$ printf '% 20d' '1' | xxd -
00000000: 2020 2020 2020 2020 2020 2020 2020 2020
00000010: 2020 2031
Without the space, they both agree the last word is 2031. There are other
failures on Windows that I don't understand that are introduced by 46b828b85eb7
and 18415fc918a1.
Differential Revision: https://phab.mercurial-scm.org/D10763
Matt Harbison <matt_harbison@yahoo.com> [Thu, 20 May 2021 19:30:56 -0400] rev 47324
tests: fix test-transaction-safety.t on Windows
Mostly this is about `C:\dir\sub` style paths being turned into `C:dirsub` by
MSYS, and not being about to execute shell scripts in hooks directly. See also
a12819559ccb::9e3979a25bfe.
Differential Revision: https://phab.mercurial-scm.org/D10762
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 19 May 2021 19:57:55 +0200] rev 47323
docket: move the uid logic in the `revlogutils.docket` module
We want to use it for revlog-v2 (& Co), it seems more logical to have the logic
lives inside the `docket` file than the `nodemap` file.
Differential Revision: https://phab.mercurial-scm.org/D10755
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 19 May 2021 17:12:06 +0200] rev 47322
persistent-nodemap: use the intended uuid size
I overlooked this in the initial implementation. The format already supporting
any uuid size so, nothing horrible should happens.
Differential Revision: https://phab.mercurial-scm.org/D10754