Raphaël Gomès <rgomes@octobus.net> [Mon, 14 Nov 2022 15:29:58 +0100] rev 49750
hg-core: remove unneeded trait now that we support Rust 1.52+
Raphaël Gomès <rgomes@octobus.net> [Mon, 14 Nov 2022 15:20:48 +0100] rev 49749
rust: remove newly redundant `use` statements with the 2021 edition prelude
https://doc.rust-lang.org/edition-guide/rust-2021/prelude.html
Raphaël Gomès <rgomes@octobus.net> [Mon, 14 Nov 2022 15:19:27 +0100] rev 49748
rust: move all crates in the main workspace to edition 2021
We've changed our minimum Rust version to 1.61.0 in the previous patch,
and edition 2021 predates that version.
Raphaël Gomès <rgomes@octobus.net> [Thu, 20 Oct 2022 12:26:57 +0200] rev 49747
rust: upgrade supported Rust toolchain version
A few months ago¹, a decision was made to move the Rust toolchain target to
whatever Debian Testing was tracking. I didn't have the bandwidth to act on
it until now.
This is starting to be even more problematic than before, now that edition 2021
is out.
The CI has been updated to track the current Debian testing version, 1.61.0.
[1] https://lists.mercurial-scm.org/pipermail/mercurial-packaging/2022-April/000338.html
Raphaël Gomès <rgomes@octobus.net> [Sat, 19 Nov 2022 20:40:47 +0100] rev 49746
branching: merge stable into default
Pierre-Yves David <pierre-yves.david@octobus.net> [Sat, 19 Nov 2022 16:43:02 +0100] rev 49745
tests: fix test-sparse-revlog
This one is not covered by the CIbecause I requires an expensive artifact to be
cached. So it goes out of think on regular basis (we should fix that…)
The test ouput was affected by e706bb41fdb3 as we filtering now happens sooner,
removing for the output.
Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 06 Nov 2022 15:03:31 -0500] rev 49744
delta-find: adjust the moment when we mark something as "tested"
In a coming change, not all elements of `group` might get tested. So we need to
have more control about when a revision is actually added to the `tested` set.
So we move to a more verbose (and more fragile) version.
Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 06 Nov 2022 12:53:57 -0500] rev 49743
delta-find: rename a variable for clarity
the index in the delta-chain is also the snapshot depth. So we rename the
variable for clarity.
Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 06 Nov 2022 12:53:03 -0500] rev 49742
delta-find: small documentation update
This is not a 1-1 mapping, but a 1-n mapping. Lets make the associated comment clearer.
Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 06 Nov 2022 12:51:50 -0500] rev 49741
delta-find: move pre-filtering with other pre-filtering logic
This is more consistent and will help use to be in a clean state before dealing
with the "too large group" issue.
As a side effect, the debug output now skip some useless cases, making it more useful.
Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 06 Nov 2022 13:46:08 -0500] rev 49740
delta-find: expand a function definition and call before extendin it
This make the next changeset more compact.
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 17 Oct 2022 22:19:43 +0200] rev 49739
debug: add an option to display statistic about a unbundling operation
This will helps a lot to understand how the bundling decision have actually
impacted pull/unbundle on the other side.
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 15 Nov 2022 16:25:23 +0100] rev 49738
debug: add an option to display statistic about a bundling operation
This will helps a lot to understand how the bundling decision might impact
pull/unbundle on the other side.
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 15 Nov 2022 18:08:56 +0100] rev 49737
delta-find: add debug information about reuse of cached data
This will help us to understand the behavior of find-delta during a pull.
Matt Harbison <matt_harbison@yahoo.com> [Tue, 08 Nov 2022 18:05:19 -0500] rev 49736
cffi: fix a bytes vs str issue on macOS when listing directories
This code hasn't been touched in recent years, and the other implementation
return bytes for the filename, so I assume this is a holdover from the py2 days.
I was unable to test it on mac though, because the `_osutil` import failed.
Matt Harbison <matt_harbison@yahoo.com> [Tue, 08 Nov 2022 14:31:10 -0500] rev 49735
typing: fix the typehint for `skip` arg on `osutil.listdir()` to be optional
The cffi and pure implementations both default to `None`, and the args are
parsed in C with `"y#|OO:listdir"`, with the C variable initialized to NULL.
Matt Harbison <matt_harbison@yahoo.com> [Tue, 08 Nov 2022 14:17:56 -0500] rev 49734
typing: add type hints to mpatch implementations
Again, using `merge-pyi` to apply the stubs in cext and then manually type the
private methods. The generated stub without these hints inferred very little,
and the stuff it did was wrong.
Matt Harbison <matt_harbison@yahoo.com> [Tue, 08 Nov 2022 13:59:16 -0500] rev 49733
typing: add type hints to bdiff implementations
Not super important code, but this was an exercise in using `merge-pyi` to fold
type stubs back into the code on something small. The cext stubs don't seem to
be getting used (at least the only thing in `.pytype/pyi/mercurial/cext` after a
run generating the stubs is `__init__.pyi`), so maybe this will help some.
Matt Harbison <matt_harbison@yahoo.com> [Tue, 08 Nov 2022 13:52:46 -0500] rev 49732
cffi: adjust the list returned by bdiff.blocks to never have a None entry
This was flagged by pytype after merging the corresponding bdiff.pyi in cext:
File ".../mercurial/cffi/bdiff.py", line 44, in blocks: bad return type [bad-return-type]
Expected: List[Tuple[int, int, int, int]]
Actually returned: List[None]
AFAICT, all callers immediately unpack the tuple into 4 variables, so a `None`
entry would simply crash if they aren't all overwritten. As long a `count` and
the link list are consistent, this shouldn't be a problem.
This placates both pytype and PyCharm (which complained about the `i` in `rl[i]`
having the wrong type with the old code).
Matt Harbison <matt_harbison@yahoo.com> [Tue, 08 Nov 2022 13:38:06 -0500] rev 49731
typing: fix a syntax error in mercurial/cext/bdiff.pyi
I noticed because `merge-pyi` on the non-cext implementations with this file as
input skipped the return type for this. `pytype-single --parse-pyi` confirmed
it was a problem.
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 04 Nov 2022 19:38:47 -0400] rev 49730
debug-delta-find: add a --source option
This will help us to understand the delta-find operation in different
situations.
Jason R. Coombs <jaraco@jaraco.com> [Wed, 02 Nov 2022 09:34:03 -0400] rev 49729
demandimport: ensure lazyloaderex sets loader attributes (issue6725)
Adds test capturing missed expectation.
Jason R. Coombs <jaraco@jaraco.com> [Tue, 30 Aug 2022 09:59:53 -0400] rev 49728
shelve: add test for Shelf.changed_files
Jason R. Coombs <jaraco@jaraco.com> [Mon, 29 Aug 2022 14:11:26 -0400] rev 49727
shelve: add Shelf.changed_files for resolving changed files in a plugin
Raphaël Gomès <rgomes@octobus.net> [Mon, 14 Nov 2022 10:59:09 +0100] rev 49726
branching: merge stable into default
Mathias De Mare <mathias.de_mare@nokia.com> [Mon, 07 Nov 2022 09:25:20 +0100] rev 49725
configitems: enable pullbundles by default
The use of pullbundles is already protected: they are only used
when a pullbundles.manifest file is created on the server.
Having an additional flag doesn't really make sense
and can confuse users (as indicated in the mercurial mailing list
on the topic "Can't get pull-bundles working").
Matt Harbison <matt_harbison@yahoo.com> [Fri, 04 Nov 2022 22:59:16 -0400] rev 49724
typing: add basic type hints to stringutil.py
Matt Harbison <matt_harbison@yahoo.com> [Fri, 04 Nov 2022 17:54:43 -0400] rev 49723
vfs: make the default opener mode binary
The default was already binary for `abstractvfs`, and the `vfs` implementation
adds binary mode if the caller didn't supply it. Therefore, it should be safe
for all vfs objects (and I don't think we want text reads anyway).
Matt Harbison <matt_harbison@yahoo.com> [Wed, 02 Nov 2022 17:30:57 -0400] rev 49722
typing: add basic type hints to vfs.py
Again, there's a lot more that could be done, but this sticks to the obviously
correct stuff that is related to primitives or `vfs` objects. Hopefully this
helps smoke out more path related bytes vs str issues in TortoiseHg.
PyCharm seems smart enough to apply hints from annotated superclass functions,
but pytype isn't (according to the *.pyi file generated), so those are annotated
too.
There was some discussion about changing the default path arg from `None` to
`b''` in order to avoid the more verbose `Optional` declarations. This would be
more in line with `os.path.join()` (which rejects `None`, but ignores empty
strings), and still not change the behavior for callers still passing `None`
(because the check is `if path` instead of an explicit check for `None`). But I
didn't want to hold this up while discussing that, so this documents what _is_.
Matt Harbison <matt_harbison@yahoo.com> [Fri, 04 Nov 2022 17:35:44 -0400] rev 49721
util: implement `writelines()` on atomictempfile
With typehints on the vfs objects, pytype will flag this:
FAILED: /mnt/c/Users/Matt/hg/.pytype/pyi/mercurial/patch.pyi
/usr/bin/python3.8 -m pytype.single
--imports_info /mnt/c/Users/Matt/hg/.pytype/imports/mercurial.patch.imports
--module-name mercurial.patch -V 3.7
-o /mnt/c/Users/Matt/hg/.pytype/pyi/mercurial/patch.pyi
--analyze-annotated --nofail --quick
/mnt/c/Users/Matt/hg/mercurial/patch.py
File "/mnt/c/Users/Matt/hg/mercurial/patch.py", line 535, in writerej:
No attribute 'writelines' on mercurial.util.atomictempfile [attribute-error]
In Union[
mercurial.util.atomictempfile,
mercurial.vfs.checkambigatclosing,
mercurial.vfs.delayclosedfile,
mercurial.windows.fdproxy,
mercurial.windows.mixedfilemodewrapper
]
It's not a real problem there (atomictempfile is only created by passing
different args), but it's reasonable for this to implement the function and
behave like a normal file. There are other functions missing that can be added
if/when needed.
Matt Harbison <matt_harbison@yahoo.com> [Wed, 02 Nov 2022 16:43:01 -0400] rev 49720
typing: add basic type hints to localrepo.py
There's a lot more that could be done, but this sticks to the obviously correct
stuff that is either related to existing imports or primitives. Hopefully this
helps smoke out more path related bytes vs str issues in TortoiseHg.
I'm avoiding the interfaces for now, because they seem to confuse pytype and/or
PyCharm. It might be worth typing the return of `makelocalrepository` to
`localrepository`, but that leaks an implementation detail, so that can be
revisited later.
Matt Harbison <matt_harbison@yahoo.com> [Sat, 05 Nov 2022 00:38:02 -0400] rev 49719
check-code: drop the check for whitespace around named parameters
This check flags py3 annotations of named parameters, because `black` adds
spaces around the assignment in this case. Since the chosen formatter has
opinions (and pylint also wants the space in the case of annotations), drop the
check so we can use py3 annotations.
Raphaël Gomès <rgomes@octobus.net> [Wed, 19 Oct 2022 12:38:06 +0200] rev 49718
rust-status: query fs traversal metadata lazily
Currently, any time the status algorithm needs to read a directory from the
filesystem (because the stat-only optimization is not available), it also
stats each directory entry eagerly.
Stat'ing the entries is only needed in a few cases (like when checking
the mtime of a directory for caching): this patch creates a wrapper struct
`DirEntry` that only stats the directory entry it represents when needed.
Excerpt of an `strace` before this change on Mozilla Central:
```
openat(AT_FDCWD, ".", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 3
newfstatat(3, "", {st_mode=S_IFDIR|0755, st_size=3540, ...}, AT_EMPTY_PATH) = 0
getdents64(3, 0x55dc970bd440 /* 139 entries */, 32768) = 5072
statx(3, ".hg", AT_STATX_SYNC_AS_STAT|AT_SYMLINK_NOFOLLOW, STATX_ALL, {stx_mask=STATX_ALL|STATX_MNT_ID, stx_attributes=0, stx_mode=S_IFDIR|0755, stx_size=772, ...}) = 0
[... 135 other successful `statx` calls]
getdents64(3, 0x55dc970bd440 /* 0 entries */, 32768) = 0
close(3) = 0
```
After this change:
```
openat(AT_FDCWD, ".", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 3
newfstatat(3, "", {st_mode=S_IFDIR|0755, st_size=3540, ...}, AT_EMPTY_PATH) = 0
getdents64(3, 0x561567c10190 /* 139 entries */, 32768) = 5072
getdents64(3, 0x561567c10190 /* 0 entries */, 32768) = 0
close(3) = 0
```
Raphaël Gomès <rgomes@octobus.net> [Wed, 19 Oct 2022 14:46:19 +0200] rev 49717
rust-status: make `DirEntry` attributes clearer
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 22 Sep 2022 16:27:17 +0200] rev 49716
tests: remove non-python3 line matching and tests block
We don't support Python2 anymore
Anton Shestakov <av6@dwimlabs.net> [Mon, 31 Oct 2022 16:50:22 +0400] rev 49715
pywatchman: remove obsolete comments about importing from future
See 6000f5b25c9b.
Anton Shestakov <av6@dwimlabs.net> [Mon, 31 Oct 2022 16:36:00 +0400] rev 49714
demandimport: remove an obsolete comment about importing from future
See 6000f5b25c9b.
Matt Harbison <matt_harbison@yahoo.com> [Wed, 19 Oct 2022 11:10:54 -0400] rev 49713
mr-template: wrap the instructions inside a comment block
At least in preview mode, this hides the text so the user doesn't have to delete
it. It's still visible in edit mode, so the user sees it.
Matt Harbison <matt_harbison@yahoo.com> [Wed, 19 Oct 2022 11:50:40 -0400] rev 49712
revlog: use the user facing filename as the display_id for filelogs
I had trouble isolating some LFS blob corruption detected by `hg verify` because
the traceback referenced a file, but with the `data/` prefix in the `.hg/store`
path, so it couldn't be located with the `file()` revset:
```
Traceback (most recent call last):
File "/mnt/d/mercurial/mercurial/revlog.py", line 3209, in verifyintegrity
_verify_revision(self, skipflags, state, node)
File "/mnt/d/mercurial/hgext/lfs/wrapper.py", line 246, in _verify_revision
orig(rl, skipflags, state, node)
File "/mnt/d/mercurial/mercurial/revlog.py", line 158, in _verify_revision
rl.revision(node)
File "/mnt/d/mercurial/mercurial/revlog.py", line 1816, in revision
return self._revisiondata(nodeorrev, _df)
File "/mnt/d/mercurial/mercurial/revlog.py", line 1870, in _revisiondata
self.checkhash(text, node, rev=rev)
File "/mnt/d/mercurial/mercurial/revlog.py", line 1996, in checkhash
% (self.display_id, pycompat.bytestr(revornode))
mercurial.error.RevlogError: integrity check failed on data/EXE/PPC/shrinksrec.exe:0
```
(I'm a little surprised it resulted in a stacktrace instead of just a message,
but that's a different issue. I'm also not sure how to trigger the simplestore
case, since IIUC, it's also a revlog based store.)
It's not clear how to handle the changelog and manifest (because the user
doesn't interact with them as a file), so those cases are left alone. The other
thing that would be nice to improve somehow is to indicate that the ":0" is a
revlog revision, not the changeset revision that users are used to. I'm not
sure how to handle the "or node" part though.
Matt Harbison <matt_harbison@yahoo.com> [Wed, 19 Oct 2022 11:24:20 -0400] rev 49711
revlog: drop an unused variable assignment
It's assigned again 2 lines later.
Matt Harbison <matt_harbison@yahoo.com> [Thu, 20 Oct 2022 13:12:37 -0400] rev 49710
lfs: improve an exception message for blob corruption detected on transfer
The message about the server crash originated in 0ee0a3f6a990 (after support for
serving blobs was added), but was copied from the Facebook repo that forked
prior to server side support. Therefore, this message only displayed in their
client, so it was safe to assume the server crashed. But that was never the
case for vanilla Mercurial, as I saw this in a server log.
Also, display the blob reference so that it's easier to figure out where the
problem was when a bunch of blobs are transferred at once.
Raphaël Gomès <rgomes@octobus.net> [Mon, 24 Oct 2022 17:35:30 +0200] rev 49709
branching: merge stable into default
Raphaël Gomès <rgomes@octobus.net> [Thu, 02 Mar 2023 15:21:36 +0100] rev 49708
Added signature for changeset 8830004967ad
Raphaël Gomès <rgomes@octobus.net> [Thu, 02 Mar 2023 15:21:23 +0100] rev 49707
Added tag 6.3.3 for changeset 8830004967ad
Raphaël Gomès <rgomes@octobus.net> [Thu, 02 Mar 2023 15:07:47 +0100] rev 49706
relnotes: add 6.3.3
Matt Harbison <matt_harbison@yahoo.com> [Tue, 28 Feb 2023 16:42:38 -0500] rev 49705
typing: add typehints to mercurial/diffutil.py
Lack of typehints here caused the fact that TortoiseHg was passing str instead
of bytes as the key in `opts` to be missed, resulting in shelf corruption in
cases where `diff.git` is required.
Matt Harbison <matt_harbison@yahoo.com> [Tue, 28 Feb 2023 18:14:11 -0500] rev 49704
patchbomb: respect the `--git` option
I *think* this is the only diffopt exposed on the command line. TortoiseHg had
a similar issue creating diffopts, and this was caught by type hints in the next
commit.
Raphaël Gomès <rgomes@octobus.net> [Wed, 01 Mar 2023 16:48:09 +0100] rev 49703
rhg: remember the inode of .hg/dirstate
This allows us to detect changes of `.hg/dirstate`, which is either the
full dirstate (in dirstate-v1) or the docket file (v2) without relying on
data inside the file. It only works on UNIX systems.
This fixes a race condition for dirstate-v1 (as demonstrated by
the test changes) and adds a confortable layer of sanity for dirstate-v2.
Raphaël Gomès <rgomes@octobus.net> [Tue, 28 Feb 2023 17:58:15 +0100] rev 49702
rust-dirstate-v2: don't write dirstate if data file has changed
This fixes the following race:
- process A reads the dirstate
- process B reads and writes the dirstate
- process A writes the dirstate
This either resulted in losing what process B had just written or a crash
because the `uuid` had changed and we were trying to write to a file that
doesn't exist. More explanations inside.
This doesn't fix the issue for dirstate-v1, a later patch addresses it.
Raphaël Gomès <rgomes@octobus.net> [Mon, 12 Dec 2022 17:08:12 +0100] rev 49701
rust-dirstate: remember the data file uuid dirstate was loaded with
This will be used in the next patch to fix a race condition.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 01 Mar 2023 02:38:20 +0100] rev 49700
dirstate: set identity whenever we read the dirstate's v2 docket
The docket can be loaded outside of a full read (for exemple when
pre-fetching parents), so the current code would read/set the identity
after loading the data, opening a race condition:
A0: first process docket is read
B0: other process appends new data to the dirstate (and changes the docket)
A1: first process sets the identity (based on pre-B content, but with post-B identity)
A1: first process loads the dirstatemap from the data file
A1: first process does not detect the race and overwrites the update from B.
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 21 Feb 2023 15:10:12 +0100] rev 49699
dirstate: factor the identity setting code in the dirstate map
We need it in more locations, so let us start factoring thing out first
to make sure the same code is called everywhere.
This bears some similarity with 85746485a4dd on default, but at a smaller
scope and for a different purpose.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 01 Mar 2023 00:07:26 +0100] rev 49698
dirstate: simplify the dirstate's read race testing
Now that most code behaves properly, we can simplify the expected matching.
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 28 Feb 2023 19:36:46 +0100] rev 49697
dirstate: deal with read-race for pure rust code path (rhg)
If we cannot read the dirstate data, this is probably because a writing process
wrote it under our feet. So refresh the docket and try again a handful of time.
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 28 Feb 2023 23:35:52 +0100] rev 49696
dirstate: deal with read-race for python code using rust object
If we cannot read the dirstate data, this is probably because a writing process
wrote it under our feet. So refresh the docket and try again a handful of time.
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 28 Feb 2023 19:01:20 +0100] rev 49695
dirstate: deal with read-race for pure python code
If we cannot read the dirstate data, this is probably because a writing process
wrote it under our feet. So refresh the docket and try again a handful of time.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 01 Mar 2023 16:05:28 +0100] rev 49694
dirstate: abstract the reading of the data file in v2 in a method
We will need more changes to avoid some race conditions during read,
so we first isolate the simple logic before making it more complicated.
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 27 Feb 2023 03:14:30 +0100] rev 49693
dirstate: add append/new-file variants in the dirstate's read race tests
This covers more ground and finds more bugs.
At that point I gave up on making things as `known-bad-output` /
`missing-correct-output` as this gets too messy.
Raphaël Gomès <rgomes@octobus.net> [Tue, 13 Dec 2022 14:51:36 +0100] rev 49692
dirstate: add a synchronisation point in the middle of the read
This will be useful to test some more race conditions around dirstate.
Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 26 Feb 2023 16:27:50 +0100] rev 49691
dirstate: add v1-v2 variants to the dirstate's read race tests
More cases mean different issues.
Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 26 Feb 2023 08:17:23 +0100] rev 49690
dirstate: check dirstate race condition around status
More problems to solve.
Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 26 Feb 2023 07:08:16 +0100] rev 49689
dirstate: check dirstate race condition around update
More problems to solve.
Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 26 Feb 2023 07:02:13 +0100] rev 49688
dirstate: check dirstate race condition around commit
Once in a while, rhg is the only one to behave right here.
Pierre-Yves David <pierre-yves.david@octobus.net> [Sat, 25 Feb 2023 00:54:30 +0100] rev 49687
dirstate: initial creation of a test file to check dirstate race read
More problems to solve… yeah ! (I guess)
Pierre-Yves David <pierre-yves.david@octobus.net> [Sat, 25 Feb 2023 01:07:44 +0100] rev 49686
dirstate: add a synchronisation point before doing a full dirstate read
This will be useful to test some race conditions around the dirstate.
Raphaël Gomès <rgomes@octobus.net> [Tue, 28 Feb 2023 12:15:19 +0100] rev 49685
rust-repo: move dirstate-v2 opening to a separate method
The next changeset will make changes to this logic, it helps to have it
in order first.
Raphaël Gomès <rgomes@octobus.net> [Tue, 28 Feb 2023 16:19:21 +0100] rev 49684
rhg: fix race when an ambiguous file is deleted on disk
There are two places in the status code where we handle files whose status
we are unsure of based off of metadata alone: this one is the first one to
actually disambiguate, and the second one is later in the code (but updated
in the previous commit) for files that are actually clean to update the
dirstate. Since there is a chance that the contents have changed between
those two moments, we need to stat the files again, since re-using the old
stat could lie about the clean state of the file.
Raphaël Gomès <rgomes@octobus.net> [Mon, 27 Feb 2023 15:18:50 +0100] rev 49683
rhg: fix race when a fixup file is deleted on disk
See next changeset for the other race in the same kind of logic and why
there are two different places.
Pierre-Yves David <pierre-yves.david@octobus.net> [Sat, 25 Feb 2023 06:11:14 +0100] rev 49682
dirstate: test a `hg status` raced by a `hg remove`
This shows that `rhg` is misbehaving here.
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 24 Feb 2023 01:19:37 +0100] rev 49681
dirstate: tests racing status with both dirstate-v2 append and rewrite
The way the racing process touches the dirstate results in different challenges
for the raced process.
We now test each variant in the `test-dirstate-status-race.t` tests.
Raphaël Gomès <rgomes@octobus.net> [Tue, 28 Feb 2023 15:49:53 +0100] rev 49680
dirstate-v2: add devel config option to control write behavior
This will help us to write predictable tests checking behavior in each case.
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 24 Feb 2023 18:21:54 +0100] rev 49679
dirstate: use more than a bool to control append behavior
When writing dirstate-v2, we might either append to the existing file, or
create a new file.
We are about to introduce some configuration to control this behavior.
As a prelude, we change the current way the behavior was automatically
controlled to make the change smaller/clearer.
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 24 Feb 2023 01:15:45 +0100] rev 49678
dirstate: cover each dirstate version when testing for status race
Previously we were only testing it with the default (dirstate-v1 currently).
Now we explicitly test each variant.
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 24 Feb 2023 01:09:11 +0100] rev 49677
dirstate: test a `hg status` raced by another `hg status`
This shows that `rhg` is misbehaving here.
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 24 Feb 2023 01:01:04 +0100] rev 49676
dirstate: test a `hg status` raced by a `hg update`
This shows that `rhg` is misbehaving here.
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 24 Feb 2023 00:55:13 +0100] rev 49675
dirstate: test a `hg status` raced by a `hg commit`
This shows that `rhg` is misbehaving here.
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 24 Feb 2023 16:12:01 +0100] rev 49674
dirstate: test a `hg status` raced by a `hg add`
This shows that `rhg` is misbehaving here.
Raphaël Gomès <rgomes@octobus.net>, Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 28 Feb 2023 15:25:47 +0100] rev 49673
dirstate: add a way to test races happening during status
We add the `devel.sync.status.pre-dirstate-write-file` config option to easily
test what happens when other operations happen during the window where
`hg status` is done working but has not updated the cache on disk yet.
We introduce the framework for testing such races too, actual tests will be
added in the next changesets. For now the test is only checking dirstate-v1. We
will extend the test coverage later too.
Check test documentation for details.
Code change from Raphaël Gomès <rgomes@octobus.net>
Test change from Pierre-Yves David <pierre-yves.david@octobus.net>
Raphaël Gomès <rgomes@octobus.net> [Tue, 28 Feb 2023 00:01:41 +0100] rev 49672
testing: introduce util function to synchronize concurrent commands on files
This is an extension of mechanisms that the tests have been using for a while.
To be able to also control the execution in Rust, we introduce utility to
perform such `wait_on_file` logic based on some configuration value.
This will be used in the tests introduced in the next changesets.
Raphaël Gomès <rgomes@octobus.net> [Tue, 28 Feb 2023 00:04:32 +0100] rev 49671
dirstate: add some debug output when writing the dirstate
This helps debugging Mercurial behavior in the dirstate-v2 case.
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 31 Jan 2023 13:16:39 +0100] rev 49670
run-tests: make it possible to nest conditionals
This is not that hard to implement and makes our life easier on a regular basis.
Raphaël Gomès <rgomes@octobus.net> [Mon, 28 Nov 2022 12:33:20 +0100] rev 49669
dirstate-v2: don't mmap the data file when on NFS
`mmap` on NFS will trigger a SIGBUS when the mmap'ed file is deleted, which
wouldn't work in our case. Also, the performance advantage of using mmap on
NFS is debatable at best.
Raphaël Gomès <rgomes@octobus.net> [Thu, 08 Dec 2022 16:38:39 +0100] rev 49668
rust-dirstate: trace append/no append to help debugging
Raphaël Gomès <rgomes@octobus.net> [Mon, 12 Dec 2022 16:38:05 +0100] rev 49667
rust: add debug log about skipping dirstate update
Raphaël Gomès <rgomes@octobus.net> [Mon, 09 Jan 2023 15:17:48 +0100] rev 49666
test-dirstate: use more robust method to trigger a data-file append
The previous method was fragile and somewhat flaky on fast machines.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 22 Feb 2023 18:10:26 +0100] rev 49665
transaction: tests we don't overwrite bookmark activation on abort
We actually do not! Great.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 22 Feb 2023 18:09:12 +0100] rev 49664
transaction: tests we don't overwrite updates on abort
spoiler: we do… /o\
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 22 Feb 2023 18:07:34 +0100] rev 49663
transaction: tests we don't overwrite branch changes on abort
We actually do not! Great.
…Why are doing a backup of the `branch` files at transaction creation then‽
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 22 Feb 2023 18:05:36 +0100] rev 49662
transaction: tests we don't overwrite tracking to changed file on abort
spoiler: we do…
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 22 Feb 2023 18:03:18 +0100] rev 49661
transaction: the base of a new test file checking transaction abort issue
See inline documentation for details. See other changesets for actual cases.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 22 Feb 2023 18:30:47 +0100] rev 49660
setup: support building from an ongoing merge
Before this change the two parents from the merge would duplicate some command
output and modify some others in a way python 3.11 chokes on.
Raphaël Gomès <rgomes@octobus.net> [Thu, 16 Feb 2023 11:42:34 +0100] rev 49659
rust: upgrade minimum `rayon` dependency
The 1.6.0 and 1.6.1 releases fixes a soundness issue and a performance issue
that are both important to us, since both correctness and performance are
paramount in the Rust extensions.
Pierre-Yves David <pierre-yves.david@octobus.net> [Sat, 18 Feb 2023 01:21:51 +0100] rev 49658
test-chg: use a different log to avoid flakyness
The test was deleting the log file to start anew. However a trailing working
process might still be alive at this time, and recreate the very same log on
exit.
We see the trace of such worker in the expected content of server.log (see the
trace modified by this patch). This is flaky because we don't know *when* the
worker will write to the file and there is a race with the `hg init cached`
command.
A much simpler and reliable way to start anew without having such race is… to
write to a different log file. No reuse → no conflict, no conflict → no race, no
race → no flakiness.
Arseniy Alekseyev <aalekseyev@janestreet.com> [Fri, 18 Nov 2022 13:51:40 +0000] rev 49657
dirstate-v2: complain early on docket name collision
The alternative is that the dirstate gets deleted so the
corruption persists and is hard to investigate.
This happened to me in tests, where the dirstate names
are taken from file, since the file got reverted.
I expect this can also happen in prod with non-trivial
probability (1/4 billion).
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 17 Feb 2023 16:45:36 +0100] rev 49656
setup: further improve the error path for version retrieval
This is a new take at the problem that 8d390a13474d tried to tackle. There was
two issues after that previous improvement:
- the 0.0+ version could survive a bit too long and reaching the installer
version and staying there.
- multiple use case where still failing.
So the new code is better at:
- always succeeding when running `make local` so that we can
bootstrap a local version
- no using that fallback outside of `make local` to avoid distribution of
version with the buggy version number.
The setup.py is a gigantic pile of spaghetti code, to the point where
pastafarian pilgrim started knocking at its core.
However I refrained from cleaning that up since the more to a `setup.cfg` means
this code should be deleted soon™.
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 17 Feb 2023 14:00:39 +0100] rev 49655
dirstate: handle missing backup file on restoration
This is the stable counter part to e358f6e0e50e.
Since 6.4 will stop writing undo.dirstate in some case (actually… at all), a
transaction created with 6.4 and recover/rolledback with 6.3 need to work to a
certain degreee. This changeset add the necessary bits so that we don't get a
traceback from 6..3 in this cases.