Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 11 Mar 2024 12:03:40 +0100] rev 51498
tests: fix test-patchbomb-tls.t instability
The flakiness on chg is caused by a client that exit faster than the server
output log.
So actively wait for the server to issue the expected output (with a small
timeout)
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 11 Mar 2024 16:05:28 +0100] rev 51497
test-lock: use synchronisation file instead of sleep
This will prevent the test to be flaky on load.
Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 10 Mar 2024 03:29:12 +0100] rev 51496
branchcache: use update_disk to refresh 'served' and 'served.hidden'
The `update_disk` method is dedicated to this kind of usecase. Now that the writting patterns are more consistent, we can use it to warm these two important cache.
I am dropping the first comment about "refreshing all the others" because it is
false. If a branchmap already exist for "served", none of the subset will be
updated.
Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 10 Mar 2024 03:25:04 +0100] rev 51495
branchcache: explictly update disk state only if no transaction exist
If a transaction exist the `write_dirty` call will eventually be done and the state will be synched on disk. It is better to no interfer with that.
Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 10 Mar 2024 03:32:50 +0100] rev 51494
branchcache: do not use `__getitem__` in updatecache
The `update_disk` method uses `updatecache` and the point of `update_disk` is to be able to do alternative processing to the one we do in `__getitem__`. So we calling `__getitem__` in `updatecache` defeat this purpose.
Instead we do the equivalent explicitly to preserve the spirit of `update_disk` (that we will actually put to use soon, I promise)
Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 10 Mar 2024 05:10:00 +0100] rev 51493
branchcache: explicitly track inheritence "state"
We move from a binary "dirty" flag to a three value "state": "clean", "inherited", "dirty".
The "inherited" means that the branch cache is not only "clean", but it is a
duplicate of its parent filter.
If a branch cache is "inherited", we can non only skip writing its value on
disk, but it is a good idea to delete any stale value on disk, as those will
just waste time (and possibly induce bug) in the future.
We only do this in the update related to transaction or explicit cache update
(e.g `hg debugupdatecache`). Deleting the file when we simply detected a stall
cache during a read only operation seems more dangerous.
We rename `copy` to `inherit_for` to clarify we associate a stronger semantic
to the operation.
Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 10 Mar 2024 04:53:17 +0100] rev 51492
branchcache: stop writing more branchcache file on disk than needed
Before this change, we were unconditionally writing a branchmap file for the
filter level passed to `update_disk`. This is actually counter productive if no
update were needed for this filter level. In many case, the branch cache for a
filter level is identical to its parent "subset" and it is better to simply
keep the subset update and reuse it every time instead of having to do identical
work for similar subset.
So we change the `update_disk` method to only write a file when that filter
level differ from its parent. This removes many cases where identical files were
written, requiring multiple boring update in the test suite.
The only notable changes is the change to `test-strip-branch-cache.t`, this
case was checking a scenario that no longer reproduce the bug as writing less
branchmap file result in less stalled cache on disk.
Strictly speaking, we could create a more convoluted scenario that create a
similar issue. However the next changeset would also cover that scenario so we
directly updated that test case to a "no longer buggy" state.
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 08 Mar 2024 16:49:06 +0100] rev 51491
branchcache: do not copy the `_dirty` flag
If the inherited branch cache is dirty, it will be written on disk, and the
super-set did not need to modify it, the on disk value for the subset will be
re-useable as is. So the super set does not needs to write the very same content
itself.
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 08 Mar 2024 16:52:08 +0100] rev 51490
branchcache: explicitly assert that copy is always about inheritance
This would catch cases where copy is used for something else if any existed.
Pierre-Yves David <pierre-yves.david@octobus.net> [Sat, 09 Mar 2024 02:07:15 +0100] rev 51489
branchcache: stop using `copy(…)` in `replace(…)`
The `copy` method is mostly used for a filter level to inherit the branchmap
from a subset. So we stop using (abusing) it in "replace" to ensure `copy` is
used only for inheritance purposes.
Since `replace` is a method of the BranchMapCache, it seems fine to do lower
level operation there.
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 08 Mar 2024 16:47:32 +0100] rev 51488
branchcache: change the _delayed flag to an explicit `_dirty` flag
This is more consistent with the logic we use for other object and it open the way to a clearer management of the cache state.
Now, cache are created clean, cache update mark them dirty, writing them on
disk mark them clean again.
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 08 Mar 2024 15:50:15 +0100] rev 51487
branchcache: write branchmap in subset inheritance order
This way, we can guarantee a valid subset has been written before touching the
branchmap of another filter.
This is especially useful as we are bout to start deleting outdated branchmap
file.
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 08 Mar 2024 15:06:54 +0100] rev 51486
branchcache: do not accept "empty update"
This currently does not happens and it will be simpler that is remains that way.
If all update do something, we will be able to simply declare, in a later
changesets, that all update to result in a dirty branchcache.
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 07 Mar 2024 11:04:34 +0100] rev 51485
branchcache: avoid created a `None` filter repoview when writing
The repoview class is not intended to be used for unfiltered repository.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 28 Feb 2024 22:49:55 +0100] rev 51484
stream-clone-tests: stop filtering non existent warning
This filtering was introduced in
74c004a515bc, however there is already no
warning in that changeset. So I guess the warnings existed when we the patch
was created but the problem was solved in another changeset that
74c004a515bc,
rebased on.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 28 Feb 2024 22:46:12 +0100] rev 51483
stream-clone-test: simplify case testing obsolescence
There is only two important things in this test:
- the number of file we send, to show we picked the obsstore.
- the resulting state, to show we did alter things in the process.
The rest are of the number are very fragile and consume a lot of time for little
value when adjusting formats, caches, and protocol.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 28 Feb 2024 22:43:07 +0100] rev 51482
stream-clone-test: simplify the case testing phases
There is only two important things in this test:
- the number of file we send, to show we picked the phase roots.
- the resulting phases, to show we did not modified them.
The rest are of the number are very fragile and consume a lot of time for little
value when adjusting formats, caches, and protocol.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 28 Feb 2024 22:39:10 +0100] rev 51481
stream-clone-test: simplify bookmark clone
The important things to test here is the number of file included (to catch that
the bookmark file was sent). So we keep that part non glob'ed but glob the
rest.
The glob'ed numbers are very fragile and consume a lot of time for little value
when adjusting formats, caches, and protocol.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 28 Feb 2024 22:31:42 +0100] rev 51480
stream-clone-test: add a verify call to the "clone while changing" case
It seems useful to very that the clone did not result in a corrupted copy.
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 08 Mar 2024 10:59:51 +0100] rev 51479
stream-clone-test: add title to various test cases
These case are fine as is, but as we are adding title to all the other as we
simplify them, lets add title for all cases.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 28 Feb 2024 22:28:07 +0100] rev 51478
stream-clone-test: simplify testing of secret cloning restriction
Here, we just want to check if the streaming clone is allowed and used or not.
We do not care about the details of the clone itself.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 28 Feb 2024 22:26:27 +0100] rev 51477
stream-clone-test: simplify the background file closing test
Here we just care about the fact the background file closing logic actually ran. We don't need to check the details of the cloning.
The details of the output is very fragile and consume a lot of time for little
value when adjusting formats, caches, and protocol. So we filter it out.
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 08 Mar 2024 10:51:01 +0100] rev 51476
stream-clone-test: simplify the --uncompressed alias check
To check that --uncompressed is an alias we just need to check it trigger a
stream clone, we don't need to check anything else.
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 08 Mar 2024 10:50:42 +0100] rev 51475
stream-clone-test: drop an automatic pattern replacement
That pattern is nice, but it prevent us to glob the number of bytes when we
don't care about them. We don't care about them more often that what we
currently checks so dropping this pattern will help use to simplify various
tests.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 28 Feb 2024 22:15:33 +0100] rev 51474
stream-clone-test: simplify the test for getbundle with stream=1
The core of this tests is about checking we receive a stream bundle with such
request. We don't need to look at too much of the details of the stream itself.
Since the content of the stream if shifting overtime, Such check is very
fragile and consume a lot of time for little value when adjusting formats,
caches, and protocol.
So we reduce the size of what we check to focus on "is this a stream clone"
question.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 28 Feb 2024 22:05:28 +0100] rev 51473
stream-clone-test: factor some piece of basic clone test out
Multiple parts of this case (listing cache, checking error) are common to all
cases and don't need to be in the conditionnal block.
This simplify the test update.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 28 Feb 2024 22:01:09 +0100] rev 51472
stream-clone-test: simplify the case where server disabled it
We have an option to disable it, we don't need to test it with all protocol
variants.
In addition there is little value in looking at the bytes to bytes details of
the reply. Such check is very fragile and consume a lot of time for little
value when adjusting formats, caches, and protocol.
Georges Racinet <georges.racinet@octobus.net> [Mon, 11 Mar 2024 13:36:25 +0100] rev 51471
rust-matchers: raw regular expression builder
Extracting this `re_builder()` from `re_matcher()` makes it reusable
in more general cases than matching `HgPath` instances and would
help reducing code duplication in RHGitaly.
Georges Racinet <georges.racinet@octobus.net> [Mon, 11 Mar 2024 13:23:18 +0100] rev 51470
rust-filepatterns: export glob_to_re function
Making this function public should not risk freezing the internal API,
and it can be useful for all downstream code that needs to perform
glob matching against byte strings, such as RHGitaly where it will
be useful to match on branches and tags.
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 11 Mar 2024 01:20:12 +0100] rev 51469
repoview: prevent `None` to be passed as the filtername
We let such instantiation slip in a previous commit, so we add an explicit check
to prevent it to happen in the future.