Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 24 Nov 2021 04:51:05 +0100] rev 48399
dirstate-item: implement the comparison logic for mtime-second-ambiguous
If the flag is set we now process it properly.
We "just" need to actually set it and persist it.
Differential Revision: https://phab.mercurial-scm.org/D11843
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 24 Nov 2021 04:40:00 +0100] rev 48398
dirstate-item: add a "second_ambiguous` flag in the mtime tuple
This will be used to support the `mtime-second-ambiguous` flag from dirstate
v2. See format documentation for details.
For now, we only make it possible to store the information, no other logic have
been added.
Differential Revision: https://phab.mercurial-scm.org/D11842
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 23 Nov 2021 19:27:17 +0100] rev 48397
dirstate: drop comparison primitive on the timestamp class
All comparison are now managed without using operator :
- the status mtime comparisons is handled by the DirstateItem,
- the fixup reliability check,
- the update "hack".
So we no longer needs the operator and should discourage its usage.
Differential Revision: https://phab.mercurial-scm.org/D11841
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 23 Nov 2021 18:13:33 +0100] rev 48396
largefile: use the proper "mtime boundary" logic during fixup
This will prevent ambiguous cache entry to be used in racy situation. This fix
flakiness in test and some real live misbehavior.
Differential Revision: https://phab.mercurial-scm.org/D11800
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 23 Nov 2021 18:03:51 +0100] rev 48395
status: move the boundary comparison logic within the timestamp module
Some extensions will need it too. So lets isolate the logic. It also makes
things clearer.
Differential Revision: https://phab.mercurial-scm.org/D11799
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 23 Nov 2021 18:11:42 +0100] rev 48394
tests: remove potential mtime ambiguity in a dirstate test
If the test was fast enough, some mtime where not stored. We now wait long
enough to ensure the mtime is no longer ambiguous.
Differential Revision: https://phab.mercurial-scm.org/D11798
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 19 Nov 2021 03:03:01 +0100] rev 48393
dirstate: cleanup remaining of "now" during write
Since the whole `need_delay` have been removed, we no longer need this.
Differential Revision: https://phab.mercurial-scm.org/D11797
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 19 Nov 2021 03:04:42 +0100] rev 48392
dirstate: remove need_delay logic
Now that all¹ stored mtime are non ambiguous, we no longer need to apply the `need_delay` step.
The need delay logic was not great are mtime gathered during longer operation
could be ambiguous but younger than the `dirstate.write` call time.
So, we don't need that logic anymore and can drop it
This make the code much simpler. The code related to the test extension faking
the dirstate write is now obsolete and associated test will be migrated as
follow up. They currently do not break.
[1] except the ones from `hg update`, but `need_delay` no longer help for them
either.
Differential Revision: https://phab.mercurial-scm.org/D11796
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 25 Oct 2021 11:36:22 +0200] rev 48391
dirstate: remove `lastnormaltime` mechanism
This is now redundant with the new, simpler `mtime_boundary` one.
Differential Revision: https://phab.mercurial-scm.org/D11795
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 18 Nov 2021 13:12:40 +0100] rev 48390
status: use filesystem time boundary to invalidate racy mtime
We record the filesystem time at the start of the status walk and use that as a
boundary to detect files that might be modified during (or right after) the
status run without the mtime allowing that edition to be detected. We
currently do this at a second precision. In a later patch, we will use
nanosecond precision when available.
To cope with "broken" time on the file system where file could be in the
future, we also keep mtime for file over one day in the future. See inline
comment for details.
Large file tests get a bit more confused as we reduce the odds for race
condition.
As a "side effect", the win32text extension is happy again.
Differential Revision: https://phab.mercurial-scm.org/D11794
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 18 Nov 2021 15:00:13 +0100] rev 48389
test: use a different timestamp for the updated file
In the test we want to trigger a write after the underlying dirstate changed. To
do so, we need a write. And as we are about to make dirstate update smarter we
need to meddle with the script a bit to make sure there will be a write.
Differential Revision: https://phab.mercurial-scm.org/D11793
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 23 Nov 2021 03:22:30 +0100] rev 48388
win32text: drop associated dirstate cache information on revert
Otherwise the could get size from one version of the file while the on-disk
version is still clean but with another size.
This fix the previously introduced error.
Differential Revision: https://phab.mercurial-scm.org/D11792
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 17 Nov 2021 20:27:27 +0100] rev 48387
dirstate: stop gathering parentfiledata in update_file
Gathering information here assume that they are valid cache information for a
clean file. It is true most of the time, but not garanteed.
Accurate data can still be explicitly provided.
We drop the spontaneous and will let the next `hg status` call record actual information.
Differential Revision: https://phab.mercurial-scm.org/D11791
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 17 Nov 2021 20:26:33 +0100] rev 48386
dirstate: stop gathering parentfiledata in update_file_p1
Gathering information here assume that they are valid cache information for a
clean file. It is true most of the time, but not garanteed.
So we drop this and will let the next `hg status` call record actual information.
Differential Revision: https://phab.mercurial-scm.org/D11790
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 17 Nov 2021 20:26:14 +0100] rev 48385
dirstate: make it mandatory to provide parentfiledata in `set_clean`
Gathering the mode, size and mtime, independently from determining that the file
is clean is a race-machine. So we just make these information required arguments.
(note that the data is still gathered in a racy way in practice, but at least
the API is no longer encouraging it.)
Differential Revision: https://phab.mercurial-scm.org/D11789
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 22 Nov 2021 15:58:51 +0100] rev 48384
dirstate: do no use `set_clean` in revert
The current `set_clean` usage is racy (the file might be modified between its
restoration and the `set_clean` call).
So we simply leave the file as ambiguous and the next status will fix that.
We still have to make sure the copy information is dropped, so we teach dirstate
how to do that.
The win32txt extension is confused after this because current logic is broken in
more location. However this series will ultimately fix that so we "ignore" it
for now. Fixing it now is complicated without some extra fix landing later.
Differential Revision: https://phab.mercurial-scm.org/D11788
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 18 Nov 2021 22:49:05 +0100] rev 48383
status: adapt the "keyword" extensions to gather stats at lookup time
See main core code for details.
We don't factor the code in a common function yet, because we will have to adapt
a bit more things in the keyword case at the end of the series.
Differential Revision: https://phab.mercurial-scm.org/D11787
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 18 Nov 2021 22:46:50 +0100] rev 48382
status: adapt largefile to gather stats at lookup time
See the core code for details of why we are doing this.
We don't factor the code in a common function yet, because we will have to adapt
a bit more things in the largefile case at the end of the series.
Differential Revision: https://phab.mercurial-scm.org/D11786
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 17 Nov 2021 23:37:47 +0100] rev 48381
status: gather fixup info at comparison time
This is still racy, but on a much small windows. In addition, the API now make
it possible for it to not be racy. This also unlock other cleanups that we are
about to do regarding mtime ambiguity at gathering time.
Differential Revision: https://phab.mercurial-scm.org/D11785
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 17 Nov 2021 10:22:15 +0100] rev 48380
update: filter the ambiguous mtime in update directly
Right now, this filtering is done by `dirstate.write` using the time of
`dirstate.write` method call. However that filtering is done "too late"
It works "fine" as most command are "fast enough", and race rare enough.
We are about to change the mtime filtering logic in the dirstate to be more
accurate and reliable.
However `hg update` will still need such filtering (mostly because it is
actually quite racy, even with the existing filtering). So we explicitly
implement a similar logic here. Before removing the older one later in the
series.
Differential Revision: https://phab.mercurial-scm.org/D11784
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 17 Nov 2021 12:24:00 +0100] rev 48379
dirstate: move "get fs now" in the timestamp utility module
We will need it during update.
Differential Revision: https://phab.mercurial-scm.org/D11783
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 17 Nov 2021 10:26:48 +0100] rev 48378
dirstate-item: allow mtime to be None in "parentdata"
This will be useful to filter out unreliable mtime.
Differential Revision: https://phab.mercurial-scm.org/D11782
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 17 Nov 2021 02:58:44 +0100] rev 48377
dirstate: add a comment about a racy piece of code during updates
This is a bit that is not really correct but works "fine" in practice. Let us
write the details down so that people stop wondering how that logic might be
correct… It is not.
Differential Revision: https://phab.mercurial-scm.org/D11781
Raphaël Gomès <rgomes@octobus.net> [Mon, 25 Oct 2021 15:11:53 +0200] rev 48376
tests: add missing `head` for when things go wrong
See comment above the changed line, you can get a millions of line of output in
case of failure. When don't need to have them all.
Differential Revision: https://phab.mercurial-scm.org/D11780