dirstate: initial creation of a test file to check dirstate race read More problems to solve… yeah ! (I guess)

dirstate: add a synchronisation point before doing a full dirstate read This will be useful to test some race conditions around the dirstate.

rust-repo: move dirstate-v2 opening to a separate method The next changeset will make changes to this logic, it helps to have it in order first.

rhg: fix race when an ambiguous file is deleted on disk There are two places in the status code where we handle files whose status we are unsure of based off of metadata alone: this one is the first one to actually disambiguate, and the second one is later in the code (but updated in the previous commit) for files that are actually clean to update the dirstate. Since there is a chance that the contents have changed between those two moments, we need to stat the files again, since re-using the old stat could lie about the clean state of the file.

rhg: fix race when a fixup file is deleted on disk See next changeset for the other race in the same kind of logic and why there are two different places.

dirstate: test a `hg status` raced by a `hg remove` This shows that `rhg` is misbehaving here.

dirstate: tests racing status with both dirstate-v2 append and rewrite The way the racing process touches the dirstate results in different challenges for the raced process. We now test each variant in the `test-dirstate-status-race.t` tests.

dirstate-v2: add devel config option to control write behavior This will help us to write predictable tests checking behavior in each case.

dirstate: use more than a bool to control append behavior When writing dirstate-v2, we might either append to the existing file, or create a new file. We are about to introduce some configuration to control this behavior. As a prelude, we change the current way the behavior was automatically controlled to make the change smaller/clearer.

dirstate: cover each dirstate version when testing for status race Previously we were only testing it with the default (dirstate-v1 currently). Now we explicitly test each variant.

dirstate: test a `hg status` raced by another `hg status` This shows that `rhg` is misbehaving here.

dirstate: test a `hg status` raced by a `hg update` This shows that `rhg` is misbehaving here.

dirstate: test a `hg status` raced by a `hg commit` This shows that `rhg` is misbehaving here.

dirstate: test a `hg status` raced by a `hg add` This shows that `rhg` is misbehaving here.

dirstate: add a way to test races happening during status We add the `devel.sync.status.pre-dirstate-write-file` config option to easily test what happens when other operations happen during the window where `hg status` is done working but has not updated the cache on disk yet. We introduce the framework for testing such races too, actual tests will be added in the next changesets. For now the test is only checking dirstate-v1. We will extend the test coverage later too. Check test documentation for details. Code change from Raphaël Gomès <rgomes@octobus.net> Test change from Pierre-Yves David <pierre-yves.david@octobus.net>

testing: introduce util function to synchronize concurrent commands on files This is an extension of mechanisms that the tests have been using for a while. To be able to also control the execution in Rust, we introduce utility to perform such `wait_on_file` logic based on some configuration value. This will be used in the tests introduced in the next changesets.

hghave: drop py27 and py35 support These versions of python are no longer supported by Mercurial.

hghave: byteify a path passed to a core API It looks like this predicate isn't used at all(?)

dirstate: add some debug output when writing the dirstate This helps debugging Mercurial behavior in the dirstate-v2 case.

run-tests: make it possible to nest conditionals This is not that hard to implement and makes our life easier on a regular basis.

rust: box ConfigValueParseError to avoid large result types clippy emits a warning that all the Result types are way too large because of HgError includes ConfigValueParseError as one of the variants, so its size is 136 bytes. By boxing ConfigValueParseError we're hopefully making everything faster "for free".

dirstate: drop `identity` from the public API We no longer needs it.

delta-find: rename `delta-reuse-policy` to `pulled-delta-reuse-policy` This make it clearer which type of delta we are talking about.

config-item: declare undeclared path suboption This should prevent issue like the previous commit.

delta-find: declare the "paths..*:delta-reuse-policy option While looking into renaming the option I realized it was not declared.

delta-find: adjust the default candidate group chunk size We move from 10 to 20 as the default as some usage in the wild saw a small degradation in storage quality when using `10`.

record: extract a closure to the module level This clean up is almost as gratuituous as this closure was.

record: drop a now useless overlay that grab the lock Since 28dfb2df4ab9, commit grab the wlock and the extra layer grabing the lock in record is no longer needed. We clean up the code to make this simpler (and add a small assert for extra security against future change).

bundlerepo: fix string interpolation Matt Harbison is saying we cannot `%s` a type into a byte string and that seems reasonable.

bundlerepo: apply phase data stored in the bundle instead of assuming `draft` The phase information contained in the changegroup part and the explicit `phase-heads` part are now taken in account. Initial changes and test by Matt Harbison, code rework by Pierre-Yves David.

bundlerepo: handle changegroup induced phase movement in the associated method These movement comes from handling the changegroup part, so we keeps the code grouped. This will be important when handling more part (and more changegroup part in the future) This induce a small code duplication, but it does not looks terrible.

bundlerepo: move most attribute declaration earlier in __init__ The expected attribute are clearer this way. The bundle handling code is responsible for setting most of it.

bundlerepo: move the handling of bundl1 in its own method This should make the overall flow simpler to follow.

bundlerepo: expliclty handing cg part from bundle2 We will handle other types of parts soon (phase-heads) so we need some cleanup first.

transaction: use the standard transaction mechanism to backup branch Branch is a bit special : - It currently does not collaborate with the transaction (or any scoping) for writing (this is bad) - It can change without the lock being taken (it is protected by `wlock`) So we rely on the same mechanism as for the backup of the other dirstate file: - we only do a backup if we hold the wlock - we force a backup though the transaction Since "branch" write does not collaborate with the transaction, we cannot back it up "at the last minute" as we do for the dirstate. We have to back it up "upfront". Since we have a backup, the transaction is no longer doing its "quick_abort" and get noisy. Which is quite annoying. To work around this, and to avoid jumping in yet-another-rabbit-hole of "getting branch written properly", I am doing horrible things to the transaction in the meantime. We should be able to get this code go away during the next cycle. In the meantime, I prefer to take this small stop so that we stop abusing the "journal" and "undo" mechanism instead of the proper backup mechanism of the transaction. Also note that this change regress the warning message for the legacy fallback introduced in 2008 when issue902 got fixed in dd5a501cb97f (Mercurial 1.0). I feel like this is fine as issue 902 remains fixed, and this would only affect people deploying a mix of 15 year old Mercurial and modern mercurial, and using branch and rollback extensively.

transaction: no longer explicitly cache bookmarks The transaction file generation is already dealing with the backup for this. So, no need to duplicate such backup.

transaction: no longer explicitly cache phaseroots The transaction file generation is already dealing with the backup for this. So, no need to duplicate such backup.

narrow: enforce that narrow spec is written within a transaction

narrow: write the narrow spec in a transaction during share It will be simpler if all write happens within transaction.

narrow: open the transaction sooner when unbundling That way, the narrow spec changes will be done within a transaction.

narrow: write the narrow spec in a transaction during clone It will be simpler if all write happens within transaction.

narrow: drop the dedicated backup code Now that the transaction manage the writes, we can simply use the transaction for backup. Some extra cleanup to ensure all changes happens within a transaction will be made in the next changesets.

narrow: delegate the dirstate's narrow spec writing to the transaction This make it more transactional and will help us to simplify their backup. The implementation is not great, but it keep the patch simple as this is not the time for a larger refactoring yet.

narrow: delegate the narrow spec writing to the transaction This make it more transactional and will help us to simplify their backup. The implementation is not great, but it keep the patch simple as this is not the time for a larger refactoring yet.

narrow: get the narrow patterns from the repository object instead of disk Relying on disk data make the transactionally of this change complicated, so let us start reading data from other API instead.

narrow: widden the lock context in `tracking` The tracking configuration we modify must be read under lock. So we grab the lock sooner.

narrow: move `only_show` handling sooner in `tracked` This will help us to improve the locking scope in the `tracked` command.

tests: in test-fncache.t, be more tolerant to the "Killed" message format For example, on my box I'm seeing: $TESTTMP.sh: line 250: $pid Killed hg ci -qAm z

convert: use a priority queue for sorting commits, to make sorting faster To achieve this, we turn commit sorters into classes so they can encapsulate state. This reduces the sorting time from ~30s to ~10s on a 500k-commit prefix of a repo I tried to convert. (and probably reduces the time to sort the whole repo from many tens of minutes to minutes, but I didn't try that again) The date caching gets removed because priority queue already caches the key.

dirstate-v2: don't mmap the data file when on NFS `mmap` on NFS will trigger a SIGBUS when the mmap'ed file is deleted, which wouldn't work in our case. Also, the performance advantage of using mmap on NFS is debatable at best.

rust-dirstate: trace append/no append to help debugging

rust: add debug log about skipping dirstate update

test-dirstate: use more robust method to trigger a data-file append The previous method was fragile and somewhat flaky on fast machines.

typing: add the return type hint to pycompat.rangelist() Not bothering with the args, because there are a few overloads and only 2 callers in the codebase, one of which is a test.

typing: add type hints to pycompat.maplist() The typeshed hints define 5 overloads with an increasing number of parameters on the passed function, and then a catchall that ignores the argument list on the passed function and allows an `*iterators` arg. All of our uses are fulfilled by the 1 function + 1 iterable overload, but add the second overload as a hint in case it's needed in the future.

branching: merge stable into default This show that the recent changes on default fixed the issue with transaction overwriting content in `test-transaction-wc-rollback-race.t`

transaction: tests we don't overwrite bookmark activation on abort We actually do not! Great.

transaction: tests we don't overwrite updates on abort spoiler: we do… /o\

transaction: tests we don't overwrite branch changes on abort We actually do not! Great. …Why are doing a backup of the `branch` files at transaction creation then‽

transaction: tests we don't overwrite tracking to changed file on abort spoiler: we do…

transaction: the base of a new test file checking transaction abort issue See inline documentation for details. See other changesets for actual cases.

setup: support building from an ongoing merge Before this change the two parents from the merge would duplicate some command output and modify some others in a way python 3.11 chokes on.

rhg: in path_encode, simplify a bit more Use the slices for `basename` and `ext` instead of dealing with offsets.

rhg: in path_encode, be a bit more conservative about memory usage Use [shrink_to_fit] to match the previous behavior more closely, and potentially save (a tiny bit) of memory. FWIW, I suspect this is unnecessary, but this whole MR is about simplifying things while preserving any existing optimizations.

rhg: small refactor: stop using a magical constant "+ 1" Instead, directly do what the "+ 1" was supposed to achive: call hash_encode.

rhg: in path_encode, use Vec directly instead of VecDest No need to have a trivial wrapper over the type. There's nothing confusing about vec.write_bytes(...), after all.

rhg: in path_encode, split Dest into VecDest and MeasureDest Two separate types make the write semantics easier to understand because we can consider the two sinks separately. Having two independent compiled functions for size measurement and for actual encoding seems likely to improve performance, too. (and maybe we should get rid of measurement altogether) Getting rid of [Dest] also removes the ugly option rewrapping code, which is good.

rhg: use generic DestArr in hash_mangle This simplifies code a bit more, but comes with an extra memory copy in case [destlen == dest_vec.len()]. This is probably fine, but a follow-up change is removing that too.

rhg: in path_encode, make DestArr generic over its size

rhg: in path_encode add a DestArr type This is an implementation of Sink trait that writes into a fixed-size buffer on the stack, so identical to what was done before, but it makes the code of [hash_encode] easier to understand by dropping all these slice manipulations.

rhg: reduce verbosity in path_encode by using a trait for writing Hopefully this makes the code easier to read and understand and shorter overall. It also lets us later tweak the type we use as a [Sink], without having to change the encoding functions, including using two different types for size measurement and for the actual serialization.

refactor: simplify code in rust version of path_encode Moving the addition of '/' separator to the end of the loop makes the rest of the logic much simpler because the first iteration is no longer special.

dirstate: phase-divergent update to 4e95341c89aa Heptapod published the obsolete version of those.

dirstate: phase-divergent update to 65943224c184 Heptapod published the obsolete version of those.

dirstate: phase-divergent update to 3433723d1b9b Heptapod published the obsolete version of those.

dirstate: enforce change context for hacky_extension_update_file This was the last method not scoped yet

large-files: use a `changing_files` context when initializing the dirstate We are obviously mutating the dirstate, so lets scope this mutation.

dirstate: enforce context set_clean and set_possibly_dirty We don't want them called within a `changing_parents` context, but we still want them called within a context. So we update the decorator accordingly

keyword: wrap dirstate mutation in `changing_files` context This is the way.

keyword: wrap dirstate mutation in `changing_files` context This is the way.

dirstate: enforce `running_status` context for calling `status` Now that the context is working as intended and that the callers are updated. We can enforce it.

dirstate: have `running_status` warn when exiting with a dirty dirstate If running_status was started without the lock, all changes should have been explicitly written (with the lock) or invalidated before exiting the context.

dirstate: have `running_status` write the dirstate when holding the lock This is simple and harmless.

dirstate: check that dirstate is clean at the initial context opening More checking that we are not doing anything weird.

dirstate: start tracking that we are within a `running_status` context

dirstate: add documentation about the expectation of `running_status` context

contrib-perf: use `running_status` in `perf::status` This is the way.

large-files: also open the context in the subdirstate

large-files: use `running_status` in `mergeupdate`

large-files: use `running_status` in `scmutiladdremove` This is the way.

large-files: open the transaction sooner in `scmutiladdremove` We want it to encompass the status call.

large-files: use `running_status` in `overriderevert` This is the way

large-files: use `running_status` in `updatestandinsbymatch` This is the way.

large-files: wrap reposetup's status in a `running_status` context This is the way.

narrow: use `running_status` in `updateworkingcopy` This is the way.

status: use `running_status` in dirstate status This is the way.

status: pre-indent the dirstate status code This make the next changeset clearer.

dirstate: introduce a (noop) running_status context Let us start with a simplistic context so we can scope the appropriate code before adding more logic.

status: invalidate dirstate on LockError If we cannot take the lock, someone else is modifying the repository. Let us discard dirstate uncommitted data before exiting the status code. Having a clean dirstate after such operation seems safer. Strictly speaking, there is a small behavior change in the following situation: * process A call `status` outside of the `wlock` * process B grab the `wlock` * process A fails to acquires the lock to write status fixup * process B release the `wlock` *without touching the dirstate* * process A later grab the `wlock` * process A can write dirstate update from earlier `status` However this is a fairly hypothetical situation : * process A has to be raced * process B have to not update the dirstate * process A has to run another *unrelated* operation later. This seems rare enough to overlook. I am stating that the two operations in process A has to be unrelated. Otherwise, collecting status data outside of the lock to use them inside the lock is racy. Any other process could move things around (eg: the working copy) making the data collected during status irrelevantor even harmful. If such code exists, it should be fixed ASAP.

status: simplify the post status fixup phases With the wlock automatically discarding changes when applicable, we can simplify the code a bit. * we perform the fixup operation before trying to grab the lock to narrow the `try/except` * we no longer need to explicitly complare dirstate identities. We can trust the dirstate internal refresh for that. It would invalidate dirty data when needed. * detect still data invalidation by checking the dirty flag before and after taking the lock. Doing this is actually only necessary to issue the debug message, we could blindy trust the dirstate internal to ignore the `write` call on a non-dirty dirstate.

dirstate: cleanup the `_map` property cache The removed code was duplicating the effect of `@propertycache`. This is a gratuitous cleanup.

dirstate: only reload the dirstate when it may have changed This reinstall the equivalent of what the `filecache` was doing. However it does it at the dirstate level. There is a double motivation for this: - This avoid duplicating logic with the dirstate "identity" logic. - This increase the lifetime of the `dirstate` object, helping to implement change scoping.

dirstate: directly manage the dirstate property on localrepo Before we had: * the filecache layer on `localrepo` doing some caching * the dirstate having some internal invalidation/refresh machanism * the status code doing some identity validation. To clean this up, we are dropping the first item "localrepo `filecache`" from the equation in a favor of an approach integrated into the dirstate (second item) in the next changesets. This changeset will be a small windows where some things will be a bit slower. This will be fixed in the next changesets.

dirstate: factor the identity getting/setting code in the dirstate map We are doing the same things twice and we will add more logic in the next changesets. So lets start factoring things out now.

dirstate: use `cachestat` object for dirstatemap identity There is a class dedicated to this kind of cache check, let us use it. We will generalize this code in the next changesets, but we do the "behavior changing" pass on our own.

automv: lock the repository before searching for renames I detected this while debugging something else.

dirstate: distinct transaction callback from largefile This has not caused any issue so far because the large-files dirstate bypass the transaction logic. We might want to change that. In anyway, let us disarm this bug nest before it actually explode.

dirstate: track that changes are pending in a transaction Nothing is currently broken because if this, but this make the `_invalidated_context` attribute more accurate. Being more accurate here will help us later, when dealing with `status` call.

dirstate: add small asserts for double security We don't need this, but it does not hurt.

dirstate: simplify the invalidation management on context exit Since the `invalidate` call will directly reset the `_invalidated_context` attribut, we can simplify the code. In the same go, we move most of the logic out of the `finally` clause. It seems cleaner and safer. If we are handling an exception, we don't need the `write` code anyway.

dirstate: use the new `check_invalidated` decorator for `_changing` WeeeEEeee, less code.

dirstate: introduce a check_invalidated decorator This is a common need, so let us consolidate it to simplify the code.

dirstate: warn if dirty when starting an edition The dirstate should be clean before we start changing it. Otherwise we might write unrelated changes. Having a dirty dirstate laying around is also suspicious. This is similar to what we do when opening a new transaction, but this time this affect dirstate changes outside of a transaction.

large-files: make sure we write newly initialized standin file early Any changing context will have to initialized it before anything else. Not flushing the default (pre-change) content mean we would enter the changing context with a dirty dirstate, which is odd.

dirstate: mark `clear` and `rebuild` as `require_changing_parents` Yeah, more scoping!

dirstate: add a comment about the semantic of `dirstate.clear` This method is weird, lets flag it as such.

debugrebuildstate: wrap the operation in a `changing_parents` context This ismaybe a "changing_files" case? However this would be the only usage of `rebuild` outside a `changing_parents` context and this is a debug command, so lets not make the code base more complex because of that one command.

strip: use a `changing_parents` context for --keep update These are now properly scoped. note: it would be neat to reuse this in `hg rollback`.

mq: wrap the dirstate's rebuild in a `changing_parents` context This code is dealing with `qreshesh` failure. In that case the working copy will be left on the parent of the refreshed patch, so the parents are changing and `changing_parents` make sens.

lfconvert: use a `changing_parents` context to clear the dirstate Not sure if this is the right context, but it works and it is consistent with the other usages of `dirstate.clear`.

dirstate: mark the `copy` method as requiring a `changing_any` context This is used both when changing parents (e.g. merging with rename) and changing files (e.g. running `hg rename`).

dirstate: add a `require_changing_any` decorator We will need it for a couple of usecase (e.g `dirstate.copy`).

rebase: scope parent change into a changing_parents context If we are actually altering the working copy (i.e. we are not in memory), we should properly scope the working copy update.

dirstate: requires being in a `changing_parents` `context to set_parents` Enforcing proper operation scoping on all methods that mutate the dirstate will tighten correctness and reduce the risk of bugs. The context to use for this method is obvious, and all code was already compliant ☺

dirstate: invalidate on all exceptions Previously, we would miss SystemExit, KeyboardInterrupt etc. This "fix" on the bug tested in "test-largefiles-update.t" by preventing the precisely tested situation to happens at all. However this reveal a similar bug with a different timing. I have not been able to deal with that pre-existing bug so far. So I updated the test to point that out.

large-files: prepare a test for more changes The behavior around this test is about to change. We update the test to make it more robust and the changes clearer.

large-files: larger "changing_parents" context in mergeupdate override This since we are updating the lfdirstate early, it seems reasonable to include the full function in that scope.

large-files: use `hacky_extension_update_file` one more time This override is about merging and can be used in a `changing_parents` context. So lets use the method dedicated to hacky stuff when doing hacky stuff.

typing: disable `signature-mismatch` warnings on a few bytestr functions Recent versions of pytype complain about this, but it seems like expected behavior.

rust: upgrade minimum `rayon` dependency The 1.6.0 and 1.6.1 releases fixes a soundness issue and a performance issue that are both important to us, since both correctness and performance are paramount in the Rust extensions.

branching: merge with stable

test-chg: use a different log to avoid flakyness The test was deleting the log file to start anew. However a trailing working process might still be alive at this time, and recreate the very same log on exit. We see the trace of such worker in the expected content of server.log (see the trace modified by this patch). This is flaky because we don't know *when* the worker will write to the file and there is a race with the `hg init cached` command. A much simpler and reliable way to start anew without having such race is… to write to a different log file. No reuse → no conflict, no conflict → no race, no race → no flakiness.

dirstate-v2: complain early on docket name collision The alternative is that the dirstate gets deleted so the corruption persists and is hard to investigate. This happened to me in tests, where the dirstate names are taken from file, since the file got reverted. I expect this can also happen in prod with non-trivial probability (1/4 billion).

rhg: fix a bug in path_encode This makes rhg able to access long paths at the root of the tree just fine.

rhg: demonstrate a bug in path_encode The bug means that long filenames at the root of the tree are encoded incorrectly by rhg, so rhg crashes when trying to access them.

rhg: nicer error message

typing: add type hints to argument checking functions in cmdutil These might be surprising, since they can take strings instead of bytes. The way `AnyStr` works is that it must be all bytes or all str for any given invocation. The wildcard here will be the `opts` that get passed in- if the type is unknown and defaults to `Any`, there's no enforcement that the dict key type matches the additional args. But a lot of uses should be using `**opts` from the command method, which has a str key. The uses of these methods in this module are now typed because their internals force a specific type, and it can't just be inferred from the caller.

setup: further improve the error path for version retrieval This is a new take at the problem that 8d390a13474d tried to tackle. There was two issues after that previous improvement: - the 0.0+ version could survive a bit too long and reaching the installer version and staying there. - multiple use case where still failing. So the new code is better at: - always succeeding when running `make local` so that we can bootstrap a local version - no using that fallback outside of `make local` to avoid distribution of version with the buggy version number. The setup.py is a gigantic pile of spaghetti code, to the point where pastafarian pilgrim started knocking at its core. However I refrained from cleaning that up since the more to a `setup.cfg` means this code should be deleted soon™.

tag: move the prohibition of tagging the `null` rev up to the `wdir()` check It makes sense to do these together, and avoid another revision lookup.

branching: merge with default the first part of the fix is there

dirstate: handle missing backup file on restoration This is the stable counter part to e358f6e0e50e. Since 6.4 will stop writing undo.dirstate in some case (actually… at all), a transaction created with 6.4 and recover/rolledback with 6.3 need to work to a certain degreee. This changeset add the necessary bits so that we don't get a traceback from 6..3 in this cases.

tests: remove unnecessary --traceback argument

tag: disallow tagging the working directory It's kinda silly, but a clear error message is better than a stacktrace about subscripting `None` when trying to generate the default commit message. I'm surprised that `.revsingle(..).node()` returns None instead of `nodemod.wdirid`, but now there's a test to catch if this changes.

dirstate: remove the dedicated backup logic When alone, the dirstate can now take care of its commit/rollback pattern itself. When in a transaction, the transaction deal with commit/rollback pattern quite fine. Why did you have a dedicated backup logic?

localrepo: stop doing special dirstate backup at transaction open Since the dirstate writes are already managed by the transaction, we already do a backup of the dirstate when necessary (and even trigger one to keep `hg rollback` happy). We needs some special code to deal with the initial empty checkout, but it is not too complicated. Managing variable filename (as dirstate-v2 uses) at the "journalfile" level, is complex and fragile (which is consistent with the fact these files are not journal…). If we no longer do it, our life is significantly simpler. In some sense, we apply the xkcd-1134¹ solution to our savebackup/restorebackup problem. [1] https://xkcd.com/1134/ (the change to test-hardlink are expect as decreasing the number of duplicated backup drive the hardlink count down)

localrepo: "blindly" do a dirstate backup at the end of the transaction Having the file backup mechanism dealing with file backup as benefit. So lets move closer to that. The fact `hg rollback` even needs this is sad. I hope to have the time to implement one of the alternative soon.

test-hardlink: stop explicitly listing `undo.dirstate` The files is on its way out.

localrepo: enforce a clean dirstate when the transaction open This is important to simplify the backup process. This also seems like a quite reasonable expectation. See inline documentation for details.

dirstate: explicitly backup the datafile Since the transaction does not know about this file, we need to explicitly mark it for backup before touching it.

mq: write the dirstate before stripping I am not certain why it is dirty (probably the `repo.setparent` call a right before. However I know we need it flushed before the repository starts to be stripped.

dirstate: simplify the shelve hack to not go through the disk We already have the data in memory, so why not simply keep the data in memory? This avoid abusing the `savebackup/restorebackup` logic and will make our life easier.

test: fix the flakyness in test-remotefilelog-local.t I now get about 80% of my `test-chg` CI run that fails on flakyness in this tests. It turns out this is only ambiguous status that end up doing file download. So… calling status early will do that potential download separately and the calls we scrutinize during that test will be just fine.

dirstate: detect potential fishy transaction patterns while changing If we rely on the transaction more, we should be more careful about the transaction. Adding extra security seems reasonable.

dirstate: generalize the dirstate's invalidation on transaction abort The previous code was too specific, we should add the invalidation at next to the `addfilegenerator` in the `write` call.

dirstate: simplify some methods' decorator Since `changing_parents` and `changing_files` are mutually exclusive, having both `@requires_changing_files` and `@requires_not_changing_parents` on a method is redundant.

dirstate: document the functions that need consolidation They are more functions that make the dirstate dirty but does not currently enforce a clean change scoping, we add comments in front of each of them so clarify this. There is no urgency to it, but the world will be a better place when all of them have proper scoping in place.

dirstate: make `restorebackup` more robust when it is a noop If there are no data_file, we might not be able to determine its filename. Which can be safely ignored. (all this code is on the way out anyway)

dirstate-guard: remove the feature The dirstate guard duplicated some of the logic already implemented in the transaction (and now the changing_* context). However the feature was incomplete, for example, living only in memory meant we could not recover from the hardest crash. In addition this duplicated with the transaction logic meant things could go out of sync or step on each other. Removing the feature now that we no longer needs it seems the safest.

rollback: remove the dirstateguard usage Thanks to the previous changeset, we no longer needs it. begone !

rollback: explicitly skip dirstate rollback when applicable instead of letting the transaction logic overwrite the dirstate and then overwrite that overwrite with a backup. We simply do not restore the dirstate related file during the _playback call. This open the way to removing the last dirstate guard usage.

rollback: detect "parentgone" case earlier Detecting this earlier will help us to affect the rollback process sooner, which will help the next changesets. To keep things simple, we degrade the behavior a bit when the `undo.desc` is missing. However since `hg rollback` is not supposed to be used and the `undo.desc` file have been introduced in mercurial 1.6 (2010). I think this is an acceptable evil.

rollback: avoid a `hg commit --addremove` at a critical point The rollback behavior around `hg commit --addremove` has changed slightly. It does not really matters here but keeping that variant out of the way cannot hurt.

rollback: display some graphlog before/after a test piece This make the situation clearer.

rollback: show that the safety works in a associated test That test section is checking that --force is overcoming the safety check, however we did not check that the safety properly detect the situation. We now do for clarity.

dirstate-guard: remove its usage in `backout` We can simply replace it with a transaction.

dirstate-guard: remove the usage in `import` It is now redundant with the transaction spawning the same scope.

dirstate-guard: replace a usage in `rebase` with a transaction Opening the transaction sooner will provide us with the same benefit.

dirstate-guard: remove usage in `rebase` Now that the dirstate change and write are clearer, it does not seems we need to use a dirstate-guard here anymore. The transaction already wrap the full operation.

dirstate-guard: remove it usage in `mq` The code it covered is also covered by a `changing_parents` context.

dirstate: enforce the use of `changing_files` context to change tracking Now that everything is using the new context, we can start enforcing its usage.

dirstate: warn if we write to the dirstate without holding the wlock Writing the dirstate without holding the wlock can race with other update and overwrite important change. The comment questionning the current state of things was right, we should the `wlock` should always cover the dirstate. (Yes, this is me, agreeing with myself, half a decade later).

dirstate: avoid transaction backup/restore if we do not hold the lock Doing overwriting the dirstate content without holding the lock is a recipe for disaster.

dirstate: issue a developer warning on implicit write on wlock release Our goal is to get rid of all these to clarify the writing pattern, so it is time to warn about this (and later, forbid it).

status: fix post status invalidation If the dirstate changed under us, we should throw away what we have a reload it, should we not ?

status: fix post status writing With dirstate-v2, the status process itself might update internal states. So we make sure this get written on disk

dirstate: use `dirstate.change_files` to scope the change in `shelve` This is the way.

dirstate: use `dirstate.change_files` to scope the change in `unshelve` This is the way.

shelve: adjust what happens in some `changing_parents` context It seems like the rest is more about changing tracked_files, so we let start with moving them out of the `changing_parents` contexts.

dirstate: use `dirstate.change_files` to scope the change in `lfconvert` This is the way.

largefiles: rely on main scoping for writing dirstate in `markcommitted` Yeah, cleaner code.

largefiles: rely on main scoping for writing dirstate in `mergeupdate` Yeah, cleaner code.

largefiles: rely on the higher level `changing_giles` in `mergerecordupdates` Now that context open on the main dirstate also affect the underlying one, we can skip opening our own in `mergerecordupdates`

dirstate: use wlock and `dirstate.change_files` to scope the change in `mq` This is the way.

subrepo: use `changing_files` context in subrepository code This is better, not ideal, but better.

subrepo: let black expand some call on multiple lines early newer version of black are righfully doing this, so I apply it before more semantic change to reduce noise in the next changeset.

dirstate: use `dirstate.change_files` to scope the change in `import` This is the way.

dirstate: use `dirstate.change_files` to scope the change in `automv` This is... mostly... the way.

dirstate: use `dirstate.change_files` to scope the change in `amend` This is the way.

dirstate: use the `changing_files` context in the `keyword` demo This is the way.

dirstate: wrap repository change in appropriate context in `test-context` We need the `wlock` (to add files), the `lock` (to commit), a `transaction` (to commit) and a `changing_files` context (to add files). Strictly speaking, we could let the commit take the `lock` and create a `transaction`, but it seems more consistent that way.

dirstate: use wlock and changing_files context in `test-revlog-ancestry` This is the way.

dirstate: use `dirstate.change_files` to scope the change in `revert` This is the way.