dirstate-map: factor out the change to _dirs and _alldirs on removing This logic is complicated enough to deserves its own function. So it now does. This will make it easier to reuse that logic in later changeset. Differential Revision: https://phab.mercurial-scm.org/D11131

dirstate-map: factor out the change to _dirs and _alldirs on dropping This logic is complicated enough to deserves its own function. So it now does. This will make it easier to reuse that logic in later changeset. Differential Revision: https://phab.mercurial-scm.org/D11130

dirstate-map: factor out the change to _dirs and _alldirs on adding This logic is complicated enough to deserves its own function. So it now does. This will make it easier to reuse that logic in later changeset. Differential Revision: https://phab.mercurial-scm.org/D11129

dirstate-item: introduce a `dm_otherparent` property See inline documentation for details. Differential Revision: https://phab.mercurial-scm.org/D11124

dirstate-item: introduce a `dm_nonnormal` property See inline documentation for details. Differential Revision: https://phab.mercurial-scm.org/D11123

dirstate-v2: Reserve a few bytes of space for future extensions See doc-comment Differential Revision: https://phab.mercurial-scm.org/D11101

dirstate-v2: Separate iterators for dirfoldmap and debugdirstate `dirstatemap.dirfoldmap` was recently changed to re-use a Rust iterator that was added for the `hg debugdirstate` command. That iterator was based on all nodes in the tree dirstate without an entry only existing to hold child nodes, and therefore being directories. However to optimize status further we may want to store additional nodes for unknown or ignored files and directories. At that point the two users of this iterator will want different things, so let’s make two iterators instead. See doc-comments in `dispatch.rs`. Differential Revision: https://phab.mercurial-scm.org/D11099

dirstate-v2: Move fixed-size tree metadata into the docket file Before this changeset, the dirstate-v2 data file contained not only nodes and paths that may be reused when appending to an existing file, but also some fixed-size metadata that applies to the entire tree and was added at the end of the data file for every append. This moves that metadata into the docket file, so that repeated "append" operations without meaningful changes don’t actually need to grow any file. Differential Revision: https://phab.mercurial-scm.org/D11098

dirstate-v2: Add heuristic for when to create a new data file … instead of appending to the existing one. This is based on keeping track of how much of the existing data is not used anymore. Differential Revision: https://phab.mercurial-scm.org/D11097

dirstate-v2: Reuse existing paths when appending to a data file When writing a dirstate in v2 format by appending to an existing data file, filenames / paths that are borrowed from the previous on-disk representation can be reused. Differential Revision: https://phab.mercurial-scm.org/D11096

dirstate-v2: Reuse existing nodes when appending to a data file When writing a dirstate in v2 format by appending to an existing data file, nodes that are still "unparsed" from the previous on-disk representation have been unchanged and can therefore be reused. This makes the new data point to previously-existing data for tree nodes. Differential Revision: https://phab.mercurial-scm.org/D11095

dirstate-v2: Support appending to the same data file For now we’re still writing the entire data every time, so appending is not useful yet. Later we’ll have new nodes pointing to some existing data for nodes and paths that haven’t changed. The decision whether to append is pseudo-random in order to make tests exercise both code paths. This will be replaced by a heuristic based on the amount of unused existing data. Differential Revision: https://phab.mercurial-scm.org/D11094

dirstate-v2: shrink on-disk path lengths to 16-bits Differential Revision: https://phab.mercurial-scm.org/D11091

dirstate-v2: Rename Header to Root, move it to the end of the data file Now that they don’t have to be at the start, a given data file may contain multiple "roots". A docket only points to one of them, and previous ones are left unused to allow allow append-only in-place writing to an existing data file. Differential Revision: https://phab.mercurial-scm.org/D11090

dirstate-v2: Enforce data size read from the docket file The data file may not be shorter than its size given by the docket. It may be longer, but additional data is ignored. Differential Revision: https://phab.mercurial-scm.org/D11089

dirstate-v2: Introduce a docket file .hg/dirstate now only contains some metadata to point to a separate data file named .hg/dirstate.{}.d with a random hexadecimal identifier. For now every update creates a new data file and removes the old one, but later we’ll (usually) append to an existing file. Separating into two files allows doing the "write to a temporary file then atomically rename into destination" dance with only a small docket file, without always rewriting a lot of data. Differential Revision: https://phab.mercurial-scm.org/D11088

dirstate: replace a dead conditional branch with an assert in `update_file` This is a case we never meet, so lets trim it away to simplify the code before more changes. Differential Revision: https://phab.mercurial-scm.org/D11128

dirstate-map: do not use `size` to gate copy dropping during remove_file This get us close to moving the block right above withing the DirstateItem object. Doing so will help us getting rid of magic constant at the dirstatemap level. Differential Revision: https://phab.mercurial-scm.org/D11127

dirstate: drop a duplicated assert This very case is checked in 3 line above that one. Differential Revision: https://phab.mercurial-scm.org/D11126

dirstate: drop duplicated check This is covered by the `@requires_parents_change` decorator that this function use. Differential Revision: https://phab.mercurial-scm.org/D11125

dirstate: add a `set_possibly_dirty` in `fakedirstatewritetime` Differential Revision: https://phab.mercurial-scm.org/D11122

dirstatemap: use `set_possibly_dirty` in `clearambiguoustimes` lets put this new method to use Differential Revision: https://phab.mercurial-scm.org/D11121

dirstate-item: use `set_possibly_dirty` in `pure.pack_dirstate` Lets make use of the new function. Differential Revision: https://phab.mercurial-scm.org/D11120

dirstate-item: add a `set_possibly_dirty` method See inline documentation for details. The pushes the AMBIGUOUS_TIME implementation further down the line within the DirstateItem only. When this cleanup is done we will be able to stop using this representation internally. Differential Revision: https://phab.mercurial-scm.org/D11119

dirstate-item: add a `from_v1_data` constructor This class method is dedicated to building a DirstateItem from the data available in the "dirstate-v1" format. Since that format is frozen, this constructor will never change (unlike the `__init__` one). Differential Revision: https://phab.mercurial-scm.org/D11118

dirstate-item: use an explicit __init__ function instead of the attrs one For now, this is not doing anything special, however we plan to make it different in the future. So we start simple. Differential Revision: https://phab.mercurial-scm.org/D11117

dirstate: deprecated `drop` outside of `update/merge` All core users have been migrated. Differential Revision: https://phab.mercurial-scm.org/D11116

dirstate: deprecated `remove` outside of `update/merge` All core users have been migrated. Differential Revision: https://phab.mercurial-scm.org/D11115

dirstate: deprecate the `add` method outside of update/merge context All core users have been updated. Differential Revision: https://phab.mercurial-scm.org/D11114

upgrade: avoid a traceback in case of unrecognized revlog Without this, in case of revlog name not matching any know pattern we would get a traceback. Raising a clear error seems simpler. Differential Revision: https://phab.mercurial-scm.org/D11202

walk: no longer ignore revlogs of files starting with `undo.` (issue6542) Changeset 0b569c75d180 introduced new code in store.walk to filter out undo files left behind by the transaction. However doing so is also filtering out legitimate revlog file starting with `undo.` So this changeset is mostly rolling back that change and adding tests tests to catch this kind of error in the future. As a result we the transaction undo files a considered again by various code (in practice mostly persistent nodemap related). We either live with it (low inconvenient) or explicitly work around it for now. This should be good enough to no longer block the 5.9rc release with this issue. We shall build something cleaner within the 6.0 cycle. Differential Revision: https://phab.mercurial-scm.org/D11201

largefiles: replace use of synclfdirstate with drop Replace calling a complex function with a simple dedicated one Differential Revision: https://phab.mercurial-scm.org/D11113

testing: do not stop waiting if timeout is 0 (issue6541) Differential Revision: https://phab.mercurial-scm.org/D11102

largefile: use `parentchange` markcommitted Differential Revision: https://phab.mercurial-scm.org/D11112

largefile: use `set_untracked` in the override of copy/rename This is the new shiny API Differential Revision: https://phab.mercurial-scm.org/D11111

largefile: use `set_untracked` in the `forget` override This is the new shinny API. Differential Revision: https://phab.mercurial-scm.org/D11110

largefile: directly use set_untracked() for removing files This is new shiny API. Differential Revision: https://phab.mercurial-scm.org/D11109

largefile: use `parentchange` during rollback rollback is updating parent without touching the working copy. It should wrapped in a `parentchange` context. Differential Revision: https://phab.mercurial-scm.org/D11108

largefile: consider `updatelfiles` as a `parentchange` This is not strictly a `parentchange`, however this is still some internal dirstate adjustment as "similar" enough that it seems find to do so. Differential Revision: https://phab.mercurial-scm.org/D11107

largefile: use `set_tracked` in the `hg copy` overwrite This is the new shiny API. Differential Revision: https://phab.mercurial-scm.org/D11106

largefile: use parentchange during mergerecordupdates This match what core is doing around recordupdates. Differential Revision: https://phab.mercurial-scm.org/D11105

largefile: use `set_tracked` in the `hg add` overwrite This is the new shiny API. Differential Revision: https://phab.mercurial-scm.org/D11104

largefile: synchronise the dirstate within a `parentchange` context Semantically these changes are made to adjust the dirstate after a commit, so it should be in a `parentchange` context. Differential Revision: https://phab.mercurial-scm.org/D11103

upgrade_utils: fix crash for removecldeltachain action This fixes issue6383. Differential Revision: https://phab.mercurial-scm.org/D10914

windows: enforce the module policy to be c for test This is the same as what we do with the linux test. Differential Revision: https://phab.mercurial-scm.org/D11100

packaging: add command line flag to add extra vars to pyoxidizer This felt simpler than the previous incarnation of injecting content into the WiX build. I decided the easiest way to pass an arbitrary map into the process was some json - I may regret this, but time will tell on that. Differential Revision: https://phab.mercurial-scm.org/D11093

pyoxidizer: add hooks to inject extra python packages and install files We need this type of hook to inject our internal extension and resource files at Google. Presumably this could be useful to others, so instead of trying to carry an internal patch we've done this in a modular way that should be of value upstream. I'm extremely puzzled by the behavior of glob() on Windows, and I'll be filing at least one (probably two) bugs upstream about it. Differential Revision: https://phab.mercurial-scm.org/D11092

doc: wrap the help text for the purge extension Not sure why `black` or some other checker didn't complain about this. Differential Revision: https://phab.mercurial-scm.org/D11036

dummyssh: call python script with python We don't want windows to grab whatever python got configured to run .py file. We need the python used during the test to be used. Differential Revision: https://phab.mercurial-scm.org/D11083

dummyssh: use subprocess instead of os.call Welcome in 2020 Differential Revision: https://phab.mercurial-scm.org/D11082

run-test: rework the redirection script for python on windows This should get use something something overall simpler and clearer. Especially, we now have a `python.exe` script (even if by default Windows has `C:\Python27\python.exe` hardcoded in the register to open .py file) Differential Revision: https://phab.mercurial-scm.org/D11081

run-test: act that we requires MSYS to run the test on Windows If this ever change we can drop that constrains, however for now it seems clear to be clear about this limitation. Differential Revision: https://phab.mercurial-scm.org/D11080

sigpipe-remote: display more information about the non-py3 python If we are running the wrong things, it is useful to know which wrong thing we are running. Differential Revision: https://phab.mercurial-scm.org/D11079

sigpipe-remote: check for python version earlier in the script The should abort early, using the wrong python is a pretty bad sign. Differential Revision: https://phab.mercurial-scm.org/D11078

check-code: add a rules to catch os.path.abspath All previous usages have been migrated. So let us add a check-code rules to catch future usages. We restrict it to mercurial/ and hgext/ because multiple other script never depends on Mercurial modules. Differential Revision: https://phab.mercurial-scm.org/D11072

windows: use abspath in convert.subversion We replace `os.path.abspath` with `util.abspath`. This should solve more "drive capitalization" issue on Windows. Differential Revision: https://phab.mercurial-scm.org/D11071

windows: use abspath in convert.git We replace `os.path.abspath` with `util.abspath`. This should solve more "drive capitalization" issue on Windows. Differential Revision: https://phab.mercurial-scm.org/D11070

windows: use abspath in convert.bzr We replace `os.path.abspath` with `util.abspath`. This should solve more "drive capitalization" issue on Windows. Differential Revision: https://phab.mercurial-scm.org/D11069

windows: use abspath in the git extension We replace `os.path.abspath` with `util.abspath`. This should solve more "drive capitalization" issue on Windows. Differential Revision: https://phab.mercurial-scm.org/D11068

windows: use abspath in url We replace `os.path.abspath` with `util.abspath`. This should solve more "drive capitalization" issue on Windows. Differential Revision: https://phab.mercurial-scm.org/D11067

windows: use abspath in subrepoutil We replace `os.path.abspath` with `util.abspath`. This should solve more "drive capitalization" issue on Windows. Differential Revision: https://phab.mercurial-scm.org/D11066

windows: use abspath in subrepo We replace `os.path.abspath` with `util.abspath`. This should solve more "drive capitalization" issue on Windows. Differential Revision: https://phab.mercurial-scm.org/D11065

windows: use abspath in hgwebdir We replace `os.path.abspath` with `util.abspath`. This should solve more "drive capitalization" issue on Windows. Differential Revision: https://phab.mercurial-scm.org/D11064

windows: use abspath in mercurial/hg.py We replace `os.path.abspath` with `util.abspath`. This should solve more "drive capitalization" issue on Windows. Differential Revision: https://phab.mercurial-scm.org/D11063

windows: use abspath in extensions We replace `os.path.abspath` with `util.abspath`. This should solve more "drive capitalization" issue on Windows. Differential Revision: https://phab.mercurial-scm.org/D11062

windows: use abspath in commandserver We replace `os.path.abspath` with `util.abspath`. This should solve more "drive capitalization" issue on Windows. Differential Revision: https://phab.mercurial-scm.org/D11061

windows: use abspath in chgserver We replace `os.path.abspath` with `util.abspath`. This should solve more "drive capitalization" issue on Windows. Differential Revision: https://phab.mercurial-scm.org/D11060

windows: introduce a `util.abspath` to replace os.path.abspath This will let us mitigate the drive letter capitalization hell. See inline comment for details. Differential Revision: https://phab.mercurial-scm.org/D11059

windows: enforce upper case drive letter for getcwd in mercurial too This is affecting code that checks if a pull/push destination is the same as a configured one. For example the one creating divergent bookmark. Doing this fixes will help fixing `test-bookmarks.t` and `test-bookflow.t` on Windows. However, we also need to fix `abspath` invocation. Differential Revision: https://phab.mercurial-scm.org/D11058

hg-ssh: normalize the drive of the current working directory on windows For some reason os.getcwd() can return either `c:` or `C:`. We normalize this to `C:` and the like. This fix `test-ssh-bundle1.t` on windows. Differential Revision: https://phab.mercurial-scm.org/D11042

run-tests: enforce the drive letter from `getcwd` to upper case For some reason os.getcwd() can return either `c:` or `C:` depending of which binary you used on Windows. We normalize this to `C:` and the like. This fix `test-run-tests.t` on windows as the drive letter in "$TESTTMP" was "wrongly" set to 'c:/' if the test path wasn't explicitly specified. Differential Revision: https://phab.mercurial-scm.org/D11035

sigpipe-remote: simply delegate pipe forwarding to subprocess we can kill Instead of using sophisticated logics with thread a non blocking pipes, we simply spawn two new process in charge of reading the pipe and sending the result to the client. When it is time to cut the pipe we violently kill them without any remorse. This close the pipe regardless of any in progress `os.read` call. Ironically this is the very same things as what the initial shell setup was doing, but in Python. This makes the test pass run properly on Windows. This also reveal that the Windows behavior is broken as the transaction is not properly rollback. However this is an adventure for another time. Making the test behave properly was enough effort. Differential Revision: https://phab.mercurial-scm.org/D11087

sigpipe-remote: verify the script is IO are unbuffered We don't want to get stuck anywhere by buffers. Differential Revision: https://phab.mercurial-scm.org/D11086

sigpipe-remote: check environment variable sooner Lets make sure we have everything before doing anything. Differential Revision: https://phab.mercurial-scm.org/D11085

test-sigpipe: run the hook using `sh` This make the hook properly run on Windows. Differential Revision: https://phab.mercurial-scm.org/D11084

windows: add windows behavior on broken pager Apparently, Windows has "better" behavior than Unix in this case. This is an edge case that led me down a rabbit hole, only to find a bug in the Python documentation... I am not planning on trying to reproduce the same behavior on Unix systems since it's not really useful, but other people are welcome to! Differential Revision: https://phab.mercurial-scm.org/D11000

mergestate: use dirstate.update_file API in recordupdates() Let's use the new single function to replace multiple different functions. This is part of dirstate API refactor to make it more clearer and generic so that it can be replicated in Rust. Differential Revision: https://phab.mercurial-scm.org/D11077

dirstate: add dedicated function for updating data of a file `dirstate.normal()` is too generic to be a user facing function for that. This is a part of effort to refactor dirstate APIs and make them clearer. Differential Revision: https://phab.mercurial-scm.org/D11076

dirstate: add a `update_file` function This function is the other side of the `set_tracked`/`set_untracked` API revamp. It is to be used when the dirstate is changing its parents during and update or a merge. It states all the information we know about the file so that the dirstate can update its internal data. Unlike the `set_tracked`/`set_untracked` it has not regards for the information previously tracked in the tristate. Differential Revision: https://phab.mercurial-scm.org/D11075

resolve: use the `parentchange` context manager to apply merge action In an ideal world, we would not have to do that. However, we are miles away from being ready to not have to do it. So we add this context manager alongside a long comment. This will help use to get to the point were have two distinct API with strict rules about when to call them. Differential Revision: https://phab.mercurial-scm.org/D11074

sparse: adjust the temporary includes within a `parentchange` context This is related to dirstate adjustment. Differential Revision: https://phab.mercurial-scm.org/D11033

amend: adjust the dirstate within a `parentchange` context The adjustment in the direct consequence of the amend and the associated parents change. Differential Revision: https://phab.mercurial-scm.org/D11032

dirstate: use the right internal API in a test script The goal of the script it to perform arbitrary internal operation to create incorrect state, so lets make it clear. Differential Revision: https://phab.mercurial-scm.org/D11031

sparse: clear rules in the context of a `parentchanges` context This is the same logic as the change we did for narrow. Differential Revision: https://phab.mercurial-scm.org/D11029

sparse: make sure we adjust the dirstate at the same time as the parent This is more correct and help our API split. Differential Revision: https://phab.mercurial-scm.org/D11028

narrow: update narrow spec within a dirstate.parentchange context Even if the parents does not changes, the parents' content we consider is changed. So this seems legitimate. Differential Revision: https://phab.mercurial-scm.org/D11027

revert: use `set_untracked` instead of `drop` when applicable This is the new shiny API. Differential Revision: https://phab.mercurial-scm.org/D11026

revert: use `set_untracked` when performing a revert This is the new shiny API. Differential Revision: https://phab.mercurial-scm.org/D11022

mq: use `set_untracked` in `qrename` This is the new shiny API. Differential Revision: https://phab.mercurial-scm.org/D11021

context: use `dirstate.set_untracked` in `context.forget` This is the new shiny API. Differential Revision: https://phab.mercurial-scm.org/D11020

dirstate: add a `set_untracked` method for "hg remove"-like usage This is a step further toward clarifying the semantic of various dirstate call. See the justification for adding `set_tracked` for details. Differential Revision: https://phab.mercurial-scm.org/D11019

mq: use `set_tracked` in `qrename` This is the new shiny API. Differential Revision: https://phab.mercurial-scm.org/D11018

mq: update the dirstate and its parent within a `parentchange` context This is more correct, and move our plan of separated API for different dirstate usage forward. note: maybe the `parentchange` context manager should replace the dirstateguard entirely ? (in this case we should probably deprecated dirstateguard). Differential Revision: https://phab.mercurial-scm.org/D11017

context: use `dirstate.set_tracked` for `revert` This is the new shiny API. Differential Revision: https://phab.mercurial-scm.org/D11016

context: use `dirstate.set_tracked` in context.copy This is the new shiny API. Differential Revision: https://phab.mercurial-scm.org/D11015

context: use `dirstate.set_tracked` in `context.add` This is the new shiny API. Differential Revision: https://phab.mercurial-scm.org/D11014