dirstate: introduce a `set_clean` method This will provide a good alternative to `normal` call with a narrower semantic. Differential Revision: https://phab.mercurial-scm.org/D11146

largefile: use `update_file` instead of `normal` This is the newer, more semantic API. Differential Revision: https://phab.mercurial-scm.org/D11145

largefile: adjust the dirstate post update within a `parentchange` context This is doing an update, so this seems more appropriate. No other change have been made, but the next changeset will :-) Differential Revision: https://phab.mercurial-scm.org/D11144

dirstate: deprecate `dirstate.remove` in all cases This will make sure that code will use the newer, more semantic API sooner than later. Differential Revision: https://phab.mercurial-scm.org/D11143

largefile: use `update_file` instead of `remove` in `synclfdirstate` This is the newer, more semantic API. Differential Revision: https://phab.mercurial-scm.org/D11142

largefile: rearrange conditionnal in `synclfdirstate` We can liquidate the special case early, so do we. Differential Revision: https://phab.mercurial-scm.org/D11141

largefile: use `update_file` instead of `remove` in `mergerecordupdates` This is the newer, more semantic API. Differential Revision: https://phab.mercurial-scm.org/D11140

mq: use `update_file_p1` instead of `remove` when adjusting dirstate This is the new API. Differential Revision: https://phab.mercurial-scm.org/D11139

dirstate: properly update `_lastnormaltime` in `update_file_p1` This was previously overlooked. Differential Revision: https://phab.mercurial-scm.org/D11138

dirstate: properly update `_lastnormaltime` in `update_file` This was previously overlooked. Differential Revision: https://phab.mercurial-scm.org/D11137

dirstate: use `reset_state` in `update_file_p1` Going through the same API is more consistent and allow us to push implementation lower down the call stack. Differential Revision: https://phab.mercurial-scm.org/D11136

dirstate: rename `update_file_reference` to `update_file_p1` This is a bit clearer on the intended usage of the function. (but still not great) Differential Revision: https://phab.mercurial-scm.org/D11135

dirstate-map: move most of `dirstate.update_file` logic in the dsmap A new `reset_state` method is introduced to deal with most of that logic. This move things one layer lower, but the ultimate goal is to deal with most of this at the DirstateItem level. This reveal various imperfection with the data passed to update_file by `mergestate.recordupdates`, however this is orthogonal to this patch and should be dealt with at a higher level. Differential Revision: https://phab.mercurial-scm.org/D11134

dirstate: replace `update_parent_file_data` with simpler `update_parent` call This make the dirstate API simpler. Differential Revision: https://phab.mercurial-scm.org/D11133

dirstate: factor out the part retrieve "filedata" out of `normal` We will need them elsewhere. Differential Revision: https://phab.mercurial-scm.org/D11132

dirstate-map: factor out the change to _dirs and _alldirs on removing This logic is complicated enough to deserves its own function. So it now does. This will make it easier to reuse that logic in later changeset. Differential Revision: https://phab.mercurial-scm.org/D11131

dirstate-map: factor out the change to _dirs and _alldirs on dropping This logic is complicated enough to deserves its own function. So it now does. This will make it easier to reuse that logic in later changeset. Differential Revision: https://phab.mercurial-scm.org/D11130

dirstate-map: factor out the change to _dirs and _alldirs on adding This logic is complicated enough to deserves its own function. So it now does. This will make it easier to reuse that logic in later changeset. Differential Revision: https://phab.mercurial-scm.org/D11129

dirstate-item: introduce a `dm_otherparent` property See inline documentation for details. Differential Revision: https://phab.mercurial-scm.org/D11124

dirstate-item: introduce a `dm_nonnormal` property See inline documentation for details. Differential Revision: https://phab.mercurial-scm.org/D11123

dirstate-v2: Reserve a few bytes of space for future extensions See doc-comment Differential Revision: https://phab.mercurial-scm.org/D11101

dirstate-v2: Separate iterators for dirfoldmap and debugdirstate `dirstatemap.dirfoldmap` was recently changed to re-use a Rust iterator that was added for the `hg debugdirstate` command. That iterator was based on all nodes in the tree dirstate without an entry only existing to hold child nodes, and therefore being directories. However to optimize status further we may want to store additional nodes for unknown or ignored files and directories. At that point the two users of this iterator will want different things, so let’s make two iterators instead. See doc-comments in `dispatch.rs`. Differential Revision: https://phab.mercurial-scm.org/D11099

dirstate-v2: Move fixed-size tree metadata into the docket file Before this changeset, the dirstate-v2 data file contained not only nodes and paths that may be reused when appending to an existing file, but also some fixed-size metadata that applies to the entire tree and was added at the end of the data file for every append. This moves that metadata into the docket file, so that repeated "append" operations without meaningful changes don’t actually need to grow any file. Differential Revision: https://phab.mercurial-scm.org/D11098

dirstate-v2: Add heuristic for when to create a new data file … instead of appending to the existing one. This is based on keeping track of how much of the existing data is not used anymore. Differential Revision: https://phab.mercurial-scm.org/D11097

dirstate-v2: Reuse existing paths when appending to a data file When writing a dirstate in v2 format by appending to an existing data file, filenames / paths that are borrowed from the previous on-disk representation can be reused. Differential Revision: https://phab.mercurial-scm.org/D11096

dirstate-v2: Reuse existing nodes when appending to a data file When writing a dirstate in v2 format by appending to an existing data file, nodes that are still "unparsed" from the previous on-disk representation have been unchanged and can therefore be reused. This makes the new data point to previously-existing data for tree nodes. Differential Revision: https://phab.mercurial-scm.org/D11095

dirstate-v2: Support appending to the same data file For now we’re still writing the entire data every time, so appending is not useful yet. Later we’ll have new nodes pointing to some existing data for nodes and paths that haven’t changed. The decision whether to append is pseudo-random in order to make tests exercise both code paths. This will be replaced by a heuristic based on the amount of unused existing data. Differential Revision: https://phab.mercurial-scm.org/D11094

dirstate-v2: shrink on-disk path lengths to 16-bits Differential Revision: https://phab.mercurial-scm.org/D11091

dirstate-v2: Rename Header to Root, move it to the end of the data file Now that they don’t have to be at the start, a given data file may contain multiple "roots". A docket only points to one of them, and previous ones are left unused to allow allow append-only in-place writing to an existing data file. Differential Revision: https://phab.mercurial-scm.org/D11090

dirstate-v2: Enforce data size read from the docket file The data file may not be shorter than its size given by the docket. It may be longer, but additional data is ignored. Differential Revision: https://phab.mercurial-scm.org/D11089

dirstate-v2: Introduce a docket file .hg/dirstate now only contains some metadata to point to a separate data file named .hg/dirstate.{}.d with a random hexadecimal identifier. For now every update creates a new data file and removes the old one, but later we’ll (usually) append to an existing file. Separating into two files allows doing the "write to a temporary file then atomically rename into destination" dance with only a small docket file, without always rewriting a lot of data. Differential Revision: https://phab.mercurial-scm.org/D11088

dirstate: replace a dead conditional branch with an assert in `update_file` This is a case we never meet, so lets trim it away to simplify the code before more changes. Differential Revision: https://phab.mercurial-scm.org/D11128

dirstate-map: do not use `size` to gate copy dropping during remove_file This get us close to moving the block right above withing the DirstateItem object. Doing so will help us getting rid of magic constant at the dirstatemap level. Differential Revision: https://phab.mercurial-scm.org/D11127

dirstate: drop a duplicated assert This very case is checked in 3 line above that one. Differential Revision: https://phab.mercurial-scm.org/D11126

dirstate: drop duplicated check This is covered by the `@requires_parents_change` decorator that this function use. Differential Revision: https://phab.mercurial-scm.org/D11125

dirstate: add a `set_possibly_dirty` in `fakedirstatewritetime` Differential Revision: https://phab.mercurial-scm.org/D11122

dirstatemap: use `set_possibly_dirty` in `clearambiguoustimes` lets put this new method to use Differential Revision: https://phab.mercurial-scm.org/D11121

dirstate-item: use `set_possibly_dirty` in `pure.pack_dirstate` Lets make use of the new function. Differential Revision: https://phab.mercurial-scm.org/D11120

dirstate-item: add a `set_possibly_dirty` method See inline documentation for details. The pushes the AMBIGUOUS_TIME implementation further down the line within the DirstateItem only. When this cleanup is done we will be able to stop using this representation internally. Differential Revision: https://phab.mercurial-scm.org/D11119

dirstate-item: add a `from_v1_data` constructor This class method is dedicated to building a DirstateItem from the data available in the "dirstate-v1" format. Since that format is frozen, this constructor will never change (unlike the `__init__` one). Differential Revision: https://phab.mercurial-scm.org/D11118

dirstate-item: use an explicit __init__ function instead of the attrs one For now, this is not doing anything special, however we plan to make it different in the future. So we start simple. Differential Revision: https://phab.mercurial-scm.org/D11117

dirstate: deprecated `drop` outside of `update/merge` All core users have been migrated. Differential Revision: https://phab.mercurial-scm.org/D11116

dirstate: deprecated `remove` outside of `update/merge` All core users have been migrated. Differential Revision: https://phab.mercurial-scm.org/D11115

dirstate: deprecate the `add` method outside of update/merge context All core users have been updated. Differential Revision: https://phab.mercurial-scm.org/D11114

upgrade: avoid a traceback in case of unrecognized revlog Without this, in case of revlog name not matching any know pattern we would get a traceback. Raising a clear error seems simpler. Differential Revision: https://phab.mercurial-scm.org/D11202

walk: no longer ignore revlogs of files starting with `undo.` (issue6542) Changeset 0b569c75d180 introduced new code in store.walk to filter out undo files left behind by the transaction. However doing so is also filtering out legitimate revlog file starting with `undo.` So this changeset is mostly rolling back that change and adding tests tests to catch this kind of error in the future. As a result we the transaction undo files a considered again by various code (in practice mostly persistent nodemap related). We either live with it (low inconvenient) or explicitly work around it for now. This should be good enough to no longer block the 5.9rc release with this issue. We shall build something cleaner within the 6.0 cycle. Differential Revision: https://phab.mercurial-scm.org/D11201

largefiles: replace use of synclfdirstate with drop Replace calling a complex function with a simple dedicated one Differential Revision: https://phab.mercurial-scm.org/D11113

testing: do not stop waiting if timeout is 0 (issue6541) Differential Revision: https://phab.mercurial-scm.org/D11102

largefile: use `parentchange` markcommitted Differential Revision: https://phab.mercurial-scm.org/D11112

largefile: use `set_untracked` in the override of copy/rename This is the new shiny API Differential Revision: https://phab.mercurial-scm.org/D11111

largefile: use `set_untracked` in the `forget` override This is the new shinny API. Differential Revision: https://phab.mercurial-scm.org/D11110

largefile: directly use set_untracked() for removing files This is new shiny API. Differential Revision: https://phab.mercurial-scm.org/D11109

largefile: use `parentchange` during rollback rollback is updating parent without touching the working copy. It should wrapped in a `parentchange` context. Differential Revision: https://phab.mercurial-scm.org/D11108

largefile: consider `updatelfiles` as a `parentchange` This is not strictly a `parentchange`, however this is still some internal dirstate adjustment as "similar" enough that it seems find to do so. Differential Revision: https://phab.mercurial-scm.org/D11107

largefile: use `set_tracked` in the `hg copy` overwrite This is the new shiny API. Differential Revision: https://phab.mercurial-scm.org/D11106

largefile: use parentchange during mergerecordupdates This match what core is doing around recordupdates. Differential Revision: https://phab.mercurial-scm.org/D11105

largefile: use `set_tracked` in the `hg add` overwrite This is the new shiny API. Differential Revision: https://phab.mercurial-scm.org/D11104

largefile: synchronise the dirstate within a `parentchange` context Semantically these changes are made to adjust the dirstate after a commit, so it should be in a `parentchange` context. Differential Revision: https://phab.mercurial-scm.org/D11103

upgrade_utils: fix crash for removecldeltachain action This fixes issue6383. Differential Revision: https://phab.mercurial-scm.org/D10914

windows: enforce the module policy to be c for test This is the same as what we do with the linux test. Differential Revision: https://phab.mercurial-scm.org/D11100

packaging: add command line flag to add extra vars to pyoxidizer This felt simpler than the previous incarnation of injecting content into the WiX build. I decided the easiest way to pass an arbitrary map into the process was some json - I may regret this, but time will tell on that. Differential Revision: https://phab.mercurial-scm.org/D11093

pyoxidizer: add hooks to inject extra python packages and install files We need this type of hook to inject our internal extension and resource files at Google. Presumably this could be useful to others, so instead of trying to carry an internal patch we've done this in a modular way that should be of value upstream. I'm extremely puzzled by the behavior of glob() on Windows, and I'll be filing at least one (probably two) bugs upstream about it. Differential Revision: https://phab.mercurial-scm.org/D11092

doc: wrap the help text for the purge extension Not sure why `black` or some other checker didn't complain about this. Differential Revision: https://phab.mercurial-scm.org/D11036

dummyssh: call python script with python We don't want windows to grab whatever python got configured to run .py file. We need the python used during the test to be used. Differential Revision: https://phab.mercurial-scm.org/D11083

dummyssh: use subprocess instead of os.call Welcome in 2020 Differential Revision: https://phab.mercurial-scm.org/D11082

run-test: rework the redirection script for python on windows This should get use something something overall simpler and clearer. Especially, we now have a `python.exe` script (even if by default Windows has `C:\Python27\python.exe` hardcoded in the register to open .py file) Differential Revision: https://phab.mercurial-scm.org/D11081

run-test: act that we requires MSYS to run the test on Windows If this ever change we can drop that constrains, however for now it seems clear to be clear about this limitation. Differential Revision: https://phab.mercurial-scm.org/D11080

sigpipe-remote: display more information about the non-py3 python If we are running the wrong things, it is useful to know which wrong thing we are running. Differential Revision: https://phab.mercurial-scm.org/D11079

sigpipe-remote: check for python version earlier in the script The should abort early, using the wrong python is a pretty bad sign. Differential Revision: https://phab.mercurial-scm.org/D11078

check-code: add a rules to catch os.path.abspath All previous usages have been migrated. So let us add a check-code rules to catch future usages. We restrict it to mercurial/ and hgext/ because multiple other script never depends on Mercurial modules. Differential Revision: https://phab.mercurial-scm.org/D11072

windows: use abspath in convert.subversion We replace `os.path.abspath` with `util.abspath`. This should solve more "drive capitalization" issue on Windows. Differential Revision: https://phab.mercurial-scm.org/D11071

windows: use abspath in convert.git We replace `os.path.abspath` with `util.abspath`. This should solve more "drive capitalization" issue on Windows. Differential Revision: https://phab.mercurial-scm.org/D11070

windows: use abspath in convert.bzr We replace `os.path.abspath` with `util.abspath`. This should solve more "drive capitalization" issue on Windows. Differential Revision: https://phab.mercurial-scm.org/D11069

windows: use abspath in the git extension We replace `os.path.abspath` with `util.abspath`. This should solve more "drive capitalization" issue on Windows. Differential Revision: https://phab.mercurial-scm.org/D11068

windows: use abspath in url We replace `os.path.abspath` with `util.abspath`. This should solve more "drive capitalization" issue on Windows. Differential Revision: https://phab.mercurial-scm.org/D11067

windows: use abspath in subrepoutil We replace `os.path.abspath` with `util.abspath`. This should solve more "drive capitalization" issue on Windows. Differential Revision: https://phab.mercurial-scm.org/D11066

windows: use abspath in subrepo We replace `os.path.abspath` with `util.abspath`. This should solve more "drive capitalization" issue on Windows. Differential Revision: https://phab.mercurial-scm.org/D11065

windows: use abspath in hgwebdir We replace `os.path.abspath` with `util.abspath`. This should solve more "drive capitalization" issue on Windows. Differential Revision: https://phab.mercurial-scm.org/D11064

windows: use abspath in mercurial/hg.py We replace `os.path.abspath` with `util.abspath`. This should solve more "drive capitalization" issue on Windows. Differential Revision: https://phab.mercurial-scm.org/D11063

windows: use abspath in extensions We replace `os.path.abspath` with `util.abspath`. This should solve more "drive capitalization" issue on Windows. Differential Revision: https://phab.mercurial-scm.org/D11062

windows: use abspath in commandserver We replace `os.path.abspath` with `util.abspath`. This should solve more "drive capitalization" issue on Windows. Differential Revision: https://phab.mercurial-scm.org/D11061

windows: use abspath in chgserver We replace `os.path.abspath` with `util.abspath`. This should solve more "drive capitalization" issue on Windows. Differential Revision: https://phab.mercurial-scm.org/D11060

windows: introduce a `util.abspath` to replace os.path.abspath This will let us mitigate the drive letter capitalization hell. See inline comment for details. Differential Revision: https://phab.mercurial-scm.org/D11059

windows: enforce upper case drive letter for getcwd in mercurial too This is affecting code that checks if a pull/push destination is the same as a configured one. For example the one creating divergent bookmark. Doing this fixes will help fixing `test-bookmarks.t` and `test-bookflow.t` on Windows. However, we also need to fix `abspath` invocation. Differential Revision: https://phab.mercurial-scm.org/D11058

hg-ssh: normalize the drive of the current working directory on windows For some reason os.getcwd() can return either `c:` or `C:`. We normalize this to `C:` and the like. This fix `test-ssh-bundle1.t` on windows. Differential Revision: https://phab.mercurial-scm.org/D11042

run-tests: enforce the drive letter from `getcwd` to upper case For some reason os.getcwd() can return either `c:` or `C:` depending of which binary you used on Windows. We normalize this to `C:` and the like. This fix `test-run-tests.t` on windows as the drive letter in "$TESTTMP" was "wrongly" set to 'c:/' if the test path wasn't explicitly specified. Differential Revision: https://phab.mercurial-scm.org/D11035

sigpipe-remote: simply delegate pipe forwarding to subprocess we can kill Instead of using sophisticated logics with thread a non blocking pipes, we simply spawn two new process in charge of reading the pipe and sending the result to the client. When it is time to cut the pipe we violently kill them without any remorse. This close the pipe regardless of any in progress `os.read` call. Ironically this is the very same things as what the initial shell setup was doing, but in Python. This makes the test pass run properly on Windows. This also reveal that the Windows behavior is broken as the transaction is not properly rollback. However this is an adventure for another time. Making the test behave properly was enough effort. Differential Revision: https://phab.mercurial-scm.org/D11087

sigpipe-remote: verify the script is IO are unbuffered We don't want to get stuck anywhere by buffers. Differential Revision: https://phab.mercurial-scm.org/D11086

sigpipe-remote: check environment variable sooner Lets make sure we have everything before doing anything. Differential Revision: https://phab.mercurial-scm.org/D11085

test-sigpipe: run the hook using `sh` This make the hook properly run on Windows. Differential Revision: https://phab.mercurial-scm.org/D11084

windows: add windows behavior on broken pager Apparently, Windows has "better" behavior than Unix in this case. This is an edge case that led me down a rabbit hole, only to find a bug in the Python documentation... I am not planning on trying to reproduce the same behavior on Unix systems since it's not really useful, but other people are welcome to! Differential Revision: https://phab.mercurial-scm.org/D11000

mergestate: use dirstate.update_file API in recordupdates() Let's use the new single function to replace multiple different functions. This is part of dirstate API refactor to make it more clearer and generic so that it can be replicated in Rust. Differential Revision: https://phab.mercurial-scm.org/D11077

dirstate: add dedicated function for updating data of a file `dirstate.normal()` is too generic to be a user facing function for that. This is a part of effort to refactor dirstate APIs and make them clearer. Differential Revision: https://phab.mercurial-scm.org/D11076

dirstate: add a `update_file` function This function is the other side of the `set_tracked`/`set_untracked` API revamp. It is to be used when the dirstate is changing its parents during and update or a merge. It states all the information we know about the file so that the dirstate can update its internal data. Unlike the `set_tracked`/`set_untracked` it has not regards for the information previously tracked in the tristate. Differential Revision: https://phab.mercurial-scm.org/D11075

resolve: use the `parentchange` context manager to apply merge action In an ideal world, we would not have to do that. However, we are miles away from being ready to not have to do it. So we add this context manager alongside a long comment. This will help use to get to the point were have two distinct API with strict rules about when to call them. Differential Revision: https://phab.mercurial-scm.org/D11074

sparse: adjust the temporary includes within a `parentchange` context This is related to dirstate adjustment. Differential Revision: https://phab.mercurial-scm.org/D11033

amend: adjust the dirstate within a `parentchange` context The adjustment in the direct consequence of the amend and the associated parents change. Differential Revision: https://phab.mercurial-scm.org/D11032

dirstate: use the right internal API in a test script The goal of the script it to perform arbitrary internal operation to create incorrect state, so lets make it clear. Differential Revision: https://phab.mercurial-scm.org/D11031

sparse: clear rules in the context of a `parentchanges` context This is the same logic as the change we did for narrow. Differential Revision: https://phab.mercurial-scm.org/D11029

sparse: make sure we adjust the dirstate at the same time as the parent This is more correct and help our API split. Differential Revision: https://phab.mercurial-scm.org/D11028

narrow: update narrow spec within a dirstate.parentchange context Even if the parents does not changes, the parents' content we consider is changed. So this seems legitimate. Differential Revision: https://phab.mercurial-scm.org/D11027

revert: use `set_untracked` instead of `drop` when applicable This is the new shiny API. Differential Revision: https://phab.mercurial-scm.org/D11026

revert: use `set_untracked` when performing a revert This is the new shiny API. Differential Revision: https://phab.mercurial-scm.org/D11022

mq: use `set_untracked` in `qrename` This is the new shiny API. Differential Revision: https://phab.mercurial-scm.org/D11021

context: use `dirstate.set_untracked` in `context.forget` This is the new shiny API. Differential Revision: https://phab.mercurial-scm.org/D11020

dirstate: add a `set_untracked` method for "hg remove"-like usage This is a step further toward clarifying the semantic of various dirstate call. See the justification for adding `set_tracked` for details. Differential Revision: https://phab.mercurial-scm.org/D11019

mq: use `set_tracked` in `qrename` This is the new shiny API. Differential Revision: https://phab.mercurial-scm.org/D11018

mq: update the dirstate and its parent within a `parentchange` context This is more correct, and move our plan of separated API for different dirstate usage forward. note: maybe the `parentchange` context manager should replace the dirstateguard entirely ? (in this case we should probably deprecated dirstateguard). Differential Revision: https://phab.mercurial-scm.org/D11017

context: use `dirstate.set_tracked` for `revert` This is the new shiny API. Differential Revision: https://phab.mercurial-scm.org/D11016

context: use `dirstate.set_tracked` in context.copy This is the new shiny API. Differential Revision: https://phab.mercurial-scm.org/D11015

context: use `dirstate.set_tracked` in `context.add` This is the new shiny API. Differential Revision: https://phab.mercurial-scm.org/D11014

dirstate: add a `set_tracked` method for "hg add"-like usage This is a step further toward clarifying the semantic of various dirstate call. Having a dedicated function comes with a couple of benefits: 1) we can move duplicated logic about how to handle the previous state within the dirstate. Since we are sure this is always called in the same situation, we can implement that logic once in the dirstate. 2) having a dedicated method for this case unlock also having a dedicated method for the other case and recording more information at that time. All this leading having more code within the dirstate and higher level API that are less error prone. Differential Revision: https://phab.mercurial-scm.org/D11013

dirstate: add a function to update tracking status while "moving" parents The `scmutil.dirstateparent` is moving the dirstate parent without touching the working copy. It is used by history-rewriting operations like amending of folding. The function was directly doing the "low level" computation and dirstate change. All that logic belong to the dirstate and should be moved there. For this purpose we introduce a new function that does just that and use it. Differential Revision: https://phab.mercurial-scm.org/D11012

dirstate: introduce an internal `_drop` method We want to split current user of `dirstate.drop` between `hg rm`-like cases and update of the dirstate coming from update/merge. To do this we will introduce new API. The first step is to introduces an internal function that these new API migh use (or not use) to distinct between the migrated users and the others. Differential Revision: https://phab.mercurial-scm.org/D11023

dirstate: introduce an internal `_remove` method We want to split current user of `dirstate.remove` between `hg rm`-like cases and update of the dirstate coming from update/merge. To do this we will introduce new API. The first step is to introduces an internal function that these new API migh use (or not use) to distinct between the migrated users and the others. Differential Revision: https://phab.mercurial-scm.org/D11011

dirstate: introduce an internal `_add` method We want to split current user of `dirstate.add` between `hg add`-like cases and update of the dirstate coming from update/merge. To do this we will introduce new API. The first step is to introduces an internal function that these new API migh use (or not use) to distinct between the migrated users and the others. Differential Revision: https://phab.mercurial-scm.org/D11010

run-tests: rely on an actual executable in PATH instead of alias for `hg` The alias approach is poorly inherited by other process that the test might spawn. To solve this we use the same approach as for `python`/`python3` we write an executable file explicitly. Doing this fixes `which hg` invocation that now returns the same location as `hg`. Using chg server side has some minor effect on some stdout/stderr ordering when using `chg` as the server too. Differential Revision: https://phab.mercurial-scm.org/D11053

tests: blacklist a handful of test with `rhg` or `chg` The use of `alias` to enforce `chg` and `rhg` means we are actually using a mix of `rhg`/`chg` and `hg` when calling `hg` in the test. Fixing this breaks various tests. This would be a large detour to fix that. I am disabling them for now with an appropriate comment. We would hopefully get back to them by the 5.8 release. Differential Revision: https://phab.mercurial-scm.org/D11052

run-tests: use more explicit signaling for `chg` Using a dedicated variable is clearer and less fragile. It cannot hurt. Differential Revision: https://phab.mercurial-scm.org/D11051

run-tests: drop the `rhg` flag for `hghave.py` if unset This seems cleaner. Differential Revision: https://phab.mercurial-scm.org/D11050

run-tests: introduce a `HGTEST_REAL_HG` variable for test It turns out that currently, `hg` and `which hg` can point to different things because `hg` is an alias… This is annoying because script and pieces of test are unknowingly using the wrong `hg`. We will fix it in another changeset. However some test actually need to use a real `hg` binary and not some `chg` or `rhg` equivalent. So we introduce a new variable with the right value and we put it to us in the appropriate location. Differential Revision: https://phab.mercurial-scm.org/D11049

run-test: clarify the error with a bad --with-hg is passed This helped me to understand what was going on when I got into trouble. Differential Revision: https://phab.mercurial-scm.org/D11046

windows: make sure we fully read and cleanly close the connection Maybe this will prevent server on Windows to sometimes complains about the client closing the connection too soon. So we make sure we read everything and we officially close the connection. Hopefully Windows will be happier and the test will stop being flaky. Differential Revision: https://phab.mercurial-scm.org/D11073

test-bookmark: clarify the path involved in a pull creating divergence This make the test easier to follow. Especially because this part is currently failing on windows. Differential Revision: https://phab.mercurial-scm.org/D11056

test-removeemptydirs: adjust to Windows behavior for the `histedit` case Windows stay in the "same" directory and the warning message is not applicable. This make `tests/test-removeemptydirs.t` pass on Windows. Differential Revision: https://phab.mercurial-scm.org/D11055

test-removeemptydirs: clarify the state of things in the `histedit` case This make the test easier to follow and will help to compare with the behavior on windows. Differential Revision: https://phab.mercurial-scm.org/D11054

test-infinitepush: drop unused helper They are unused since de4c2f3af97f. We drop them for clarity. Differential Revision: https://phab.mercurial-scm.org/D11048

run-tests: setup "correct" python earlier I don't see any good reason to wait. This result in slightly simpler code. Differential Revision: https://phab.mercurial-scm.org/D11047

run-tests: cleanup windows generation of the python / python3 pointeur They are no need to generate then in RUNTESTDIR and we don't need a python2 executable. (because not test ever use `python2` explicitly). This is probably still not perfect, but at least it is better. Differential Revision: https://phab.mercurial-scm.org/D11045

run-tests: always define a custom-bin directory We are always generating `python` / `python3` executable and we will soon often generate a `hg` executable. So let's always make it a thing. This will help us to make sure we have the appropriate binary available to all script. On both linux and Windows. Differential Revision: https://phab.mercurial-scm.org/D11044

run-tests: explicitly track that `pythondir` was inferred If we want to know that something happened, we should simply track that this things happened. This will be useful to start using `tmpbindir` more in a different form. Differential Revision: https://phab.mercurial-scm.org/D11043

run-tests: use a global WINDOWS constant instead of multiple tests This should make the code clearer. This required the adjustement of a hack in the code testing this code. Differential Revision: https://phab.mercurial-scm.org/D11041

test-run-test: use explicit variables to clarify paths Since $TESTTMP and $TESTDIR are defined for both the main scope and the test run during the test, the situation get confusing. To work around that, we define some new, explicit, variables with the values we needs to keep track of. Differential Revision: https://phab.mercurial-scm.org/D11034

run-tests: also catch double-escapes for $TESTTMP On Windows the `$TESTTMP` string "repr" is different than its value, because `\` become `\\`. We introduce a new `$STR_REPR_TESTTMP` substitution to match that too. Differential Revision: https://phab.mercurial-scm.org/D11001

ci-windows: introduce manual windows CI Note: most of the foundational work of this commit was done by Matt Harbison, but enough has changed that I don't feel comfortable sending this patch as his. In our quest to remove Python 2 from Mercurial, we need to get Windows CI going to check that Python 3 support is up to an acceptable standard. This is the first step: adding a manual step to run a full Windows CI, with a certain definition of "full" since some things are not installed yet, like SVN. Differential Revision: https://phab.mercurial-scm.org/D11025

dirstate-item: deprecate tuple access on the class This should help us to catch and update the last user of this, especially in extensions. People will need to run the test with --pure to actually catch it, but this is better than nothing. Differential Revision: https://phab.mercurial-scm.org/D10992

dirstate-item: use the v1_serialization method in debugstate I am assuming the debug command are looking into the serialized format so I am using the `v1_` variants. This assumption might be wrong. Differential Revision: https://phab.mercurial-scm.org/D10988

dirstate-item: also build DistateItem in dirstate.directories() The rust code was building tuple. Differential Revision: https://phab.mercurial-scm.org/D10983

parsers: move DirstateItem to attr.s This is the followup I requested on D10949 - it was easy, so I just did it. Differential Revision: https://phab.mercurial-scm.org/D11030

cmdutil: fix newandmodified file accounting for --interactive commits `originalchunks` is a misleading name, because it only contains header objects, which are flattened to selected hunks by the filter function. As such, `chunks not in originalchunks` is always True and misleading, because hunk objects never compare equal to header objects. This change fixes the internal naming and removes the useless parameter from the method. This change also fixes issue6533, by considering the filtered headers, rather than the hunks, when determining new and modified files. If a file is renamed + edited, and the edited hunks are deselected (but the file is not), the filtered chunks will contain a header for the file (because it's .special()) but but no hunks. Differential Revision: https://phab.mercurial-scm.org/D10936

test-run-tests: add missing backslash for Windows Differential Revision: https://phab.mercurial-scm.org/D11004

windows: use cpu-intensive task instead of real time in test Windows profiling breaks when using real time instead of cpu, but we need the extension function to show up, so make it do something instead of waiting. I've also duplicated the test cases so that explicit real and CPU times are tested (on platforms other than Windows) Differential Revision: https://phab.mercurial-scm.org/D11003

windows: use shell function instead of variable substitution This makes it compatible with Windows Differential Revision: https://phab.mercurial-scm.org/D11002

compat: normalise path before comparison in revlog splitting test This fixes the test on Window. Differential Revision: https://phab.mercurial-scm.org/D10999

windows: replicate the normalizing behavior of os.environ On Windows, `os.environ` normalizes environment variables to uppercase. Our current bytes-based environ substitution object is a simple dict, so we add the normalization behavior. This fixes test-http-peer.t on Windows. Differential Revision: https://phab.mercurial-scm.org/D10998

encoding: move case-related utils up This will be useful for the next commit that needs this code earlier. Differential Revision: https://phab.mercurial-scm.org/D11024

dirstate: drop the last explicit `state` usage in the module The internal detail of the `state` bytes is not longer exposed to the dirstate layer. Differential Revision: https://phab.mercurial-scm.org/D10977

test: use a python script in `test-transaction-rollback-on-sigpipe.t` This still does not work on Windows, but at least this is a python script now. Differential Revision: https://phab.mercurial-scm.org/D10947

test: make sure we hit the SIGPIPE in test-transaction-rollback-on-sigpipe With a coming change, we might not hit the sig pipe without issue and extra line of output. We do this early to make the next change clearer. Differential Revision: https://phab.mercurial-scm.org/D10946

test: clarify some output in `test-transaction-rollback-on-sigpipe` We don't want a dirty transaction to remain, but need to check the transaction was aborted. Otherwise it is easy to have no abandoned transaction if the transaction succeed. Differential Revision: https://phab.mercurial-scm.org/D10945

tests: add test case for issue 6262 If you split a commit, prune one of the sides, and then try to revive that commit by adding a successor, it will incorrectly be considered divergence. Differential Revision: https://phab.mercurial-scm.org/D10916

rewriteutil: look up common predecessor on unfiltered repo Before this patch, the code looking for divergence could crash when tried to look up a common predecessor in the filtered repo. This patch fixes that by looking up the common predecessor in an unfiltered repo. Differential Revision: https://phab.mercurial-scm.org/D10917

tests: demonstrate crash when common predecessor of divergence is hidden Differential Revision: https://phab.mercurial-scm.org/D11009

shallowutil: dedent code after the previous change Differential Revision: https://phab.mercurial-scm.org/D11008

shallowutil: narrow scope of try/except block This will make this code easier to understand in the future. Differential Revision: https://phab.mercurial-scm.org/D11007

remotefilelog: stop using RuntimeError for control flow We introduce a new exception to handle the various failure categories, rather than relying on RuntimeError. Differential Revision: https://phab.mercurial-scm.org/D11006

remotefilelog: tweak corrupt cache test to grep more flexibly I'm about to fix the abuse of RuntimeError here, which breaks the test if I don't do this. Differential Revision: https://phab.mercurial-scm.org/D11005

dirstatenonnormalcheck: fix some bytes formating on python3 Passing any object to `%s` no longer works, we need to explicitely convert the representation to bytes. Differential Revision: https://phab.mercurial-scm.org/D10993

dirstate-item: use the properties in `contrib/dirstatenonnormalcheck.py` Differential Revision: https://phab.mercurial-scm.org/D10991

dirstate-item: use the properties in largefiles I am now sure what large file is doing with then, but at least it is no longer using the tuple API to access data. Differential Revision: https://phab.mercurial-scm.org/D10990

dirstate-item: use the `state` property in debugpathcomplete Ideally we would use narrower property, but this is a good start. Differential Revision: https://phab.mercurial-scm.org/D10989

dirstate-item: use the properties in fakedirstatewritetime Differential Revision: https://phab.mercurial-scm.org/D10987

dirstate-item: use the properties in dirstatemap Differential Revision: https://phab.mercurial-scm.org/D10986

dirstate-item: use the properties in pathutil Differential Revision: https://phab.mercurial-scm.org/D10985

dirstate-item: use need_delay when packing dirstate Differential Revision: https://phab.mercurial-scm.org/D10984

dirstate-item: rename the class to DirstateItem The object is no longer a tuple, so it seems clearer to rename it (and its associated method) Differential Revision: https://phab.mercurial-scm.org/D10982

dirstate-entry: add a `mtime` property This is clearer than "tuple" indexing. Differential Revision: https://phab.mercurial-scm.org/D10981

dirstate-entry: add a `size` property This is clearer than "tuple" indexing. Differential Revision: https://phab.mercurial-scm.org/D10980

dirstate-entry: add a `mode` property This is clearer than "tuple" indexing. Differential Revision: https://phab.mercurial-scm.org/D10979

dirstate: no longer pass `oldstate` to the `dropfile` The `oldstate` value come literally from `_map` so we don't need to pass tha information along. Differential Revision: https://phab.mercurial-scm.org/D10978

dirstate: drop last explicite `state` usage in status Given that the previous conditional we can safely use `tracked` here. Differential Revision: https://phab.mercurial-scm.org/D10976

dirstate: reorder "state" checking conditional This is easier to read and will make this simpler to update in the next changeset. Differential Revision: https://phab.mercurial-scm.org/D10975

dirstate-entry: add a `need_delay` method This abstract the internal processing need for entry that would have an ambiguous mtime (If I understand things correctly). Differential Revision: https://phab.mercurial-scm.org/D10974

dirstate-entry: add a `tracked` property This abstract the individual `state` value and has a clear semantic. Differential Revision: https://phab.mercurial-scm.org/D10973

dirstate: drop `state` to `_addpath` All its value are inferred within the dirstatemap now. Differential Revision: https://phab.mercurial-scm.org/D10972

dirstate: infer the 'n' state from `from_p2` This flag is only used with 'n' so lets set the state based on that parameter in this case. Differential Revision: https://phab.mercurial-scm.org/D10971

dirstate: infer the 'n' state from `possibly_dirty` This flag is only used with 'n' so lets set the state based on that parameter in this case. Differential Revision: https://phab.mercurial-scm.org/D10970

dirstate: use a `merged` parameter to _addpath Differential Revision: https://phab.mercurial-scm.org/D10969

dirstate-entry: add a `added` property Lets use more semantic property instead of the `state` implementation details. Differential Revision: https://phab.mercurial-scm.org/D10968

dirstate: use a `added` parameter to _addpath My next target are the explicit `state` byte, so let start with abstracting their initial storage. Differential Revision: https://phab.mercurial-scm.org/D10967

dirstate: no longer pass the `oldstate` value to the dirstatemap The dirstatemap already have this information. Differential Revision: https://phab.mercurial-scm.org/D10966

dirstate: stop using `oldstate` in `dirstate._addpath` We don't need to pass the oldstate around, so lets use the new property instead. Differential Revision: https://phab.mercurial-scm.org/D10965

dirstate: drop the now unused magic constants for the dirstate module We no longer need them for the dirstate logic. We only need them in the dirstate map (and parsers) logic. This smell like progress. Differential Revision: https://phab.mercurial-scm.org/D10964

dirstate: move the handling of special case within the dirstatemap The dirstatemap is as well, if not better, suited to decided how to encode the various case. So we move the associated code in the dirstatemap `addfile` method. This means the dirstate no longer need to know about the various magic value used in the dirstate V1 format. The pain of the messy API start to be quite palpable in Rust, we should clean this up once the current large refactoring is dealt with. Differential Revision: https://phab.mercurial-scm.org/D10963

rust: bump rust-cpython version for 0.6.0 That function as a nice `is_none` method on `PyObject` that we will put to use. Differential Revision: https://phab.mercurial-scm.org/D10962

rust-dirstate: make the MTIME_UNSET public We will use it in the cpython layer soon. Differential Revision: https://phab.mercurial-scm.org/D10961

rust-dirstatemap: expand the wrapping code a bit This is easier to read. Differential Revision: https://phab.mercurial-scm.org/D10960

dirstate-entry: restrict `from_p2` property to tracked file The other case is now handled in `from_p2_removed`, and should ideally be moved to a lower level. So we can drop the `state == 'n'` narrowing and move it in the property itself. Differential Revision: https://phab.mercurial-scm.org/D10959

dirstate-entry: `merged_removed` and `from_p2_removed` properties Lets start to define and use more semantic property. These two might be a bit too low level and could be shaved off later, however this seems an improvement for now. Differential Revision: https://phab.mercurial-scm.org/D10958

dirstate-entry: add a `removed` property Lets start to define and use more semantic property. Differential Revision: https://phab.mercurial-scm.org/D10957

dirstate-entry: add a `from_p2` property Lets start to define and use more semantic property. Differential Revision: https://phab.mercurial-scm.org/D10956

dirstate-entry: add a `merged` property Lets start to define and use more semantic property. Differential Revision: https://phab.mercurial-scm.org/D10955

dirstate-entry: add a `state` property (and use it) This replace the [0] access. Ultimately is we should probably get ride of this in its current form. However this is a good transitional solution to move away for tuple indexing for now. Differential Revision: https://phab.mercurial-scm.org/D10954

dirstate: move most of the `remove` logic with dirstatemap `removefile` This code deal with special logic to preserving "merged" and "from_p2" information when removing a file. These are implementation details that are more suitable for the dirstatemap layer. Since the dirstatemap layer alreaday have most of the information necessary to do so, the move is easy. This move helps us to encapsulate more implementation details within the dirstatemap and its entry. Easing the use of a different storage for dirstate v2. Differential Revision: https://phab.mercurial-scm.org/D10953

dirstate: add a `in_merge` property This factor the "p2 is not null" check and is fairly simpler to read. Differential Revision: https://phab.mercurial-scm.org/D10952

dirstate-entry: introduce dedicated accessors for v1 serialization In the spirit of changing the content and storage of the dirstate entry, we add new method that the code doing v1 serialisation can use. Adding such method to the C object is quite trivial. Differential Revision: https://phab.mercurial-scm.org/D10951

dirstate-entry: goes through the `dirstatetuple` constructor in all cases We need to make sure we build an object. Differential Revision: https://phab.mercurial-scm.org/D10950

dirstate-entry: turn dirstate tuple into a real object (like in C) With dirstate V2, the stored information and actual format will change. This mean we need to start an a better abstraction for a dirstate entry that a tuple directly accessed. By chance, the C code is already doing this and pretend to be a tuple. So it should be fairly easy. We start with turning the tuple into an object, we will slowly migrate the dirstate code to no longer use the tuple directly in later changesets. Differential Revision: https://phab.mercurial-scm.org/D10949

dirstate: split dirstatemap in its own file The dirstate file is large enough and the dirstatemap is quite insulated logic already. Differential Revision: https://phab.mercurial-scm.org/D10934

run-tests: stop writing a `python3` symlink pointing to python2 Having `python3` actually pointing to `python2` is bad. So we stop doing so. In addition we need to re-introduce a `python` executable since some of the script really need to be able to say "current python" in their shbang. For example, `hghave` is one of such script. The faulty changes where introduced by c102b704edb5. Differential Revision: https://phab.mercurial-scm.org/D10943

check-code: stop forbidding return code result There is no explication of what is the intend of that check and what is the alternative. I suspect this comes from the transition to the "unified test" format circa 2010. With the non zero return explicitly listed in the output explicit $? checking became Redundant. However there is valid use case for checking $? so I am dropping this check. Differential Revision: https://phab.mercurial-scm.org/D10994

run-tests: avoid an early return To fix the "python3 pointing to python2" we will also need to create a "python" pointer. So we will need to create multiple pointer. So we need to stop using early return. We replace the early return with a loop and a continue, since the next changeset will introduce that loop anyway. Differential Revision: https://phab.mercurial-scm.org/D10942

run-tests: clarify pyexename assignement That assignement is wrong, we clarify that it is wrong before starting to fix it. This will make the fix clearer. Differential Revision: https://phab.mercurial-scm.org/D10941

run-tests: factor appdata out This is a gratuitous readability change. Differential Revision: https://phab.mercurial-scm.org/D10940

run-tests: unroll a non-so-one-liner This is shorter and clearer. Differential Revision: https://phab.mercurial-scm.org/D10939

run-tests: unroll a non-so-one-liner This is shorter and clearer. Differential Revision: https://phab.mercurial-scm.org/D10938

testing: fix _timeout_factor With `--debug`, `run-tests.py` set the timeout to 0... that breaks the logic in `mercurial.testing`. Differential Revision: https://phab.mercurial-scm.org/D10937

exchange: use "served" repo filter to guess what the server will publish Previously, the second push in the added test would say "push would publish 2 changesets" because _checkpublish() was considering secret changesets to be "pushable" when push command had a --rev argument. Without --rev argument to push command, we already filter the repo properly. Differential Revision: https://phab.mercurial-scm.org/D10948

rust: Document the DirstateMapMethods trait Differential Revision: https://phab.mercurial-scm.org/D10919

dirstate: document the dirstatetuple content We are about to modify this, so let us document the existing code. Differential Revision: https://phab.mercurial-scm.org/D10935

dirstate: explicitely deal with the `added` case in `_addpath` This special case is now directly handled, this clarify the possible value for the other options. Differential Revision: https://phab.mercurial-scm.org/D10933

dirstate: add an explicit `possibly_dirty` parameter to `_addpath` This lets says what we mean instead of using magic value. The lower level can then decide how to express that. Differential Revision: https://phab.mercurial-scm.org/D10932

dirstate: add an explicit `from_p2` parameter to `_addpath` This lets says what we mean instead of using magic value. The lower level can then decide how to express that. Differential Revision: https://phab.mercurial-scm.org/D10931

dirstate: add default value to _addpath We want to remove the magic value usage from the higher level, so lets stop passing them explicitely when possible. Differential Revision: https://phab.mercurial-scm.org/D10930

dirstate: move the _rangemask filtering closer to its storage The goal is to get ready to move these kind of processing at a lower level. We start with move move _rangemask filtering close to where it is sent to the lower level to make the future move trivial. Differential Revision: https://phab.mercurial-scm.org/D10929

dirstate: introduce a symbolic constant for the AMBIGUOUS_TIME marker This is going to be clearer and easier to track than -1. Ultimately I would like to get ride of this special value everywhere but in the lower level, however we need to clarify the API first. This changeset is part of such clarification. Differential Revision: https://phab.mercurial-scm.org/D10928

dirstate: introduce a symbolic constant for the NONNORMAL marker This is going to be clearer and easier to track than -1. Ultimately I would like to get ride of this special value everywhere but in the lower level, however we need to clarify the API first. This changeset is part of such clarification. Differential Revision: https://phab.mercurial-scm.org/D10927

dirstate: introduce a symbolic constant for the FROM_P2 marker This is going to be clearer and easier to track than -2. Ultimately I would like to get ride of this special value everywhere but in the lower level, however we need to clarify the API first. This changeset is part of such clarification. Differential Revision: https://phab.mercurial-scm.org/D10926

dirstate: split a not-so-one-liner This is shorter and simpler to read. Differential Revision: https://phab.mercurial-scm.org/D10925

dirstate: split a not-so-one-liner This is simpler to read. Differential Revision: https://phab.mercurial-scm.org/D10924

dirstate: split a not-so-one-liner This is shorter and simpler to read. Differential Revision: https://phab.mercurial-scm.org/D10923

dirstate-tree: Keep a counter of descendant nodes that have an entry … and change the `DirstateMap::has_dir` method to be based on this counter being non-zero instead of the mere presence of a node. A node with zero descendent with an entry currently should be removed from the tree, but soon we’ll make the dirstate track additional nodes. (Specifically, for non-ignored directories in order to keep track of their mtime and optimize status by doing fewer `read_dir` calls.) Differential Revision: https://phab.mercurial-scm.org/D10922

dirstate: Removed unused instances of `DirsMultiset` … in Rust-backed dirstatemap. The Python class `dirstatemap` had cached properties `_dirs` and `_alldirs` that were not used for `hastrackeddir` and `hasdir` since they were redundant with corresponding fields for the Rust `DirstateMap` struct. `dirfoldmap` is modified to reuse instead the directory iterator introduced in 3b9914b28133c0918186b6e8b9e4f1916e21338d. Differential Revision: https://phab.mercurial-scm.org/D10921

dirstate-v2: Use 32-bit integers instead of 64-bit for offsets This saves 12 bytes per node. (Nodes representing files or directories.) These are offsets to other parts of the file. This would only be a limitation for a `.hg/dirstate` file larger than 4 GiB, which would only happen for a repository with dozens of millions of files and directories. Differential Revision: https://phab.mercurial-scm.org/D10920

status: Extend read_dir caching to directories with ignored files See code comments Differential Revision: https://phab.mercurial-scm.org/D10909

dirstate-v2: Drop cached read_dir results after .hgignore changes Soon we’ll want the status algorithm to be able to skip `std::fs::read_dir` in more cases, notabling when listing unknown files but not ignored files. When ignore patterns change (which we detect by their hash, added to the dirstate-v2 format in a previous changeset), a formerly-ignored file could become unknown without changing its parent directory’s modification time. Therefore we remove any directory mtime from the dirstate, effictively invalidating the existing caches. Differential Revision: https://phab.mercurial-scm.org/D10907

censor: split the core of the logic into its own function We now have a "generic" rewrite function (only able to do censoring for now) and a thin wrapper that implement the `censor` API with it. We are now ready to start incorporating strip specific changes. Differential Revision: https://phab.mercurial-scm.org/D10904

censor: migrate the logic to a set of `censor_revs` Instead of considering a special unique censored revision within the code, we consider a set of revision (currently always of size 1). This make the main code less censor-centric and prepare for the usage of a similar approach for stripping changesets. Differential Revision: https://phab.mercurial-scm.org/D10903

censor: rename `rl` to `revlog` in the main function Now that the bulk of the code moved to smaller function we are less restricted on line length and we can use more explicite naming. Differential Revision: https://phab.mercurial-scm.org/D10902

censor: extract the part about creating and opening new files in a function The v2_censor function is huge, now that its content has settled a bit it is a good time to split individual part inside dedicated function. The last part is the file copying and opening logic. It now have its own function. Differential Revision: https://phab.mercurial-scm.org/D10901

censor: extract the part about recomputing delta in a function The v2_censor function is huge, now that its content has settled a bit it is a good time to split individual part inside dedicated function. It is now the turn of the logic that recompute new delta to replace the one based on the revision that are going away. Differential Revision: https://phab.mercurial-scm.org/D10900

censor: extract the part about writing the other revision in a function The v2_censor function is huge, now that its content has settled a bit it is a good time to split individual part inside dedicated function. We continue with a small function that add a non-censored revision back to the revlog. Differential Revision: https://phab.mercurial-scm.org/D10899

censor: extract the part about writing the censored revision in a function The v2_censor function is huge, now that its content has settled a bit it is a good time to split individual part inside dedicated function. We start with a small function that process the censored revision. Differential Revision: https://phab.mercurial-scm.org/D10898

censor: put the tuple of open files in an explicit variable This will be simpler to pass these file around in future changesets. Differential Revision: https://phab.mercurial-scm.org/D10897

revlog: rewrite `censors.py` to `rewrite.py` The logic we use for censoring revision will be mostly common with the one we needs for stripping. So we rename the module to `rewrite` to better match its future content. Differential Revision: https://phab.mercurial-scm.org/D10896

copies: Keep changelog sidedata file open during copy tracing Instead of having a callback that opens and closes that file many times, a add and use a context manager method on the `revlog` class that keeps files open for its duration. Differential Revision: https://phab.mercurial-scm.org/D10888

purge: restore the `testedwith` declaration Otherwise, this extension gets highlighted as a 3rd party extension when an exception occurs. Differential Revision: https://phab.mercurial-scm.org/D10895

rust: remove dead utils module This was introduced a while back while not being imported anywhere in the module system. How pertinent this Facebook module was to the codebase was debatable at the time, nowadays with the Rust VFS layer having laid its foundations, it is clear that we should drop it. Differential Revision: https://phab.mercurial-scm.org/D10890

rebase: use str-keyed opts in remaining places Differential Revision: https://phab.mercurial-scm.org/D10892

rebase: keep str-keyed opts long enough to make `action` a str This is just another little py3 cleanup. Differential Revision: https://phab.mercurial-scm.org/D10891

benchmarks: restore `output` variable lost in D10884 Noticed by pyflakes (which I didn't have installed for Python 3.9 when I sent D10884). Differential Revision: https://phab.mercurial-scm.org/D10894

censor: implement censoring for revlogv2 It is a bit verbose and rough, but it works. Most of that logic can be common for `stripping`, so we can expect more refactoring of that code to accommodate both needs. However I wanted to keep this changesets "simple enough" and before moving forward. We also need to properly delete the older index/data/sidedata file, but this has implication for streaming clone and transaction, so this will come later. Differential Revision: https://phab.mercurial-scm.org/D10869

deltas: at a `target_rev` parameter to finddeltainfo Otherwise, recomputing a delta for a revision might result in a delta against a later revision or a full snapshot thinking we are appending a new revision. We will make use of this during censoring (and later, stripping). Differential Revision: https://phab.mercurial-scm.org/D10887

revlog: list older-but-still-around file in `files` These older files will eventually be removed, but we have to take them in account until they are. So we update `files` to keep fncache happy. Differential Revision: https://phab.mercurial-scm.org/D10868

revlog: add docket method to request new content files If we want to write content into new files, we need to be able to ask for them. Differential Revision: https://phab.mercurial-scm.org/D10867

revlog: add a way to keep track of older uids in the docket When the revlog content is rewritten, we will use new files, to avoid truncating the previous ones. We need some way to keep track of the older files before we clean them up. Differential Revision: https://phab.mercurial-scm.org/D10866

revlog: factor the logic to determine the delta compression out This make the logic clearly isolated and documented and it will help use to reuse it during censors/strip. Differential Revision: https://phab.mercurial-scm.org/D10889

nodemap: fix some comment formatting Black seems unhappy about it. Differential Revision: https://phab.mercurial-scm.org/D10865

revlogv2: fix `hg verify` with revlog v2 We added a bunch of new files but we never teached some internal about it. This is now fixed. Differential Revision: https://phab.mercurial-scm.org/D10858

revlogv2: also test that local clone works This is now the case so lets add a test. Differential Revision: https://phab.mercurial-scm.org/D10857

clone: also report the bookmark file as copied This is a small UI adjustement, but this is easy enough to do. Differential Revision: https://phab.mercurial-scm.org/D10856

clone: reuse the stream clone logic for local clone Streaming clone and local (non `--pull`) clone do mostly the same thing, however they were using different logic to do so. This means the logic frequently went out of sync and that new case had to be dealt with twice. This is fragile and anoying. So we replace this with a re-use of the logic we use for streaming clone. I can see various test changes: - a more precise progress output, - armless fncache loading during clone, - fncache is no longer hardlinked (since we write it by hand). I am not reinstalling the `reposimplestore` specific output, as far as I understand this variant have been broken for years. Differential Revision: https://phab.mercurial-scm.org/D10855

copyfiles: add a way to relax the file system checking for hardlink This is critical for transaction file, less for hardlink/copy clone as we are about to do. Since `pure` build does not have a `getfstype` implementation this would disable hardlink clone for all pure build. So we add a parameter to control that extra check. Differential Revision: https://phab.mercurial-scm.org/D10854

copyfile: add a option callback for failed hardlinking Local clone, adjust its UI depending on the success of using hardlinking, so we add a small callback making it possible for `copyfile` to signal if the requested hardlinking failed. Differential Revision: https://phab.mercurial-scm.org/D10853

streamingclone: extract the scanning part from the generation part We will reuse the scanning part for local clone, so we need it in a dedicated function. Differential Revision: https://phab.mercurial-scm.org/D10852

vfs: add a `register_file` method on the vfs class This is used by the fncache vfs to register new file. Until now, `fncache` have been doing this "automatically" by monitoring write pattern. However this is fragile and when we copy files in place by other means, we need something more robuts. So we add an explicit method to do so. Differential Revision: https://phab.mercurial-scm.org/D10851