dirstate: introduce an internal `_add` method We want to split current user of `dirstate.add` between `hg add`-like cases and update of the dirstate coming from update/merge. To do this we will introduce new API. The first step is to introduces an internal function that these new API migh use (or not use) to distinct between the migrated users and the others. Differential Revision: https://phab.mercurial-scm.org/D11010

run-tests: rely on an actual executable in PATH instead of alias for `hg` The alias approach is poorly inherited by other process that the test might spawn. To solve this we use the same approach as for `python`/`python3` we write an executable file explicitly. Doing this fixes `which hg` invocation that now returns the same location as `hg`. Using chg server side has some minor effect on some stdout/stderr ordering when using `chg` as the server too. Differential Revision: https://phab.mercurial-scm.org/D11053

tests: blacklist a handful of test with `rhg` or `chg` The use of `alias` to enforce `chg` and `rhg` means we are actually using a mix of `rhg`/`chg` and `hg` when calling `hg` in the test. Fixing this breaks various tests. This would be a large detour to fix that. I am disabling them for now with an appropriate comment. We would hopefully get back to them by the 5.8 release. Differential Revision: https://phab.mercurial-scm.org/D11052

run-tests: use more explicit signaling for `chg` Using a dedicated variable is clearer and less fragile. It cannot hurt. Differential Revision: https://phab.mercurial-scm.org/D11051

run-tests: drop the `rhg` flag for `hghave.py` if unset This seems cleaner. Differential Revision: https://phab.mercurial-scm.org/D11050

run-tests: introduce a `HGTEST_REAL_HG` variable for test It turns out that currently, `hg` and `which hg` can point to different things because `hg` is an alias… This is annoying because script and pieces of test are unknowingly using the wrong `hg`. We will fix it in another changeset. However some test actually need to use a real `hg` binary and not some `chg` or `rhg` equivalent. So we introduce a new variable with the right value and we put it to us in the appropriate location. Differential Revision: https://phab.mercurial-scm.org/D11049

run-test: clarify the error with a bad --with-hg is passed This helped me to understand what was going on when I got into trouble. Differential Revision: https://phab.mercurial-scm.org/D11046

windows: make sure we fully read and cleanly close the connection Maybe this will prevent server on Windows to sometimes complains about the client closing the connection too soon. So we make sure we read everything and we officially close the connection. Hopefully Windows will be happier and the test will stop being flaky. Differential Revision: https://phab.mercurial-scm.org/D11073

test-bookmark: clarify the path involved in a pull creating divergence This make the test easier to follow. Especially because this part is currently failing on windows. Differential Revision: https://phab.mercurial-scm.org/D11056

test-removeemptydirs: adjust to Windows behavior for the `histedit` case Windows stay in the "same" directory and the warning message is not applicable. This make `tests/test-removeemptydirs.t` pass on Windows. Differential Revision: https://phab.mercurial-scm.org/D11055

test-removeemptydirs: clarify the state of things in the `histedit` case This make the test easier to follow and will help to compare with the behavior on windows. Differential Revision: https://phab.mercurial-scm.org/D11054

test-infinitepush: drop unused helper They are unused since de4c2f3af97f. We drop them for clarity. Differential Revision: https://phab.mercurial-scm.org/D11048

run-tests: setup "correct" python earlier I don't see any good reason to wait. This result in slightly simpler code. Differential Revision: https://phab.mercurial-scm.org/D11047

run-tests: cleanup windows generation of the python / python3 pointeur They are no need to generate then in RUNTESTDIR and we don't need a python2 executable. (because not test ever use `python2` explicitly). This is probably still not perfect, but at least it is better. Differential Revision: https://phab.mercurial-scm.org/D11045

run-tests: always define a custom-bin directory We are always generating `python` / `python3` executable and we will soon often generate a `hg` executable. So let's always make it a thing. This will help us to make sure we have the appropriate binary available to all script. On both linux and Windows. Differential Revision: https://phab.mercurial-scm.org/D11044

run-tests: explicitly track that `pythondir` was inferred If we want to know that something happened, we should simply track that this things happened. This will be useful to start using `tmpbindir` more in a different form. Differential Revision: https://phab.mercurial-scm.org/D11043

run-tests: use a global WINDOWS constant instead of multiple tests This should make the code clearer. This required the adjustement of a hack in the code testing this code. Differential Revision: https://phab.mercurial-scm.org/D11041

test-run-test: use explicit variables to clarify paths Since $TESTTMP and $TESTDIR are defined for both the main scope and the test run during the test, the situation get confusing. To work around that, we define some new, explicit, variables with the values we needs to keep track of. Differential Revision: https://phab.mercurial-scm.org/D11034

run-tests: also catch double-escapes for $TESTTMP On Windows the `$TESTTMP` string "repr" is different than its value, because `\` become `\\`. We introduce a new `$STR_REPR_TESTTMP` substitution to match that too. Differential Revision: https://phab.mercurial-scm.org/D11001

ci-windows: introduce manual windows CI Note: most of the foundational work of this commit was done by Matt Harbison, but enough has changed that I don't feel comfortable sending this patch as his. In our quest to remove Python 2 from Mercurial, we need to get Windows CI going to check that Python 3 support is up to an acceptable standard. This is the first step: adding a manual step to run a full Windows CI, with a certain definition of "full" since some things are not installed yet, like SVN. Differential Revision: https://phab.mercurial-scm.org/D11025

dirstate-item: deprecate tuple access on the class This should help us to catch and update the last user of this, especially in extensions. People will need to run the test with --pure to actually catch it, but this is better than nothing. Differential Revision: https://phab.mercurial-scm.org/D10992

dirstate-item: use the v1_serialization method in debugstate I am assuming the debug command are looking into the serialized format so I am using the `v1_` variants. This assumption might be wrong. Differential Revision: https://phab.mercurial-scm.org/D10988

dirstate-item: also build DistateItem in dirstate.directories() The rust code was building tuple. Differential Revision: https://phab.mercurial-scm.org/D10983

parsers: move DirstateItem to attr.s This is the followup I requested on D10949 - it was easy, so I just did it. Differential Revision: https://phab.mercurial-scm.org/D11030

cmdutil: fix newandmodified file accounting for --interactive commits `originalchunks` is a misleading name, because it only contains header objects, which are flattened to selected hunks by the filter function. As such, `chunks not in originalchunks` is always True and misleading, because hunk objects never compare equal to header objects. This change fixes the internal naming and removes the useless parameter from the method. This change also fixes issue6533, by considering the filtered headers, rather than the hunks, when determining new and modified files. If a file is renamed + edited, and the edited hunks are deselected (but the file is not), the filtered chunks will contain a header for the file (because it's .special()) but but no hunks. Differential Revision: https://phab.mercurial-scm.org/D10936

test-run-tests: add missing backslash for Windows Differential Revision: https://phab.mercurial-scm.org/D11004

windows: use cpu-intensive task instead of real time in test Windows profiling breaks when using real time instead of cpu, but we need the extension function to show up, so make it do something instead of waiting. I've also duplicated the test cases so that explicit real and CPU times are tested (on platforms other than Windows) Differential Revision: https://phab.mercurial-scm.org/D11003

windows: use shell function instead of variable substitution This makes it compatible with Windows Differential Revision: https://phab.mercurial-scm.org/D11002

compat: normalise path before comparison in revlog splitting test This fixes the test on Window. Differential Revision: https://phab.mercurial-scm.org/D10999

windows: replicate the normalizing behavior of os.environ On Windows, `os.environ` normalizes environment variables to uppercase. Our current bytes-based environ substitution object is a simple dict, so we add the normalization behavior. This fixes test-http-peer.t on Windows. Differential Revision: https://phab.mercurial-scm.org/D10998

encoding: move case-related utils up This will be useful for the next commit that needs this code earlier. Differential Revision: https://phab.mercurial-scm.org/D11024

dirstate: drop the last explicit `state` usage in the module The internal detail of the `state` bytes is not longer exposed to the dirstate layer. Differential Revision: https://phab.mercurial-scm.org/D10977

test: use a python script in `test-transaction-rollback-on-sigpipe.t` This still does not work on Windows, but at least this is a python script now. Differential Revision: https://phab.mercurial-scm.org/D10947

test: make sure we hit the SIGPIPE in test-transaction-rollback-on-sigpipe With a coming change, we might not hit the sig pipe without issue and extra line of output. We do this early to make the next change clearer. Differential Revision: https://phab.mercurial-scm.org/D10946

test: clarify some output in `test-transaction-rollback-on-sigpipe` We don't want a dirty transaction to remain, but need to check the transaction was aborted. Otherwise it is easy to have no abandoned transaction if the transaction succeed. Differential Revision: https://phab.mercurial-scm.org/D10945

tests: add test case for issue 6262 If you split a commit, prune one of the sides, and then try to revive that commit by adding a successor, it will incorrectly be considered divergence. Differential Revision: https://phab.mercurial-scm.org/D10916

rewriteutil: look up common predecessor on unfiltered repo Before this patch, the code looking for divergence could crash when tried to look up a common predecessor in the filtered repo. This patch fixes that by looking up the common predecessor in an unfiltered repo. Differential Revision: https://phab.mercurial-scm.org/D10917

tests: demonstrate crash when common predecessor of divergence is hidden Differential Revision: https://phab.mercurial-scm.org/D11009

shallowutil: dedent code after the previous change Differential Revision: https://phab.mercurial-scm.org/D11008

shallowutil: narrow scope of try/except block This will make this code easier to understand in the future. Differential Revision: https://phab.mercurial-scm.org/D11007

remotefilelog: stop using RuntimeError for control flow We introduce a new exception to handle the various failure categories, rather than relying on RuntimeError. Differential Revision: https://phab.mercurial-scm.org/D11006

remotefilelog: tweak corrupt cache test to grep more flexibly I'm about to fix the abuse of RuntimeError here, which breaks the test if I don't do this. Differential Revision: https://phab.mercurial-scm.org/D11005

dirstatenonnormalcheck: fix some bytes formating on python3 Passing any object to `%s` no longer works, we need to explicitely convert the representation to bytes. Differential Revision: https://phab.mercurial-scm.org/D10993

dirstate-item: use the properties in `contrib/dirstatenonnormalcheck.py` Differential Revision: https://phab.mercurial-scm.org/D10991

dirstate-item: use the properties in largefiles I am now sure what large file is doing with then, but at least it is no longer using the tuple API to access data. Differential Revision: https://phab.mercurial-scm.org/D10990

dirstate-item: use the `state` property in debugpathcomplete Ideally we would use narrower property, but this is a good start. Differential Revision: https://phab.mercurial-scm.org/D10989

dirstate-item: use the properties in fakedirstatewritetime Differential Revision: https://phab.mercurial-scm.org/D10987

dirstate-item: use the properties in dirstatemap Differential Revision: https://phab.mercurial-scm.org/D10986

dirstate-item: use the properties in pathutil Differential Revision: https://phab.mercurial-scm.org/D10985

dirstate-item: use need_delay when packing dirstate Differential Revision: https://phab.mercurial-scm.org/D10984

dirstate-item: rename the class to DirstateItem The object is no longer a tuple, so it seems clearer to rename it (and its associated method) Differential Revision: https://phab.mercurial-scm.org/D10982

dirstate-entry: add a `mtime` property This is clearer than "tuple" indexing. Differential Revision: https://phab.mercurial-scm.org/D10981

dirstate-entry: add a `size` property This is clearer than "tuple" indexing. Differential Revision: https://phab.mercurial-scm.org/D10980

dirstate-entry: add a `mode` property This is clearer than "tuple" indexing. Differential Revision: https://phab.mercurial-scm.org/D10979

dirstate: no longer pass `oldstate` to the `dropfile` The `oldstate` value come literally from `_map` so we don't need to pass tha information along. Differential Revision: https://phab.mercurial-scm.org/D10978

dirstate: drop last explicite `state` usage in status Given that the previous conditional we can safely use `tracked` here. Differential Revision: https://phab.mercurial-scm.org/D10976

dirstate: reorder "state" checking conditional This is easier to read and will make this simpler to update in the next changeset. Differential Revision: https://phab.mercurial-scm.org/D10975

dirstate-entry: add a `need_delay` method This abstract the internal processing need for entry that would have an ambiguous mtime (If I understand things correctly). Differential Revision: https://phab.mercurial-scm.org/D10974

dirstate-entry: add a `tracked` property This abstract the individual `state` value and has a clear semantic. Differential Revision: https://phab.mercurial-scm.org/D10973

dirstate: drop `state` to `_addpath` All its value are inferred within the dirstatemap now. Differential Revision: https://phab.mercurial-scm.org/D10972

dirstate: infer the 'n' state from `from_p2` This flag is only used with 'n' so lets set the state based on that parameter in this case. Differential Revision: https://phab.mercurial-scm.org/D10971

dirstate: infer the 'n' state from `possibly_dirty` This flag is only used with 'n' so lets set the state based on that parameter in this case. Differential Revision: https://phab.mercurial-scm.org/D10970

dirstate: use a `merged` parameter to _addpath Differential Revision: https://phab.mercurial-scm.org/D10969

dirstate-entry: add a `added` property Lets use more semantic property instead of the `state` implementation details. Differential Revision: https://phab.mercurial-scm.org/D10968

dirstate: use a `added` parameter to _addpath My next target are the explicit `state` byte, so let start with abstracting their initial storage. Differential Revision: https://phab.mercurial-scm.org/D10967

dirstate: no longer pass the `oldstate` value to the dirstatemap The dirstatemap already have this information. Differential Revision: https://phab.mercurial-scm.org/D10966

dirstate: stop using `oldstate` in `dirstate._addpath` We don't need to pass the oldstate around, so lets use the new property instead. Differential Revision: https://phab.mercurial-scm.org/D10965

dirstate: drop the now unused magic constants for the dirstate module We no longer need them for the dirstate logic. We only need them in the dirstate map (and parsers) logic. This smell like progress. Differential Revision: https://phab.mercurial-scm.org/D10964

dirstate: move the handling of special case within the dirstatemap The dirstatemap is as well, if not better, suited to decided how to encode the various case. So we move the associated code in the dirstatemap `addfile` method. This means the dirstate no longer need to know about the various magic value used in the dirstate V1 format. The pain of the messy API start to be quite palpable in Rust, we should clean this up once the current large refactoring is dealt with. Differential Revision: https://phab.mercurial-scm.org/D10963

rust: bump rust-cpython version for 0.6.0 That function as a nice `is_none` method on `PyObject` that we will put to use. Differential Revision: https://phab.mercurial-scm.org/D10962

rust-dirstate: make the MTIME_UNSET public We will use it in the cpython layer soon. Differential Revision: https://phab.mercurial-scm.org/D10961

rust-dirstatemap: expand the wrapping code a bit This is easier to read. Differential Revision: https://phab.mercurial-scm.org/D10960

dirstate-entry: restrict `from_p2` property to tracked file The other case is now handled in `from_p2_removed`, and should ideally be moved to a lower level. So we can drop the `state == 'n'` narrowing and move it in the property itself. Differential Revision: https://phab.mercurial-scm.org/D10959

dirstate-entry: `merged_removed` and `from_p2_removed` properties Lets start to define and use more semantic property. These two might be a bit too low level and could be shaved off later, however this seems an improvement for now. Differential Revision: https://phab.mercurial-scm.org/D10958

dirstate-entry: add a `removed` property Lets start to define and use more semantic property. Differential Revision: https://phab.mercurial-scm.org/D10957

dirstate-entry: add a `from_p2` property Lets start to define and use more semantic property. Differential Revision: https://phab.mercurial-scm.org/D10956

dirstate-entry: add a `merged` property Lets start to define and use more semantic property. Differential Revision: https://phab.mercurial-scm.org/D10955

dirstate-entry: add a `state` property (and use it) This replace the [0] access. Ultimately is we should probably get ride of this in its current form. However this is a good transitional solution to move away for tuple indexing for now. Differential Revision: https://phab.mercurial-scm.org/D10954

dirstate: move most of the `remove` logic with dirstatemap `removefile` This code deal with special logic to preserving "merged" and "from_p2" information when removing a file. These are implementation details that are more suitable for the dirstatemap layer. Since the dirstatemap layer alreaday have most of the information necessary to do so, the move is easy. This move helps us to encapsulate more implementation details within the dirstatemap and its entry. Easing the use of a different storage for dirstate v2. Differential Revision: https://phab.mercurial-scm.org/D10953

dirstate: add a `in_merge` property This factor the "p2 is not null" check and is fairly simpler to read. Differential Revision: https://phab.mercurial-scm.org/D10952

dirstate-entry: introduce dedicated accessors for v1 serialization In the spirit of changing the content and storage of the dirstate entry, we add new method that the code doing v1 serialisation can use. Adding such method to the C object is quite trivial. Differential Revision: https://phab.mercurial-scm.org/D10951

dirstate-entry: goes through the `dirstatetuple` constructor in all cases We need to make sure we build an object. Differential Revision: https://phab.mercurial-scm.org/D10950

dirstate-entry: turn dirstate tuple into a real object (like in C) With dirstate V2, the stored information and actual format will change. This mean we need to start an a better abstraction for a dirstate entry that a tuple directly accessed. By chance, the C code is already doing this and pretend to be a tuple. So it should be fairly easy. We start with turning the tuple into an object, we will slowly migrate the dirstate code to no longer use the tuple directly in later changesets. Differential Revision: https://phab.mercurial-scm.org/D10949

Added signature for changeset 411dc27fd9fd

Added tag 5.8.1 for changeset 411dc27fd9fd

corruption: backout changeset 49fd21f32695 (issue6528) inverting the parent is masking copy information leading to bad content being fetched and bad status result. Since 49fd21f32695, exchange can actively swap these parent corrupting existing changesets and triggering the corruption. Data corruption are considered critical so backing this out and doing and unscheduled release seems in order. Differential Revision: https://phab.mercurial-scm.org/D10995

corruption: add a test for issue6528 The initial reproduction script was provided by Charles Chamberlain from Jane Street. Differential Revision: https://phab.mercurial-scm.org/D10996

stream: double check that self.vfs is *not* in the vfsmap The stream clone logic allows for writing any content to any file under various vfs. This is *not* suitable for *vfs*, since writing in `.hg/` directly allow to modify the configuration and is a great and simple gateway for remote code execution. Differential Revision: https://phab.mercurial-scm.org/D10905

dirstate: split dirstatemap in its own file The dirstate file is large enough and the dirstatemap is quite insulated logic already. Differential Revision: https://phab.mercurial-scm.org/D10934

run-tests: stop writing a `python3` symlink pointing to python2 Having `python3` actually pointing to `python2` is bad. So we stop doing so. In addition we need to re-introduce a `python` executable since some of the script really need to be able to say "current python" in their shbang. For example, `hghave` is one of such script. The faulty changes where introduced by c102b704edb5. Differential Revision: https://phab.mercurial-scm.org/D10943

check-code: stop forbidding return code result There is no explication of what is the intend of that check and what is the alternative. I suspect this comes from the transition to the "unified test" format circa 2010. With the non zero return explicitly listed in the output explicit $? checking became Redundant. However there is valid use case for checking $? so I am dropping this check. Differential Revision: https://phab.mercurial-scm.org/D10994

run-tests: avoid an early return To fix the "python3 pointing to python2" we will also need to create a "python" pointer. So we will need to create multiple pointer. So we need to stop using early return. We replace the early return with a loop and a continue, since the next changeset will introduce that loop anyway. Differential Revision: https://phab.mercurial-scm.org/D10942

run-tests: clarify pyexename assignement That assignement is wrong, we clarify that it is wrong before starting to fix it. This will make the fix clearer. Differential Revision: https://phab.mercurial-scm.org/D10941

run-tests: factor appdata out This is a gratuitous readability change. Differential Revision: https://phab.mercurial-scm.org/D10940

run-tests: unroll a non-so-one-liner This is shorter and clearer. Differential Revision: https://phab.mercurial-scm.org/D10939

run-tests: unroll a non-so-one-liner This is shorter and clearer. Differential Revision: https://phab.mercurial-scm.org/D10938