hg-ssh: normalize the drive of the current working directory on windows For some reason os.getcwd() can return either `c:` or `C:`. We normalize this to `C:` and the like. This fix `test-ssh-bundle1.t` on windows. Differential Revision: https://phab.mercurial-scm.org/D11042

run-tests: enforce the drive letter from `getcwd` to upper case For some reason os.getcwd() can return either `c:` or `C:` depending of which binary you used on Windows. We normalize this to `C:` and the like. This fix `test-run-tests.t` on windows as the drive letter in "$TESTTMP" was "wrongly" set to 'c:/' if the test path wasn't explicitly specified. Differential Revision: https://phab.mercurial-scm.org/D11035

sigpipe-remote: simply delegate pipe forwarding to subprocess we can kill Instead of using sophisticated logics with thread a non blocking pipes, we simply spawn two new process in charge of reading the pipe and sending the result to the client. When it is time to cut the pipe we violently kill them without any remorse. This close the pipe regardless of any in progress `os.read` call. Ironically this is the very same things as what the initial shell setup was doing, but in Python. This makes the test pass run properly on Windows. This also reveal that the Windows behavior is broken as the transaction is not properly rollback. However this is an adventure for another time. Making the test behave properly was enough effort. Differential Revision: https://phab.mercurial-scm.org/D11087

sigpipe-remote: verify the script is IO are unbuffered We don't want to get stuck anywhere by buffers. Differential Revision: https://phab.mercurial-scm.org/D11086

sigpipe-remote: check environment variable sooner Lets make sure we have everything before doing anything. Differential Revision: https://phab.mercurial-scm.org/D11085

test-sigpipe: run the hook using `sh` This make the hook properly run on Windows. Differential Revision: https://phab.mercurial-scm.org/D11084

windows: add windows behavior on broken pager Apparently, Windows has "better" behavior than Unix in this case. This is an edge case that led me down a rabbit hole, only to find a bug in the Python documentation... I am not planning on trying to reproduce the same behavior on Unix systems since it's not really useful, but other people are welcome to! Differential Revision: https://phab.mercurial-scm.org/D11000

mergestate: use dirstate.update_file API in recordupdates() Let's use the new single function to replace multiple different functions. This is part of dirstate API refactor to make it more clearer and generic so that it can be replicated in Rust. Differential Revision: https://phab.mercurial-scm.org/D11077

dirstate: add dedicated function for updating data of a file `dirstate.normal()` is too generic to be a user facing function for that. This is a part of effort to refactor dirstate APIs and make them clearer. Differential Revision: https://phab.mercurial-scm.org/D11076

dirstate: add a `update_file` function This function is the other side of the `set_tracked`/`set_untracked` API revamp. It is to be used when the dirstate is changing its parents during and update or a merge. It states all the information we know about the file so that the dirstate can update its internal data. Unlike the `set_tracked`/`set_untracked` it has not regards for the information previously tracked in the tristate. Differential Revision: https://phab.mercurial-scm.org/D11075

resolve: use the `parentchange` context manager to apply merge action In an ideal world, we would not have to do that. However, we are miles away from being ready to not have to do it. So we add this context manager alongside a long comment. This will help use to get to the point were have two distinct API with strict rules about when to call them. Differential Revision: https://phab.mercurial-scm.org/D11074

sparse: adjust the temporary includes within a `parentchange` context This is related to dirstate adjustment. Differential Revision: https://phab.mercurial-scm.org/D11033

amend: adjust the dirstate within a `parentchange` context The adjustment in the direct consequence of the amend and the associated parents change. Differential Revision: https://phab.mercurial-scm.org/D11032

dirstate: use the right internal API in a test script The goal of the script it to perform arbitrary internal operation to create incorrect state, so lets make it clear. Differential Revision: https://phab.mercurial-scm.org/D11031

sparse: clear rules in the context of a `parentchanges` context This is the same logic as the change we did for narrow. Differential Revision: https://phab.mercurial-scm.org/D11029

sparse: make sure we adjust the dirstate at the same time as the parent This is more correct and help our API split. Differential Revision: https://phab.mercurial-scm.org/D11028

narrow: update narrow spec within a dirstate.parentchange context Even if the parents does not changes, the parents' content we consider is changed. So this seems legitimate. Differential Revision: https://phab.mercurial-scm.org/D11027

revert: use `set_untracked` instead of `drop` when applicable This is the new shiny API. Differential Revision: https://phab.mercurial-scm.org/D11026

revert: use `set_untracked` when performing a revert This is the new shiny API. Differential Revision: https://phab.mercurial-scm.org/D11022

mq: use `set_untracked` in `qrename` This is the new shiny API. Differential Revision: https://phab.mercurial-scm.org/D11021

context: use `dirstate.set_untracked` in `context.forget` This is the new shiny API. Differential Revision: https://phab.mercurial-scm.org/D11020

dirstate: add a `set_untracked` method for "hg remove"-like usage This is a step further toward clarifying the semantic of various dirstate call. See the justification for adding `set_tracked` for details. Differential Revision: https://phab.mercurial-scm.org/D11019

mq: use `set_tracked` in `qrename` This is the new shiny API. Differential Revision: https://phab.mercurial-scm.org/D11018

mq: update the dirstate and its parent within a `parentchange` context This is more correct, and move our plan of separated API for different dirstate usage forward. note: maybe the `parentchange` context manager should replace the dirstateguard entirely ? (in this case we should probably deprecated dirstateguard). Differential Revision: https://phab.mercurial-scm.org/D11017

context: use `dirstate.set_tracked` for `revert` This is the new shiny API. Differential Revision: https://phab.mercurial-scm.org/D11016

context: use `dirstate.set_tracked` in context.copy This is the new shiny API. Differential Revision: https://phab.mercurial-scm.org/D11015

context: use `dirstate.set_tracked` in `context.add` This is the new shiny API. Differential Revision: https://phab.mercurial-scm.org/D11014

dirstate: add a `set_tracked` method for "hg add"-like usage This is a step further toward clarifying the semantic of various dirstate call. Having a dedicated function comes with a couple of benefits: 1) we can move duplicated logic about how to handle the previous state within the dirstate. Since we are sure this is always called in the same situation, we can implement that logic once in the dirstate. 2) having a dedicated method for this case unlock also having a dedicated method for the other case and recording more information at that time. All this leading having more code within the dirstate and higher level API that are less error prone. Differential Revision: https://phab.mercurial-scm.org/D11013

dirstate: add a function to update tracking status while "moving" parents The `scmutil.dirstateparent` is moving the dirstate parent without touching the working copy. It is used by history-rewriting operations like amending of folding. The function was directly doing the "low level" computation and dirstate change. All that logic belong to the dirstate and should be moved there. For this purpose we introduce a new function that does just that and use it. Differential Revision: https://phab.mercurial-scm.org/D11012

dirstate: introduce an internal `_drop` method We want to split current user of `dirstate.drop` between `hg rm`-like cases and update of the dirstate coming from update/merge. To do this we will introduce new API. The first step is to introduces an internal function that these new API migh use (or not use) to distinct between the migrated users and the others. Differential Revision: https://phab.mercurial-scm.org/D11023

dirstate: introduce an internal `_remove` method We want to split current user of `dirstate.remove` between `hg rm`-like cases and update of the dirstate coming from update/merge. To do this we will introduce new API. The first step is to introduces an internal function that these new API migh use (or not use) to distinct between the migrated users and the others. Differential Revision: https://phab.mercurial-scm.org/D11011

dirstate: introduce an internal `_add` method We want to split current user of `dirstate.add` between `hg add`-like cases and update of the dirstate coming from update/merge. To do this we will introduce new API. The first step is to introduces an internal function that these new API migh use (or not use) to distinct between the migrated users and the others. Differential Revision: https://phab.mercurial-scm.org/D11010

run-tests: rely on an actual executable in PATH instead of alias for `hg` The alias approach is poorly inherited by other process that the test might spawn. To solve this we use the same approach as for `python`/`python3` we write an executable file explicitly. Doing this fixes `which hg` invocation that now returns the same location as `hg`. Using chg server side has some minor effect on some stdout/stderr ordering when using `chg` as the server too. Differential Revision: https://phab.mercurial-scm.org/D11053

tests: blacklist a handful of test with `rhg` or `chg` The use of `alias` to enforce `chg` and `rhg` means we are actually using a mix of `rhg`/`chg` and `hg` when calling `hg` in the test. Fixing this breaks various tests. This would be a large detour to fix that. I am disabling them for now with an appropriate comment. We would hopefully get back to them by the 5.8 release. Differential Revision: https://phab.mercurial-scm.org/D11052

run-tests: use more explicit signaling for `chg` Using a dedicated variable is clearer and less fragile. It cannot hurt. Differential Revision: https://phab.mercurial-scm.org/D11051

run-tests: drop the `rhg` flag for `hghave.py` if unset This seems cleaner. Differential Revision: https://phab.mercurial-scm.org/D11050

run-tests: introduce a `HGTEST_REAL_HG` variable for test It turns out that currently, `hg` and `which hg` can point to different things because `hg` is an alias… This is annoying because script and pieces of test are unknowingly using the wrong `hg`. We will fix it in another changeset. However some test actually need to use a real `hg` binary and not some `chg` or `rhg` equivalent. So we introduce a new variable with the right value and we put it to us in the appropriate location. Differential Revision: https://phab.mercurial-scm.org/D11049

run-test: clarify the error with a bad --with-hg is passed This helped me to understand what was going on when I got into trouble. Differential Revision: https://phab.mercurial-scm.org/D11046

windows: make sure we fully read and cleanly close the connection Maybe this will prevent server on Windows to sometimes complains about the client closing the connection too soon. So we make sure we read everything and we officially close the connection. Hopefully Windows will be happier and the test will stop being flaky. Differential Revision: https://phab.mercurial-scm.org/D11073

test-bookmark: clarify the path involved in a pull creating divergence This make the test easier to follow. Especially because this part is currently failing on windows. Differential Revision: https://phab.mercurial-scm.org/D11056

test-removeemptydirs: adjust to Windows behavior for the `histedit` case Windows stay in the "same" directory and the warning message is not applicable. This make `tests/test-removeemptydirs.t` pass on Windows. Differential Revision: https://phab.mercurial-scm.org/D11055

test-removeemptydirs: clarify the state of things in the `histedit` case This make the test easier to follow and will help to compare with the behavior on windows. Differential Revision: https://phab.mercurial-scm.org/D11054

test-infinitepush: drop unused helper They are unused since de4c2f3af97f. We drop them for clarity. Differential Revision: https://phab.mercurial-scm.org/D11048

run-tests: setup "correct" python earlier I don't see any good reason to wait. This result in slightly simpler code. Differential Revision: https://phab.mercurial-scm.org/D11047

run-tests: cleanup windows generation of the python / python3 pointeur They are no need to generate then in RUNTESTDIR and we don't need a python2 executable. (because not test ever use `python2` explicitly). This is probably still not perfect, but at least it is better. Differential Revision: https://phab.mercurial-scm.org/D11045

run-tests: always define a custom-bin directory We are always generating `python` / `python3` executable and we will soon often generate a `hg` executable. So let's always make it a thing. This will help us to make sure we have the appropriate binary available to all script. On both linux and Windows. Differential Revision: https://phab.mercurial-scm.org/D11044

run-tests: explicitly track that `pythondir` was inferred If we want to know that something happened, we should simply track that this things happened. This will be useful to start using `tmpbindir` more in a different form. Differential Revision: https://phab.mercurial-scm.org/D11043

run-tests: use a global WINDOWS constant instead of multiple tests This should make the code clearer. This required the adjustement of a hack in the code testing this code. Differential Revision: https://phab.mercurial-scm.org/D11041

test-run-test: use explicit variables to clarify paths Since $TESTTMP and $TESTDIR are defined for both the main scope and the test run during the test, the situation get confusing. To work around that, we define some new, explicit, variables with the values we needs to keep track of. Differential Revision: https://phab.mercurial-scm.org/D11034

run-tests: also catch double-escapes for $TESTTMP On Windows the `$TESTTMP` string "repr" is different than its value, because `\` become `\\`. We introduce a new `$STR_REPR_TESTTMP` substitution to match that too. Differential Revision: https://phab.mercurial-scm.org/D11001

ci-windows: introduce manual windows CI Note: most of the foundational work of this commit was done by Matt Harbison, but enough has changed that I don't feel comfortable sending this patch as his. In our quest to remove Python 2 from Mercurial, we need to get Windows CI going to check that Python 3 support is up to an acceptable standard. This is the first step: adding a manual step to run a full Windows CI, with a certain definition of "full" since some things are not installed yet, like SVN. Differential Revision: https://phab.mercurial-scm.org/D11025

dirstate-item: deprecate tuple access on the class This should help us to catch and update the last user of this, especially in extensions. People will need to run the test with --pure to actually catch it, but this is better than nothing. Differential Revision: https://phab.mercurial-scm.org/D10992

dirstate-item: use the v1_serialization method in debugstate I am assuming the debug command are looking into the serialized format so I am using the `v1_` variants. This assumption might be wrong. Differential Revision: https://phab.mercurial-scm.org/D10988

dirstate-item: also build DistateItem in dirstate.directories() The rust code was building tuple. Differential Revision: https://phab.mercurial-scm.org/D10983

parsers: move DirstateItem to attr.s This is the followup I requested on D10949 - it was easy, so I just did it. Differential Revision: https://phab.mercurial-scm.org/D11030

cmdutil: fix newandmodified file accounting for --interactive commits `originalchunks` is a misleading name, because it only contains header objects, which are flattened to selected hunks by the filter function. As such, `chunks not in originalchunks` is always True and misleading, because hunk objects never compare equal to header objects. This change fixes the internal naming and removes the useless parameter from the method. This change also fixes issue6533, by considering the filtered headers, rather than the hunks, when determining new and modified files. If a file is renamed + edited, and the edited hunks are deselected (but the file is not), the filtered chunks will contain a header for the file (because it's .special()) but but no hunks. Differential Revision: https://phab.mercurial-scm.org/D10936

test-run-tests: add missing backslash for Windows Differential Revision: https://phab.mercurial-scm.org/D11004

windows: use cpu-intensive task instead of real time in test Windows profiling breaks when using real time instead of cpu, but we need the extension function to show up, so make it do something instead of waiting. I've also duplicated the test cases so that explicit real and CPU times are tested (on platforms other than Windows) Differential Revision: https://phab.mercurial-scm.org/D11003

windows: use shell function instead of variable substitution This makes it compatible with Windows Differential Revision: https://phab.mercurial-scm.org/D11002

compat: normalise path before comparison in revlog splitting test This fixes the test on Window. Differential Revision: https://phab.mercurial-scm.org/D10999

windows: replicate the normalizing behavior of os.environ On Windows, `os.environ` normalizes environment variables to uppercase. Our current bytes-based environ substitution object is a simple dict, so we add the normalization behavior. This fixes test-http-peer.t on Windows. Differential Revision: https://phab.mercurial-scm.org/D10998

encoding: move case-related utils up This will be useful for the next commit that needs this code earlier. Differential Revision: https://phab.mercurial-scm.org/D11024

dirstate: drop the last explicit `state` usage in the module The internal detail of the `state` bytes is not longer exposed to the dirstate layer. Differential Revision: https://phab.mercurial-scm.org/D10977

test: use a python script in `test-transaction-rollback-on-sigpipe.t` This still does not work on Windows, but at least this is a python script now. Differential Revision: https://phab.mercurial-scm.org/D10947