dirstatemap: replace `removefile` by an explicit `entry.set_untracked()` All the other caller goes through `reset_state`, so we can safely have an explicit method on `DirstateItem` object. This means that all the logic to preserve the previous state (from p2, merged, etc) is now properly encapsulated within the DirstateItem. This pave the way to using different storage for these information. Differential Revision: https://phab.mercurial-scm.org/D11315

dirstate: forward `remove` call to newer `API` The `_remove` method was only called in the deprecated `remove` function. We merge the two and express it in terms of call to new API methods. Differential Revision: https://phab.mercurial-scm.org/D11314

branching: merge stable into default

store: return just one filename in walk functions Various walk functions return `(revlog_type, decoded, encoded)` where decoded could be None. But no-one cares about `encoded` and expects `unencoded` to be present, except verify (because this can only happen with old repo formats). Simplify all this by either failing outright if a decoding a filename fails (instead of almost certainly failing with a type error due to treating None as a bytes), or skipping the filename but providing in an out argument for hg verify. Differential Revision: https://phab.mercurial-scm.org/D11248

tests: rename test-clone-uncompressed.t as clone --uncompressed is deprecated in favor of --stream Differential Revision: https://phab.mercurial-scm.org/D11237

debugrebuildfncache: add a cheaper option to rebuild the fncache On my repository, debugrebuildfncache takes 5-10min with the lock. With the flag added in this commit, it takes 10s. The tradeoff is that it only recovers from certain kinds of corruptions. It is intended to to recover faster from fncaches broken by a revlog split during a transaction that ends up being rolled back. Differential Revision: https://phab.mercurial-scm.org/D11265

test: reduce noise, so the important bits stand out Differential Revision: https://phab.mercurial-scm.org/D11264

wireprotov1peer: update all rpcs to use the new batchable scheme If desired, we could keep the future class and the function that upgrades an old style rpc instead of a new style, for extensions. Differential Revision: https://phab.mercurial-scm.org/D11212

wireprotov1peer: simplify the way batchable rpcs are defined The scheme with futures/generator is confusing due to the way communication is done by side effects, especially with two different "future" objects. Just returning a request and a function to read the response is easier to understand. There are tests failures with the largefiles extension due to it aliasing one rpc to another one, which gets fixed in the next commit. Differential Revision: https://phab.mercurial-scm.org/D11211

dirstate: directly call the dirstatemap in `set_untracked` This function is only called in two places: the deprecated "remove" method and in the new `set_untracked` method. So we simply inline the appropriate content in `set_untracked`, paving the way to dropping the deprecated code and its associated function. Differential Revision: https://phab.mercurial-scm.org/D11313

resources: narrow the try:except clause to minimum Otherwise this mind hides other import or attribute errors. Differential Revision: https://phab.mercurial-scm.org/D11312

merge with stable

hgwebdir: avoid systematic full garbage collection Forcing a systematic full garbage collection upon each request can serioulsy harm performance. This is reported as https://bz.mercurial-scm.org/show_bug.cgi?id=6075 With this change we're performing the full collection according to a new setting, `experimental.web.full-garbage-collection-rate`. The default value is 1, which doesn't change the behavior and will allow us to test on real use cases. If the value is 0, no full garbage collection occurs. Regardless of the value of the setting, a partial garbage collection still occurs upon each request (not attempting to collect objects from the oldest generation). This should be enough to take care of reference cycles that have been created by the last request (assessment of this requires changing the setting, not to be 1). In my experience chasing memory leaks in Mercurial servers, the full collection never reclaimed any memory, but this is with Python 3 and biased towards small repositories. On the other hand, as explained in the Python developer docs [1], frequent full collections are very harmful in terms of performance if lots of objects survive the collection, and hence stay in the oldest generation. Note that `gc.collect()` is indeed trying to collect the oldest generation [2]. This happens usually in two cases: - unwanted lingering objects (i.e., an actual memory leak that the GC cannot do anything about). Sadly, we have lots of those these days. - desireable long-term objects, typically in caches (not inner caches carried by repositories, which should be collected with them). This is a subject of interest for the Heptapod project. In short, the flat rate that this change still permits is probably a bad idea in most cases, and the default value can be tweaked later on (or even be set to 0) according to experiments in the wild. The test is inspired from test-hgwebdir-paths.py [1] https://devguide.python.org/garbage_collector/#collecting-the-oldest-generation [2] https://docs.python.org/3/library/gc.html#gc.collect Differential Revision: https://phab.mercurial-scm.org/D11204

obsolete: disable other evolution config options if createmarkers is off We used to raise an abort in this case, but recent changes to local clone command (377d8fc20e34) resulted in destrepo both caring about experimental.evolution config options and not initializing extensions. So imagine if you had evolve and allowdivergence enabled in your ~/.hgrc. Local clone stopped working after 377d8fc20e34 because evolve sets experimental.evolution=all, but only on srcrepo, for destrepo the extension is not initialized. It's possible to make local cloning work by initializing extensions for destrepo in some cases, but in other cases (e.g. allowdivergence in ~/.hgrc, evolve extension in original-repo/.hg/hgrc) it would still fail. In a discussion with Pierre-Yves David it was decided to simply force other evolution options to be false if createmarkers is not enabled. Differential Revision: https://phab.mercurial-scm.org/D11223

fix: use obsolete.isenabled() to check for experimental.allowdivergence Now that obsolete.isenabled() can also check if divergence is allowed, let's use it for consistency. Other experimental.evolution options are already checked via this function. Differential Revision: https://phab.mercurial-scm.org/D11222

rebase: use obsolete.isenabled() to check for experimental.allowdivergence Now that obsolete.isenabled() can also check if divergence is allowed, let's use it for consistency. Other experimental.evolution options are already checked via this function. Differential Revision: https://phab.mercurial-scm.org/D11221

typing: add several assertions to dirstatemap to appease pytype I think it's been mentioned in IRC that these can't be None in this case. This fixes: File "/mnt/c/Users/Matt/hg/mercurial/dirstatemap.py", line 213, in addfile: unsupported operand type(s) for &: 'None' and 'int' [unsupported-operands] No attribute '__and__' on None or '__rand__' on int Called from (traceback): line 290, in reset_state File "/mnt/c/Users/Matt/hg/mercurial/dirstatemap.py", line 214, in addfile: unsupported operand type(s) for &: 'None' and 'int' [unsupported-operands] No attribute '__and__' on None or '__rand__' on int Called from (traceback): line 290, in reset_state Differential Revision: https://phab.mercurial-scm.org/D11235

tests: allow Google's internal builds of clang-format to be used These builds do not actually include any Google-specific formatting changes; the only reason they don't include the LLVM version number is due to a decision to elide the version number from *all* LLVM/clang projects. For most builds of clang-format, even "unofficial" ones, the LLVM version will be displayed; example: ``` clang-format version 14.0.0 (https://github.com/llvm/llvm-project.git 1830ec94ac022ae0b6d6876fc2251e6b91e5931e) ``` The Google-internal build looks like this: ``` clang-format version google3-trunk (1830ec94ac022ae0b6d6876fc2251e6b91e5931e) ``` Differential Revision: https://phab.mercurial-scm.org/D10538

Added signature for changeset 6ee0244fc1cf

Added tag 5.9.3 for changeset 6ee0244fc1cf

relnotes: update release notes for upcoming 5.9.3 Differential Revision: https://phab.mercurial-scm.org/D11720

heptapod-ci: actually give pytest more time before timeout `HGTEST_TIMEOUT` is overridden by `HGTEST_SLOWTIMEOUT` for tests marked as slow, which `test-check-pytype.t` is. So this whole time the timeout was 1500s (or 25 minutes), which is unfortunately not long enough for a *lot* of the times it's run on the CI. Differential Revision: https://phab.mercurial-scm.org/D11717

tests: better determinism in test-chg.t chg tests fail pretty often with "Sample count: *" line disappearing. It disappears because the sample count is zero, in which case a custom message is printed. This commit makes the test succeed in that case. Differential Revision: https://phab.mercurial-scm.org/D11716

python: compatibility for python 3.11 (issue6604) The `unittest._TextTestResult` alias has been removed. The "new" name has been available since 3.2, and we only support 3.5.3+. Differential Revision: https://phab.mercurial-scm.org/D11690

pyoxidizer: force pip to not use pep517 in order to be able to install hg Mercurial is not (yet) a pep517 package, but the presence of a pyproject.toml file tells newer-ish versions of pip that it should be one. This is related to 58fe6d127a01, and fixes pyoxidizer builds for the Heptapod CI. Differential Revision: https://phab.mercurial-scm.org/D11710

merge-halt: fix issue with merge.on-failure=halt breaking unshelve Differential Revision: https://phab.mercurial-scm.org/D11706

merge-halt: demonstrate unshelve issue with merge.on-failure=halt Differential Revision: https://phab.mercurial-scm.org/D11705

rust-nodemap: backed out mitigation for issue 6554 This is a backout of changeset 3fffb48539ee. Issue 6554 is now considered solved, hence its mitigation has to be removed, if only for its performance cost. Differential Revision: https://phab.mercurial-scm.org/D11703

rewrite: fix issue6599 Explanation inside. Differential Revision: https://phab.mercurial-scm.org/D11685

Added signature for changeset 750920b18aaa

Added tag 5.9.2 for changeset 750920b18aaa

windows-ci: temporarily allow Windows jobs to fail We have unfortunately ran out of free credit on the runners we were using from OVH for the Windows CI. We will be disabling the two remaining ones on the 30th of September, hence we need the CI to pass even if Windows jobs cannot start as a temporary measure. Hopefully we can find another way of getting Windows runners soon. Differential Revision: https://phab.mercurial-scm.org/D11499

manifestlog: also monitor `00manifest.n` when applicable This let the locarepo's file cache detect outdated nodemap docket and reload the manifestlog after `localrepo.invalidate` when applicable. The same problem than issue6554 could affect the Manifest too. Differential Revision: https://phab.mercurial-scm.org/D11483

changelog: also monitor `00changelog.n` when applicable (issue6554) This let the locarepo's file cache detect outdated nodemap docket and reload the changelog after `localrepo.invalidate` when applicable. Differential Revision: https://phab.mercurial-scm.org/D11482

persistent-nodemap: introduce a test to highlight possible race Weakness in the current file caching of the changelog means that a writer can end up using an outdated docket. This might result in "committed" persistent-nodemap data from a previous writer to be overwritten by a later writer. This break the strong "append only" assumption of the persistent nodemap and can result in confused reader. The race windows are quite narrow. See the test documentation for details. The issues is fixed in the next changeset. Differential Revision: https://phab.mercurial-scm.org/D11481

test: enable share-safe in test-persistent-nodemap We will use some share in this test and I would rather have them safe. Differential Revision: https://phab.mercurial-scm.org/D11480

filecache: abstract the fetching of the list of tracked file We will need it for a coming fix that will requires to check a variable list of file for the changelog. Differential Revision: https://phab.mercurial-scm.org/D11479

persistent-nodemap: use quiet upgrade in tests This make them less verbose while keeping the important information in the checked output. Differential Revision: https://phab.mercurial-scm.org/D11478

persistent-nodemap: fix a typo in a test comment oops. Differential Revision: https://phab.mercurial-scm.org/D11477

patchbomb: fix traceback on outgoing when not specifying revisions A user reported that `hg email -n -o` raised with the following traceback: ``` ** unknown exception encountered, please report by visiting ** https://mercurial-scm.org/wiki/BugTracker ** Python 3.8.12 (default, Sep 13 2021, 22:12:54) [GCC 7.5.0] ** Mercurial Distributed SCM (version 5.9.1) ** Extensions loaded: patchbomb Traceback (most recent call last): File "/usr/pkg/bin/hg", line 61, in <module> dispatch.run() File "/usr/pkg/lib/python3.8/site-packages/mercurial/dispatch.py", line 144, in run status = dispatch(req) File "/usr/pkg/lib/python3.8/site-packages/mercurial/dispatch.py", line 250, in dispatch status = _rundispatch(req) File "/usr/pkg/lib/python3.8/site-packages/mercurial/dispatch.py", line 294, in _rundispatch ret = _runcatch(req) or 0 File "/usr/pkg/lib/python3.8/site-packages/mercurial/dispatch.py", line 470, in _runcatch return _callcatch(ui, _runcatchfunc) File "/usr/pkg/lib/python3.8/site-packages/mercurial/dispatch.py", line 480, in _callcatch return scmutil.callcatch(ui, func) File "/usr/pkg/lib/python3.8/site-packages/mercurial/scmutil.py", line 153, in callcatch return func() File "/usr/pkg/lib/python3.8/site-packages/mercurial/dispatch.py", line 460, in _runcatchfunc return _dispatch(req) File "/usr/pkg/lib/python3.8/site-packages/mercurial/dispatch.py", line 1273, in _dispatch return runcommand( File "/usr/pkg/lib/python3.8/site-packages/mercurial/dispatch.py", line 918, in runcommand ret = _runcommand(ui, options, cmd, d) File "/usr/pkg/lib/python3.8/site-packages/mercurial/dispatch.py", line 1285, in _runcommand return cmdfunc() File "/usr/pkg/lib/python3.8/site-packages/mercurial/dispatch.py", line 1271, in <lambda> d = lambda: util.checksignature(func)(ui, *args, **strcmdopt) File "/usr/pkg/lib/python3.8/site-packages/mercurial/util.py", line 1886, in check return func(*args, **kwargs) File "/usr/pkg/lib/python3.8/site-packages/hgext/patchbomb.py", line 817, in email revs = _getoutgoing(repo, dest, revs) File "/usr/pkg/lib/python3.8/site-packages/hgext/patchbomb.py", line 537, in _getoutgoing safe_paths = [urlutil.hidepassword(p.rawloc) for p in paths] File "/usr/pkg/lib/python3.8/site-packages/hgext/patchbomb.py", line 537, in <listcomp> safe_paths = [urlutil.hidepassword(p.rawloc) for p in paths] File "/usr/pkg/lib/python3.8/site-packages/mercurial/utils/urlutil.py", line 501, in get_push_paths msg %= dest TypeError: %b requires a bytes-like object, or an object that implements __bytes__, not 'NoneType' ``` This is due to patchbomb passing `[None]` in the `dests` parameter of the multi-paths outgoing function, we fix this in this change and add a non-regression test. Differential Revision: https://phab.mercurial-scm.org/D11456

pep-517: remove the `build-backend` key to allow for users to build extensions PEP 517 changed how projects should define their build dependencies. The presence of a `pyproject.toml` file changes the behavior of `pip` to conform to PEP 517. Since we haven't updated the `setup.py` file yet (I'm not even sure we're not an edge case which would make it harder/impossible, I have not tried yet), this is a workaround to issue6589: it allows users to pass `--no-use-pep517` to pip to revert to the legacy installer. The build backend is simply assumed to be the `build_meta:__legacy__` one. Differential Revision: https://phab.mercurial-scm.org/D11393

tests: make removeemptydirs more portable The behavior of pwd change from one system (i.e. default shell/sh) from one system to another so we use fuzzy matching for the output we do not care and we use narrower command call to highlight what we are trying to test here. Differential Revision: https://phab.mercurial-scm.org/D11449

check-code: make it possible to ignore the PWD check in some situation The check-code script does not motivate the banning of $PWD so I am not sure what it is about. And I will needs to use the env variable in the next tests. So I am adding a way to make an exception. Differential Revision: https://phab.mercurial-scm.org/D11450

relnotes: update next Differential Revision: https://phab.mercurial-scm.org/D11448

relnotes: add release notes for 5.9 Differential Revision: https://phab.mercurial-scm.org/D11447

narrow: fix commits of empty files The problem is that when committing a new file with empty contents (or in general empty file with filelog p1 = -1), hg commit with narrow doesn't create a filelog revision at all, which causes failures in further commands. The problem seems to be that: - hg thinks that instead of creating a new filelog revision, it can use the filelog's p1 (the nullrev) - because it thinks the file contents is the same in that revision and in p1 - because `narrowfilelog.cmp(nullrev, b'')` is True (unlike with `filelog.cmp`) It's not clear to me which `cmp` behaves better. But I think it makes sense to change the commit code to not to "reuse" the null rev when adding an empty file with filelog p1 == filelog p2 == -1. This is consistent with never writing the null rev in the manifest, which `hg verify` claims is an invariant: ``` inside/c@4: manifest refers to unknown revision 000000000000 ``` Differential Revision: https://phab.mercurial-scm.org/D11400

narrow: show repo corruption when commiting empty files Differential Revision: https://phab.mercurial-scm.org/D11399

filemerge: be more strict when detecting conflict markers, add `|` markers I received a user complaint about detecting a line that contained 78 `=` followed by `*/` as a conflict marker. We'll never generate that, we generate 7 identical characters and either the end of the line, or a space. Let's be explicit about detecting exactly what we produce to reduce the chances of a false positive. While we're here, add `|||||||` as a detected conflict marker (generated with the `keep-merge3` style conflicts). Differential Revision: https://phab.mercurial-scm.org/D11391

Added signature for changeset 86a60679cf61

Added tag 5.9.1 for changeset 86a60679cf61

fix: again allow formatting the working copy while merging I forgot about unfinished merges (I think I was thinking only about unfinished merge conflicts) when I wrote https://phab.mercurial-scm.org/D11209. As a coworker (hg contributor dploch) reported to me, this led to `hg fix --working-dir` failing when you have an uncommitted merge. The fix is trivial: just move the assertion to just before the call to `scmutil.movedirstate()` where it actually matters. Differential Revision: https://phab.mercurial-scm.org/D11376

pyoxidizer: add arch to PyOxidizer MSIs This is the proper filename pattern. This filename format wasn't properly implemented when PyOxidizer took over Python 3 MSI generation in 603efb3845ba. Differential Revision: https://phab.mercurial-scm.org/D11359

pyoxidizer: pass arch to WiXInstaller() We just upgraded Windows automation to PyOxidizer 0.17.0. This version of PyOxidizer changed the behavior of WiX installers so installer architecture defaulted to "x64" (before it defaulted to the target of the pyoxidizer.exe binary). To allow controlling the architecture of the installer, the `arch` argument was added to `WiXInstaller`. This commit passes that argument in. Differential Revision: https://phab.mercurial-scm.org/D11358

contrib: install PyOxidizer 0.17.0 This pulls in some changes we want to improve Windows MSI installers. This will need separate enhancements to the pyoxidizer.bzl. But those will be handled in a separate changeset. Differential Revision: https://phab.mercurial-scm.org/D11357

packaging: reference proper output directory 9438e9b7321a changed the name of the PyOxidizer target, which changed the name of the output directory. The code changed by this patch wasn't properly updated by that changeset. This resulted in a run-time failure due to trying to read from a non-existent directory. This change should fix the building of Python 3 Inno installers. Differential Revision: https://phab.mercurial-scm.org/D11356

packaging: pass extra_pyoxidizer_vars only to pyoxidizer Before, we would attempt to call a function (build_installer_py2exe) that didn't accept this keyword argument. This was preventing the building of py2exe installers. Differential Revision: https://phab.mercurial-scm.org/D11355

hg: don't attempt to extend `sys.path` with the user site without `APPDATA` This variable is created by the system and *should* be available, but test-lfs-bundle.t has a test where it is explicitly unset. It wasn't caught before because prior to 95af358fcdfe, it was limited to the py2exe binary. As a precaution, fix both that and the pyoxidizer case that was causing the test to fail. Differential Revision: https://phab.mercurial-scm.org/D11354

ci: only run the phabricator step if the previous on succeeded It seems like f6879956a386 regressed the intended behavior. Differential Revision: https://phab.mercurial-scm.org/D11342

clone: properly create target directories during local clone (issue6581) The store encoding was mudding the water. This lead to local clone crashing for file with long filename as their destination directory needed to be encoded. Differential Revision: https://phab.mercurial-scm.org/D11340

clone: add a file demonstrating issue6581 in test-clone-stream.t This is quite noisy so we adds it in its own changeset. Fixes for the issue are coming in the next patch. Differential Revision: https://phab.mercurial-scm.org/D11339

clone: verify the local clone in test-clone-stream.t This will help detecting corruption. Differential Revision: https://phab.mercurial-scm.org/D11338

clone: automatically glob stream clone output in test Touching `test-clone-stream.t` is very painful otherwise. Differential Revision: https://phab.mercurial-scm.org/D11337

clone: fix a comment in test-clone-stream.t Differential Revision: https://phab.mercurial-scm.org/D11336

windows: degrade to py2 behavior when reading a non-symlink as a symlink While waiting for the push to hg-committed in WSL to complete, I ran a `phabimport` from Windows and got this traceback: $ hg phabimport 11313 ** Unknown exception encountered with possibly-broken third-party extension "mercurial_keyring" (version N/A) ** which supports versions unknown of Mercurial. ** Please disable "mercurial_keyring" and try your action again. ** If that fixes the bug please report it to https://foss.heptapod.net/mercurial/mercurial_keyring/issues ** Python 3.9.5 (default, May 6 2021, 17:29:31) [MSC v.1928 64 bit (AMD64)] ** Mercurial Distributed SCM (version 5.9rc1+hg32.0e2f5733563d) ** Extensions loaded: absorb, blackbox, evolve 10.3.3, extdiff, fastannotate, fix, mercurial_keyring, mq, phabblocker 20210126, phabricator, rebase, show, strip, topic 0.22.3 Traceback (most recent call last): File "mercurial.lock", line 279, in _trylock File "mercurial.vfs", line 202, in makelock File "mercurial.util", line 2147, in makelock FileExistsError: [WinError 183] Cannot create a file when that file already exists: b'hp-omen:78348' -> b'C:\\Users\\Matt\\hg/.hg/store/lock' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "<string>", line 24, in <module> File "mercurial.dispatch", line 144, in run File "mercurial.dispatch", line 250, in dispatch File "mercurial.dispatch", line 294, in _rundispatch File "mercurial.dispatch", line 470, in _runcatch File "mercurial.dispatch", line 480, in _callcatch File "mercurial.scmutil", line 153, in callcatch File "mercurial.dispatch", line 460, in _runcatchfunc File "mercurial.dispatch", line 1273, in _dispatch File "mercurial.dispatch", line 918, in runcommand File "mercurial.dispatch", line 1285, in _runcommand File "mercurial.dispatch", line 1271, in <lambda> File "mercurial.util", line 1886, in check File "mercurial.util", line 1886, in check File "hgext.mq", line 4239, in mqcommand File "mercurial.util", line 1886, in check File "mercurial.util", line 1886, in check File "hgext.phabricator", line 314, in inner File "hgext.phabricator", line 2222, in phabimport File "hgext.phabricator", line 2123, in readpatch File "hgext.phabricator", line 2199, in _write File "mercurial.localrepo", line 2956, in lock File "mercurial.localrepo", line 2918, in _lock File "mercurial.lock", line 152, in trylock File "mercurial.lock", line 283, in _trylock File "mercurial.lock", line 314, in _readlock File "mercurial.vfs", line 221, in readlock File "mercurial.util", line 2163, in readlock File "mercurial.windows", line 619, in readlink ValueError: not a symbolic link Both exceptions look accurate (the file exists, and the Windows side can't read WSL side symlinks). I didn't try to reproduce this entirely within the Windows side, but we can do better than a cryptic stacktrace. With this change, the same scenario results in this abort: abort: C:\Users\Matt\hg/.hg/store/lock: The file cannot be accessed by the system When both the `push` and `phabimport` are done on the Windows side, it prints a message about waiting for the lock, and successfully applies the patch after the push completes. I'm not sure if there's enough info to be able to convert the abort into the wait scenario. As it stands now, we don't support symlinks on Windows, which requires either a UAC Administrator level process or an opt-in in developer mode, and there are several places where the new symlink on Windows support in py3 was explicitly disabled in order to get tests to pass quicker. Differential Revision: https://phab.mercurial-scm.org/D11333

pyoxidizer: add user-site to `sys.path` on Windows This is a port of 53221078e0de to Windows to allow pip-installed extensions to be loaded without specifying a path. It's a major headache to have an hg.exe on `PATH` that needs to have the path to the extensions specified, because WSL doesn't see the same path. This is only for Windows for now, to match the currently shipping py2 behavior. There is a better solution with using the `site` package, but this needs support in PyOxidizer[1]. [1] https://github.com/indygreg/PyOxidizer/issues/430 Differential Revision: https://phab.mercurial-scm.org/D11308

zeroconf: fix an issue concatenating bytes and str `bytes(length)` doesn't do what we want, so use `str`. There appear to be a ton more issues in this extension, including: - globals()[b'_GLOBAL_DONE'] using bytes as the key - `__author__` and similar using bytes - `BadDomainName` is feeding bytes to the Exception constructor - DNSRecord.toString() has the wrong signature (should be str, not bytes) Differential Revision: https://phab.mercurial-scm.org/D11303

revlog: fix more type confusion in index_replace_sidedata_info (issue6580) We were telling python that "rev" was a Py_ssize_t (via the "n" format), but it was actually an int. Differential Revision: https://phab.mercurial-scm.org/D11335

revlog: fix type confusion with sidedata_comp_len (issue6580) The format string uses "i" (int) for sidedata_comp_len, so we shouldn't be passing a pointer to Py_ssize_t to PyArg_ParseTuple. On 64-bit big-endian, this would result in python only writing to the upper 32 bits, and things go downhill from there. Differential Revision: https://phab.mercurial-scm.org/D11334

resources: stop important a non existent FileNotFoundError The error was hidden by the wide try/except Differential Revision: https://phab.mercurial-scm.org/D11311

template: FileNotFoundError is actually a built in exception However it is python3 only. Differential Revision: https://phab.mercurial-scm.org/D11310

check-code: drop the camelcase checks Fro about 2 years we have been using CamelCase is class names. That rules gets in the way of assigning class or exception in compatibility layers. I think it is safe to drop it now that we started using CamelCase for some case. My motivation for this is the need to assign `FileNotFoundError` in `pycompat`. Differential Revision: https://phab.mercurial-scm.org/D11309

Added signature for changeset 53221078e0de

Added tag 5.9 for changeset 53221078e0de

pyoxidizer: always run the associated CI jobs Now that all tests are passing, we can enable this by default. Differential Revision: https://phab.mercurial-scm.org/D11306

pyoxidizer: skip for ZeroConf related test for now These test are failing for obscure reason. They are fairly minor and Since they are the only thing between us and enabling pyoxidizer testing in the CI by default, we just shallow the bad output for now. Differential Revision: https://phab.mercurial-scm.org/D11305

pyoxidizer: disable the test for disabled & broken extensions help The test is a bit fragile and the feature is significantly broken for 3rd party extension already. So we disable that section with a comment about why and what is needed. Differential Revision: https://phab.mercurial-scm.org/D11304

pyoxidizer: re-install PYTHONPATH behavior Hooks and extensions can rely on PYTHONPATH value. Yet the pyoxidized binary ignore the variable. So we manually reinstall the behavior to fix various tests and restore the expected behavior. This positively affects: - test-extension.t - test-hook.t Differential Revision: https://phab.mercurial-scm.org/D11302

windows: add pytest-vcr to the dependencies This will lead to pyoxidizer including it in its binary. This fix test-phabricator.t Differential Revision: https://phab.mercurial-scm.org/D11301

pyoxidizer: wrap the pyoxidizer script on multiple line This is clearer and will help with future edition of the script. Differential Revision: https://phab.mercurial-scm.org/D11300

pyoxidized: silence the fuzzywuzzy warning about python-Levenshtein I don't think we need the fast implementation for the test so lets ignore it for now. Differential Revision: https://phab.mercurial-scm.org/D11299

pyoxidized: install fuzzywuzzy too This is necessary for some of the release note tooling. Differential Revision: https://phab.mercurial-scm.org/D11298

pyoxidized: adapt output of test-install.t The python lib and template directories are in different location that requires a different patterns. Differential Revision: https://phab.mercurial-scm.org/D11297

pyoxidized: disable part of `test-install.t` related to pip and virtualenv This part are testing manual installation of Mercurial, this is not relevant for the pyoxidizer variant. Differential Revision: https://phab.mercurial-scm.org/D11296

pyoxidized: adapt output of test-bad-extension.t The file path in the traceback become module name. Differential Revision: https://phab.mercurial-scm.org/D11295

pyoxidized: adapt output of test-flagprocessor.t The file path in the traceback become module name. Differential Revision: https://phab.mercurial-scm.org/D11294

pyoxidized: adapt output of `test-phases.t` We are getting one less line. I am not sure why, but it seems fairly minor. Differential Revision: https://phab.mercurial-scm.org/D11293

pyoxidized: add a copy of the `doc` directory in the right location This get the hacky setup closer to how an actual install is setup. Differential Revision: https://phab.mercurial-scm.org/D11292

pyoxidized: add a copy of the `contrib` directory in the right location This get the hacky setup closer to how an actual install is setup. Differential Revision: https://phab.mercurial-scm.org/D11291

pyoxidized: add a copy of the `defaultrc` file in the right location This get the hacky setup closer to how an actual install is setup. Differential Revision: https://phab.mercurial-scm.org/D11290

pyoxidized: add a copy of the `helptext` directory in the right location This get the hacky setup closer to how an actual install is setup. Differential Revision: https://phab.mercurial-scm.org/D11289

template: handle missing resource in `_readmapfile` This fix the remaining issue in test-template-map.t. Differential Revision: https://phab.mercurial-scm.org/D11288

pyoxidized: add a copy of the `templates` directory in the right location This fix about 10 extra test bring the number of currently failing test to about 12. Differential Revision: https://phab.mercurial-scm.org/D11287

pyoxidized: add a dedicated target in the tests We will need to setup more items to make the "quick" version work on stable, having a dedicated operation will allow to contains change to that operation. Differential Revision: https://phab.mercurial-scm.org/D11286

pyoxidized: adapt test-devel-warnings.t Differential Revision: https://phab.mercurial-scm.org/D11284

test-extension: adapt output to pyoxidizer The traceback use module name instead of filename. The rest of the test is still on fire for various other reasons. However that specific output mismatch is now fixed. Differential Revision: https://phab.mercurial-scm.org/D11285

pyoxidized: add a `pyoxidizer` hghave keyword for line matching Some output line can be affected by pyoxidizer, for example the source file path are replaced by the module name. We introduce a new condition keyword to cope with this. Differential Revision: https://phab.mercurial-scm.org/D11283

templater: swap `\` with `/` in more location This is similar to the previous changeset, but apply that logic to _readmapfile. Differential Revision: https://phab.mercurial-scm.org/D11282

templater: swap `\` with `/` to allow the resource logic to kicks in Without this change our wrapper to the resource logic will fails to details the nesting and try to pass `directory/file` entry to the resource module, leading to a crash. Ideally, we should track down all caller passing `\` instead of `/`, but that is far more intrusive and we would like to have a working pyoxidized version working for 5.9 It will be easy to add a warning to the above code to track down "bad" caller during the 6.0 cycle. This reduce the number of failing test with pyoxidizer from 52 to 23 Differential Revision: https://phab.mercurial-scm.org/D11281

heptapod-ci: extend timeout for pytype checking This test routinely times out for loaded or older machines. Differential Revision: https://phab.mercurial-scm.org/D11258

ci: run --pyoxidized tests on Windows They still have numerous failure, but at least we can start fixing them now. Differential Revision: https://phab.mercurial-scm.org/D11278

run-tests: introduce a --pyoxidized option This options make it possible to use the pyoxidizer version to run the tests. This is a first basic version that is windows only. The test needs a working python, with Mercurial installed. However the pyoxidizer product is "self contains" without a "usable" Python. There have been discussion to have a fully functional `hg admin::python` command providing a fully functional python interpreter, but nothing is of the sort is ready yet. In In the meantime we use an hybrid approach, similar to what we do for testing `rhg`. We install a full "normal" Mercurial, but also the pyxodizer product as the official `hg binary`. That way, we use the pyoxidizer version or everything, but test that needs to run python have it available, with a fully functional Mercurial package. This first version is pretty basic (Windows only, no --local, not --with-pyoxidized), but it runs, various bug that we will have to fix. Differential Revision: https://phab.mercurial-scm.org/D11277

ci: add a "all" template to easily control "when" test run When debugging some job (usually windows one) it is handy to have a way to pass all other jobs to manual execution. We add a "root" template to control just that. Differential Revision: https://phab.mercurial-scm.org/D11276

Added signature for changeset 2813d406b036

Added tag 5.9rc1 for changeset 2813d406b036

issue6528: add a config option to control the fixing on the fly This will allow people who know to be safe to avoid any performance overhead (and other potential issue). Differential Revision: https://phab.mercurial-scm.org/D11271

issue6528: also filter delta on the fly when applying a changegroup This ensure that corrupted clone does not spread corruption to "fixed" version. This might come at a performance cost, we will had a config option to control this behavior in the next changesets. Differential Revision: https://phab.mercurial-scm.org/D11270

filelog: open the writing context a bit earlier in `addgroup` This is a small change made ahead of the next patch for clarification. Differential Revision: https://phab.mercurial-scm.org/D11269

issue6528: implement _is_revision_affected_fast using callback The delta comming from a bundle/stream does not exists in the revlog yet, so we will need other way to retrieve the same information. To prepare for this we split the function to use callbacks in the core logic. Differential Revision: https://phab.mercurial-scm.org/D11268

issue6528: implement _is_revision_affected using callback The delta comming from a bundle/stream does not exists in the revlog yet, so we will need other way to retrieve the same information. To prepare for this we split the function to use callbacks in the core logic. Differential Revision: https://phab.mercurial-scm.org/D11267

debugcommands: add a `--paranoid` option to `debug-repair-issue-6528` See justification inline. Differential Revision: https://phab.mercurial-scm.org/D11263

repair: improve performance of detection of revisions affected by issue6528 Explanations inside the patch. I've tested this on Mozilla-Central and it's 5 times faster than the naive approach on my laptop. Differential Revision: https://phab.mercurial-scm.org/D11262

debugcommands: introduce a debug command to repair repos affected by issue6528 This command is quite basic and slow, it will loop over the entirety of the filelogs in the repository and check each revision for corruption, then fixes the affected filelogs. It takes under 25 minutes for Mozilla-Central on my not-top-of-the-line laptop, using the `--to-report` and `--from-report` options will make this pretty tolerable to use, I think. This change also introduces a test for the fix. Differential Revision: https://phab.mercurial-scm.org/D11239

contrib: switch the Windows bootstrap environment to py3.9 Use the built in `venv` module instead of `virtualenv` for simplicity, and upgrade to a modern Mercurial that supports py3. One issue here is that `venv` doesn't copy `python3{,Y}.dll` into the `Scripts` subdirectory, so running the `hg.exe` that gets installed immediately fails on a clean system because Python isn't in `PATH`. There is code in `python.exe` to detect when it is in a venv and add the original python install to the DLL lookup path, which we don't do in `hg.exe` yet. The simple workaround for now is to run the `hg` script with `python.exe`. Typically `PYTHONLEGACYWINDOWSSTDIO` must be set in the environment on Windows, but the clone process works without it. Differential Revision: https://phab.mercurial-scm.org/D11275

contrib: comment out the 64-bit py38 dependency installation on Windows Not sure what is going on here, but it appears to not install py3.8 x64 in the usual `C:\hgdev` directory. The x32 installer works fine. I'm assuming this is a bug in this version of the installer, but didn't look into it too much. Differential Revision: https://phab.mercurial-scm.org/D11274

contrib: drop python2.7 from the Windows dependency install script While we aren't quite ready to drop py27 yet, the MS compiler is no longer available from MS (there is a copy on github if it's really needed), and that causes the script to fail. Differential Revision: https://phab.mercurial-scm.org/D11273

contrib: log the command and args for every process installing windows deps This is a little noisier, but makes it simple to debug when things fail. Differential Revision: https://phab.mercurial-scm.org/D11272

windows-ci: run Windows CI automatically alongside the others This will enable us to make Windows Python 3 a first-class citizen for the next 6.0 cycle. We will probably get some flaky tests and we're missing others that are skipped, but we'll turn them on it future patches. Differential Revision: https://phab.mercurial-scm.org/D11256

vfs: always use / as file separator (issue6546) Various part of vfs already enforce `/` usage and using `\` confuse the encoded vfs. So we simply use `/` all the time. Differential Revision: https://phab.mercurial-scm.org/D11260

subrepo: compare normalised vfs path Otherwise the realpath call can turn `/` into `\` on windows confusing the check. (We probably needs this in more location) Differential Revision: https://phab.mercurial-scm.org/D11259

pager: account for flakiness in Windows output This test case is cursed and probably not worth losing more time over. This makes apparent what the intended behavior is while still removing the flakiness from the CI. Differential Revision: https://phab.mercurial-scm.org/D11257

windows-ci: clean up the Heptapod CI file now that the baseline is solid Enough work has been done one the CI side, this now works with little effort on our side. The next patch will remove the manual switch. Differential Revision: https://phab.mercurial-scm.org/D11254

test-nointerrupt: make "sure" the handler "might" trigger (issue6558) We are sure that the signal got sent in the right time frame, however, we still have race, so either the code is actually buggy or we need some security to make sure the signal get processed. We might be affected by https://bugs.python.org/issue43406 ? Differential Revision: https://phab.mercurial-scm.org/D11251

testing: make sure write_file is "atomic" This make sure viewer cannot see the new file with partial content. This was likely the cause of some flakiness in `test-nointerrupt.t` Differential Revision: https://phab.mercurial-scm.org/D11250

test: disable test-subrepo-git.t in python2 + chg I am a couple of days in try to debug that at it seems minor enough with enough other priority to simply disable it for now. Differential Revision: https://phab.mercurial-scm.org/D11253

check-module-imports: ignore non-stdlib module installed by distribution Previously, the check script would detect breezy as part of the stdlib if installed using the debian package manager. This silence the following complains: tests/test-convert-bzr.t:117: imports not lexically sorted: breezy.bzr.bzrdir < sys tests/test-convert-bzr.t:117: stdlib import "breezy.bzr.bzrdir" follows local import: breezy tests/test-convert-bzr-ghosts.t:7: imports not lexically sorted: breezy.bzr.bzrdir < sys tests/test-convert-bzr-ghosts.t:7: stdlib import "breezy.bzr.bzrdir" follows local import: breezy tests/test-convert-bzr-treeroot.t:7: imports not lexically sorted: breezy.bzr.bzrdir < sys tests/test-convert-bzr-treeroot.t:7: stdlib import "breezy.bzr.bzrdir" follows local import: breezy Differential Revision: https://phab.mercurial-scm.org/D11249

remotefilelog: fix what looks like a wrong refactoring when various store functions started returning a revlog type as the first element of the tuple. Differential Revision: https://phab.mercurial-scm.org/D11243

rust-nodemap: falling back to C impl as mitigation This is a mitigation for https://bz.mercurial-scm.org/show_bug.cgi?id=6554 We see sometimes almost all data except the most recent revisions removed from the persistent nodemap, but we don't know how to reproduce yet. This has sadly repercussions beyond just needing to reconstruct the persistent nodemap: for instance, this automatically filters out all bookmarks pointing to revisions that the nodemap cannot resolve. If such filtering happens in a transaction, the update of the bookmarks file that happens at the end of transaction loses all bookmarks that have been affected. There may be similar consequences for other data. So this is a data loss, something that we have to prevent as soon as possible. As a mitigation measure, we will now fallback to the C implementation in case nodemap lookups failed. This will add some latency, e.g., in discovery, yet less than disabling the persistent nodemap entirely. We considered implementing the fallback directly on the Python side, but `revlog.get_rev()` is not systematically used, there are also several direct calls to the index method (`self.index.rev()` for a `revlog` instance). It is therefore more direct to implement the mitigation in the rust-cpython wrapper. Differential Revision: https://phab.mercurial-scm.org/D11238

obsolete: disable other evolution config options if createmarkers is off We used to raise an abort in this case, but recent changes to local clone command (377d8fc20e34) resulted in destrepo both caring about experimental.evolution config options and not initializing extensions. So imagine if you had evolve and allowdivergence enabled in your ~/.hgrc. Local clone stopped working after 377d8fc20e34 because evolve sets experimental.evolution=all, but only on srcrepo, for destrepo the extension is not initialized. It's possible to make local cloning work by initializing extensions for destrepo in some cases, but in other cases (e.g. allowdivergence in ~/.hgrc, evolve extension in original-repo/.hg/hgrc) it would still fail. In a discussion with Pierre-Yves David it was decided to simply force other evolution options to be false if createmarkers is not enabled. Differential Revision: https://phab.mercurial-scm.org/D11223

fix: use obsolete.isenabled() to check for experimental.allowdivergence Now that obsolete.isenabled() can also check if divergence is allowed, let's use it for consistency. Other experimental.evolution options are already checked via this function. Differential Revision: https://phab.mercurial-scm.org/D11222

rebase: use obsolete.isenabled() to check for experimental.allowdivergence Now that obsolete.isenabled() can also check if divergence is allowed, let's use it for consistency. Other experimental.evolution options are already checked via this function. Differential Revision: https://phab.mercurial-scm.org/D11221

heptapod-ci: enable pytype testing for all pipelines Now that pytype agrees with the codebase, let's run it by default to catch regressions. Differential Revision: https://phab.mercurial-scm.org/D11242

pytype: add assertions to explain revlogv2 invariants to Pytype Differential Revision: https://phab.mercurial-scm.org/D11241

typing: add several assertions to dirstatemap to appease pytype (grafted from default to stable) I think it's been mentioned in IRC that these can't be None in this case. This fixes: File "/mnt/c/Users/Matt/hg/mercurial/dirstatemap.py", line 213, in addfile: unsupported operand type(s) for &: 'None' and 'int' [unsupported-operands] No attribute '__and__' on None or '__rand__' on int Called from (traceback): line 290, in reset_state File "/mnt/c/Users/Matt/hg/mercurial/dirstatemap.py", line 214, in addfile: unsupported operand type(s) for &: 'None' and 'int' [unsupported-operands] No attribute '__and__' on None or '__rand__' on int Called from (traceback): line 290, in reset_state Differential Revision: https://phab.mercurial-scm.org/D11235

windows: avoid a bytes vs unicode crash reading passwords on py2 This broke in 5b3513177f2b. Specifically, after typing in the password on py2, it would crash with: TypeError: putwch() argument 1 must be cannot convert raw buffers, not str

streamclone: ensure the server sends the right amount of data Otherwise, the client would fail with some confusing error. I have seen an error which I think is this, perhaps due to a concurrent revlog split, which streamclones do not handle correctly and would result in a short read of the index of the revlog being split. Differential Revision: https://phab.mercurial-scm.org/D11236

rewriteutil: fix crash when a rewritten message references f{6,64} Without this, the rewriteutil logic thinks it's found a reference to the wdir pseudo-revision, and then tries to look it up and rewrite it. Stop it from doing that. Amusingly, I had trouble working with this changeset when I didn't describe the defect above using a regular expression, because it would trigger the bug in my installed version of hg. Differential Revision: https://phab.mercurial-scm.org/D11232

tests: add explicit coverage for update_hash_refs from rewriteutil I couldn't find any evidence this is covered by tests in core (but there's a good chance I missed it). We've seen a cute bug in that code, but first let's just cover the cases that work correctly. Differential Revision: https://phab.mercurial-scm.org/D11231

store: document the decoding discrepancy in store.py This will help future people that might be looking into this. Differential Revision: https://phab.mercurial-scm.org/D11220

clone: add a file with special character while testing uncompressed This will make sure we currently do not have any actual impact from issue6548. Differential Revision: https://phab.mercurial-scm.org/D11219

clone: test local clone in `test-clone-uncompressed.t` too This is not an uncompressed test but needs to be tested in the same kind of constraints regarding special file name. Differential Revision: https://phab.mercurial-scm.org/D11218

test-transaction-safety: relax some of the synchronisation schedule We can have `internal` and `external` runs at the same time, so we unlock `external` a bit sooner. Differential Revision: https://phab.mercurial-scm.org/D11216

test-transaction-safety: document the test schedule This helps reader to understand how the test work and actually helped me to detect a missing synchronisation step. Differential Revision: https://phab.mercurial-scm.org/D11215

rhg: Propagate permission errors when finding a repository The Rust standard library has a `Path::is_dir` method that returns false for any I/O error (such as a permission error), not just "No such file or directory". Instead add an `is_dir` function that returns false for non-directories and for "No such file or directory" errors, but propagates other I/O errors. Differential Revision: https://phab.mercurial-scm.org/D11230

rhg: Add build and config instructions to the README file This adds documentation explaining how to compile, configure, and use rhg as well as how the fallback mechanism works. Differential Revision: https://phab.mercurial-scm.org/D11229

dirstate: use `add` in the deprecation message about `add` The previous message was talking about `remove`. Differential Revision: https://phab.mercurial-scm.org/D11224

help: correct config.profiling.freq name (frequency->freq) Differential Revision: https://phab.mercurial-scm.org/D11217

run-tests: use a small timeout for chg instance There is case where the test runner fails to clean up the temporary files in that case, spawned chg instance can stay around for 1 hours. Getting them to shut down sooner cannot hurt. (We should also use a more robust approach for this cleanup) Differential Revision: https://phab.mercurial-scm.org/D11227

run-tests: do not inherit file descriptor when running a command This is one of the difference between python2 and python3 and could have been a reason why test hang with python2 + chg. This does not seems to help the hanging issue at all… However, now that this is written lets reduce the difference between python2 and python3. Differential Revision: https://phab.mercurial-scm.org/D11226

dummyssh: make sure we don't inherit files descriptor to the children This is one of the difference between python2 and python3 and could have been a reason why test hang with python2 + chg. This does not seems to help the hanging issue at all… However, now that this is written lets reduce the difference between python2 and python3. Differential Revision: https://phab.mercurial-scm.org/D11225

tests: make test-serve works on system that allow user to bind low port It is possible to encounter linux system configured in a way that allow port < 1024 to be bound by any users. So we update a test to focus more on the actual test: "resolving `daytime` to an actual port number. The new auto-scaling CI runner provided by clever cloud is doing so. This is the initial motivation for this change, but it seems useful anyway. Differential Revision: https://phab.mercurial-scm.org/D11228

git: restore basic functionality (issue6545) It looks like a big refactor happened on dirstate, and the git extension was just ignored. Differential Revision: https://phab.mercurial-scm.org/D11234

dirstate: fix typo in docstring Spotted while repairing git extension tests. Differential Revision: https://phab.mercurial-scm.org/D11233

help: indicate how to run downgrade a repository using persistent nodemap We have a couple of report of people needing this. I am not fan of pointing people to a debug command, but we don't have much better to offer right now Differential Revision: https://phab.mercurial-scm.org/D11214

help: use the correct spelling for `slow-path` in persistent nodemap help Differential Revision: https://phab.mercurial-scm.org/D11213

fix: use scmutil.movedirstate() instead of reimplementing I wrote this patch 2 years ago as a little cleanup. I wanted to generally used `scmutil.movedirstate()` instead of manually updating the dirstate because that is easy to get wrong. I didn't know until today that the current code had a bug. So I added the test case two patches before this one and dusted off this one patch. This is a little slower than the previous code, as it diffs two manifests. However, it fixes the bug and I don't think it's going to be noticeably slower anyway. Differential Revision: https://phab.mercurial-scm.org/D11210

fix: rewrite writeworkingdir() to explicitly not work with merges `hg fix` errors out early if there is an unfinished merge, so we should have only one parent here. Making that explicit makes my next patches simpler. Differential Revision: https://phab.mercurial-scm.org/D11209

tests: demonstrate bug in `hg fix` with incorrectly dirty working copy If the parent commit needs formatting but the working copy already has the correct formatting, then the working copy will be reported as modified even though it's clean (because the size in the dirstate is incorrect). Because the bug only occurs when the size changes, I modified the formatter used in the test case to remove repeated spaces. Differential Revision: https://phab.mercurial-scm.org/D11208

upgrade: byteify an i18n message Differential Revision: https://phab.mercurial-scm.org/D11207

typing: convert an annotation to an assertion in commands.py Pytype was happy with the annotation at one point, but 2021.04.15 now complains. Differential Revision: https://phab.mercurial-scm.org/D11206

test: remove `sleep` usage in `test-nointerrupt.t` (issue6271) We move from sleep based synchronisation to file creation based synchronisation. Sleeps is the path to the dark side. Sleeps leads to flakiness. Flakiness leads to anger. Anger leads to hate. Hate leads to suffering. Differential Revision: https://phab.mercurial-scm.org/D11205

Added signature for changeset d7515d29761d

Added tag 5.9rc0 for changeset d7515d29761d

branching: merge default into stable This mark the start of the 5.9 freeze.

relnotes: document the change in the dirstate API Differential Revision: https://phab.mercurial-scm.org/D11200

dirstate: deprecate the `add` method All users have been migrated. Differential Revision: https://phab.mercurial-scm.org/D11199

mq: replace `add` call with newer API Differential Revision: https://phab.mercurial-scm.org/D11198

largefile: use `update_file` instead of `add` in `synclfdirstate` This is the newer, more semantic API. Differential Revision: https://phab.mercurial-scm.org/D11197

largefile: use `update_file` instead of `add` in `mergerecordupdates` This is the newer, more semantic API. Differential Revision: https://phab.mercurial-scm.org/D11196

dirstate: deprecate the `drop` method All users have been migrated. Differential Revision: https://phab.mercurial-scm.org/D11195

narrow: stop using `drop` in the `updateworkingcopy` Let us use the new API instead. Differential Revision: https://phab.mercurial-scm.org/D11194

mq: replace `drop` call with newer API Differential Revision: https://phab.mercurial-scm.org/D11193

largefile: use `update_file` instead of `drop` during `rollback` This is the newer, more semantic API. Differential Revision: https://phab.mercurial-scm.org/D11192

largefile: use `update_file` instead of `drop` in `synclfdirstate` This is the newer, more semantic API. Differential Revision: https://phab.mercurial-scm.org/D11191

sparse: use `update_file` instead of `drop` This is the newer, more semantic API. Differential Revision: https://phab.mercurial-scm.org/D11190

sparse: use `update_file` instead of `drop` in `refreshwdir` This is the newer, more semantic API. Differential Revision: https://phab.mercurial-scm.org/D11189

amend: use `update_file` instead of `drop` This is the newer, more semantic API. Differential Revision: https://phab.mercurial-scm.org/D11188

context: use `update_file` instead of `drop` in `markcommitted` This is the newer, more semantic API. Differential Revision: https://phab.mercurial-scm.org/D11187

dirstate: deprecate the `merge` method in all cases All code have been migrated to the new APIs. Differential Revision: https://phab.mercurial-scm.org/D11186

mq: drop the use of `dirstate.merged` Since mq does not work on merge, we do not need this. Differential Revision: https://phab.mercurial-scm.org/D11185

dirstate: deprecate the `otherparent` method in all cases All code have been migrated to the new APIs. Differential Revision: https://phab.mercurial-scm.org/D11184

dirstate: deprecate the `normallookup` method in all cases All code have been migrated to the new APIs. Differential Revision: https://phab.mercurial-scm.org/D11183

narrow: stop using `normallookup` during a test Differential Revision: https://phab.mercurial-scm.org/D11182

narrow: use `update_file` instead of `normallookup` after update This is the newer, more semantic API. Differential Revision: https://phab.mercurial-scm.org/D11181

test: use the internal `_normallookup` in `test-rebuildstate.t` This is a low level test doing low level manipulation. Differential Revision: https://phab.mercurial-scm.org/D11180

sparse: use `update_file` instead of `normallookup` in refreshwdir This is the newer, more semantic API. Differential Revision: https://phab.mercurial-scm.org/D11179

keyword: use `update_file_p1` instead of `normallookup` This is the newer, more semantic API. Differential Revision: https://phab.mercurial-scm.org/D11178

fix: use `set_possibly_dirty` instead of `normallookup` This is the newer, more semantic API. Differential Revision: https://phab.mercurial-scm.org/D11177

eol: use `set_possibly_dirty` instead of `normallookup` This is the newer, more semantic API. Differential Revision: https://phab.mercurial-scm.org/D11176

mq: replace `normallookup` call with newer API This align MQ behavior with what what `scmutil.movedirstate` does. (it should probably use `scmutil.movedirstate` itself. Differential Revision: https://phab.mercurial-scm.org/D11175

revert: use `set_possibly_dirty` instead of `normallookup` This is the newer, more semantic API. Differential Revision: https://phab.mercurial-scm.org/D11174

dirstate: enforce `possibly_dirty` in `set_tracked` There is some usecase (like `hg revert`) that requires more agressive `possibly_dirty` setting. So we take a safe path and enforce it in one of the main `normallookup` replacement. Differential Revision: https://phab.mercurial-scm.org/D11173

record: use `update_file` instead of `normallookup` We are cheating a bit to enforce we are in a pending change context. This will have to be cleaned up later. The end goal is reached, we are now using the newer, more semantic API. Differential Revision: https://phab.mercurial-scm.org/D11172

copy: use `set_tracked` instead of `normallookup` in `dirstatecopy` This is the newer, more semantic API. Differential Revision: https://phab.mercurial-scm.org/D11171

amend: use `update_file` instead of `normallookup` This is the newer, more semantic API. Differential Revision: https://phab.mercurial-scm.org/D11170

mergestate: use `update_file` to handle for `ACTION_EXEC` This is the newer, more semantic API. Differential Revision: https://phab.mercurial-scm.org/D11169

mergestate: use `update_file` to handle for `ACTION_ADD_MODIFIED` This is the newer, more semantic API. Differential Revision: https://phab.mercurial-scm.org/D11168

largefile: use `update_file` for `synclfdirstate` "m" case This is the newer, more semantic API. Differential Revision: https://phab.mercurial-scm.org/D11167

largefile: use `update_file` for `synclfdirstate` "n" case This is the newer, more semantic API. Differential Revision: https://phab.mercurial-scm.org/D11166

largefile: replace the `normallookup` call when creating a new lfdirstate This is the newer, more semantic API. Differential Revision: https://phab.mercurial-scm.org/D11165

largefile: use `update_file` instead of `normallookup` in `updatelfiles` This is the newer, more semantic API. Differential Revision: https://phab.mercurial-scm.org/D11164

largefile: use `set_possibly_dirty` instead of `normallookup` Differential Revision: https://phab.mercurial-scm.org/D11163

dirstate: add a `set_possibly_dirty` method This method will be able to replace some usage of `normallookup` in the future. Differential Revision: https://phab.mercurial-scm.org/D11162

dirstate: deprecate the `normal` method in all cases All code have been migrated to the new APIs. Differential Revision: https://phab.mercurial-scm.org/D11161

test: use `set_tracked` in `test-context` This is apparently the last user of the old API. Differential Revision: https://phab.mercurial-scm.org/D11160

mq: replace usage of `normal` with newer API Differential Revision: https://phab.mercurial-scm.org/D11159

largefile: use `update_file` instead of `normal` in `synclfdirstate` This is the newer, more semantic API. This was the last call to `normal` in largefile. Differential Revision: https://phab.mercurial-scm.org/D11158

largefile: use `set_clean` instead of `normal` in `repo.status` This is the newer, more semantic API. Differential Revision: https://phab.mercurial-scm.org/D11157

largefile: use `set_clean` instead of `normal` in `lfdirstatestatus` This is the newer, more semantic API. Differential Revision: https://phab.mercurial-scm.org/D11156

largefile: use `update_file` instead of `normal` in `updatelfiles` This is the newer, more semantic API. Differential Revision: https://phab.mercurial-scm.org/D11155

largefile: respect the `normal` signature The `parentfiledata` parameter was missing, we throw it away because I am not sure what to do with it in the largefile case. Differential Revision: https://phab.mercurial-scm.org/D11154

keyword: use `set_clean` instead of `normal` This is the newer, more semantic API. Differential Revision: https://phab.mercurial-scm.org/D11153

sparse: use `update_file` instead of `normal` in `refreshwdir` This is the newer, more semantic API. Differential Revision: https://phab.mercurial-scm.org/D11152

sparse: use `update_file` instead of `normal` during `applyupdates` This is the newer, more semantic API. (more content end up "nonnormal", see previous commit) Differential Revision: https://phab.mercurial-scm.org/D11151

sparse: apply update with in a `parentchange` context We are changing the reference so we should be within such context manager. This seems make the actual dirstate content a bit flaky with content being sometime ambigous (but maybe it was flaky before?) So I increased the matching. Differential Revision: https://phab.mercurial-scm.org/D11150

revert: use `set_clean` instead of `normal` This is the newer, more semantic API. Differential Revision: https://phab.mercurial-scm.org/D11149

context: use `update_file` instead of `normal` in `markcommitted` This is the newer, more semantic API. Differential Revision: https://phab.mercurial-scm.org/D11148

context: use new API instead of `normal` during fixup post status We use `set_clean` or `update_file` depending of if we are in the middle of an update or not. Differential Revision: https://phab.mercurial-scm.org/D11147

dirstate: introduce a `set_clean` method This will provide a good alternative to `normal` call with a narrower semantic. Differential Revision: https://phab.mercurial-scm.org/D11146

largefile: use `update_file` instead of `normal` This is the newer, more semantic API. Differential Revision: https://phab.mercurial-scm.org/D11145

largefile: adjust the dirstate post update within a `parentchange` context This is doing an update, so this seems more appropriate. No other change have been made, but the next changeset will :-) Differential Revision: https://phab.mercurial-scm.org/D11144

dirstate: deprecate `dirstate.remove` in all cases This will make sure that code will use the newer, more semantic API sooner than later. Differential Revision: https://phab.mercurial-scm.org/D11143

largefile: use `update_file` instead of `remove` in `synclfdirstate` This is the newer, more semantic API. Differential Revision: https://phab.mercurial-scm.org/D11142

largefile: rearrange conditionnal in `synclfdirstate` We can liquidate the special case early, so do we. Differential Revision: https://phab.mercurial-scm.org/D11141

largefile: use `update_file` instead of `remove` in `mergerecordupdates` This is the newer, more semantic API. Differential Revision: https://phab.mercurial-scm.org/D11140

mq: use `update_file_p1` instead of `remove` when adjusting dirstate This is the new API. Differential Revision: https://phab.mercurial-scm.org/D11139

dirstate: properly update `_lastnormaltime` in `update_file_p1` This was previously overlooked. Differential Revision: https://phab.mercurial-scm.org/D11138

dirstate: properly update `_lastnormaltime` in `update_file` This was previously overlooked. Differential Revision: https://phab.mercurial-scm.org/D11137

dirstate: use `reset_state` in `update_file_p1` Going through the same API is more consistent and allow us to push implementation lower down the call stack. Differential Revision: https://phab.mercurial-scm.org/D11136

dirstate: rename `update_file_reference` to `update_file_p1` This is a bit clearer on the intended usage of the function. (but still not great) Differential Revision: https://phab.mercurial-scm.org/D11135

dirstate-map: move most of `dirstate.update_file` logic in the dsmap A new `reset_state` method is introduced to deal with most of that logic. This move things one layer lower, but the ultimate goal is to deal with most of this at the DirstateItem level. This reveal various imperfection with the data passed to update_file by `mergestate.recordupdates`, however this is orthogonal to this patch and should be dealt with at a higher level. Differential Revision: https://phab.mercurial-scm.org/D11134

dirstate: replace `update_parent_file_data` with simpler `update_parent` call This make the dirstate API simpler. Differential Revision: https://phab.mercurial-scm.org/D11133

dirstate: factor out the part retrieve "filedata" out of `normal` We will need them elsewhere. Differential Revision: https://phab.mercurial-scm.org/D11132

dirstate-map: factor out the change to _dirs and _alldirs on removing This logic is complicated enough to deserves its own function. So it now does. This will make it easier to reuse that logic in later changeset. Differential Revision: https://phab.mercurial-scm.org/D11131

dirstate-map: factor out the change to _dirs and _alldirs on dropping This logic is complicated enough to deserves its own function. So it now does. This will make it easier to reuse that logic in later changeset. Differential Revision: https://phab.mercurial-scm.org/D11130

dirstate-map: factor out the change to _dirs and _alldirs on adding This logic is complicated enough to deserves its own function. So it now does. This will make it easier to reuse that logic in later changeset. Differential Revision: https://phab.mercurial-scm.org/D11129

dirstate-item: introduce a `dm_otherparent` property See inline documentation for details. Differential Revision: https://phab.mercurial-scm.org/D11124

dirstate-item: introduce a `dm_nonnormal` property See inline documentation for details. Differential Revision: https://phab.mercurial-scm.org/D11123

dirstate-v2: Reserve a few bytes of space for future extensions See doc-comment Differential Revision: https://phab.mercurial-scm.org/D11101

dirstate-v2: Separate iterators for dirfoldmap and debugdirstate `dirstatemap.dirfoldmap` was recently changed to re-use a Rust iterator that was added for the `hg debugdirstate` command. That iterator was based on all nodes in the tree dirstate without an entry only existing to hold child nodes, and therefore being directories. However to optimize status further we may want to store additional nodes for unknown or ignored files and directories. At that point the two users of this iterator will want different things, so let’s make two iterators instead. See doc-comments in `dispatch.rs`. Differential Revision: https://phab.mercurial-scm.org/D11099

dirstate-v2: Move fixed-size tree metadata into the docket file Before this changeset, the dirstate-v2 data file contained not only nodes and paths that may be reused when appending to an existing file, but also some fixed-size metadata that applies to the entire tree and was added at the end of the data file for every append. This moves that metadata into the docket file, so that repeated "append" operations without meaningful changes don’t actually need to grow any file. Differential Revision: https://phab.mercurial-scm.org/D11098

dirstate-v2: Add heuristic for when to create a new data file … instead of appending to the existing one. This is based on keeping track of how much of the existing data is not used anymore. Differential Revision: https://phab.mercurial-scm.org/D11097

dirstate-v2: Reuse existing paths when appending to a data file When writing a dirstate in v2 format by appending to an existing data file, filenames / paths that are borrowed from the previous on-disk representation can be reused. Differential Revision: https://phab.mercurial-scm.org/D11096

dirstate-v2: Reuse existing nodes when appending to a data file When writing a dirstate in v2 format by appending to an existing data file, nodes that are still "unparsed" from the previous on-disk representation have been unchanged and can therefore be reused. This makes the new data point to previously-existing data for tree nodes. Differential Revision: https://phab.mercurial-scm.org/D11095

dirstate-v2: Support appending to the same data file For now we’re still writing the entire data every time, so appending is not useful yet. Later we’ll have new nodes pointing to some existing data for nodes and paths that haven’t changed. The decision whether to append is pseudo-random in order to make tests exercise both code paths. This will be replaced by a heuristic based on the amount of unused existing data. Differential Revision: https://phab.mercurial-scm.org/D11094

dirstate-v2: shrink on-disk path lengths to 16-bits Differential Revision: https://phab.mercurial-scm.org/D11091

dirstate-v2: Rename Header to Root, move it to the end of the data file Now that they don’t have to be at the start, a given data file may contain multiple "roots". A docket only points to one of them, and previous ones are left unused to allow allow append-only in-place writing to an existing data file. Differential Revision: https://phab.mercurial-scm.org/D11090

dirstate-v2: Enforce data size read from the docket file The data file may not be shorter than its size given by the docket. It may be longer, but additional data is ignored. Differential Revision: https://phab.mercurial-scm.org/D11089

dirstate-v2: Introduce a docket file .hg/dirstate now only contains some metadata to point to a separate data file named .hg/dirstate.{}.d with a random hexadecimal identifier. For now every update creates a new data file and removes the old one, but later we’ll (usually) append to an existing file. Separating into two files allows doing the "write to a temporary file then atomically rename into destination" dance with only a small docket file, without always rewriting a lot of data. Differential Revision: https://phab.mercurial-scm.org/D11088

dirstate: replace a dead conditional branch with an assert in `update_file` This is a case we never meet, so lets trim it away to simplify the code before more changes. Differential Revision: https://phab.mercurial-scm.org/D11128

dirstate-map: do not use `size` to gate copy dropping during remove_file This get us close to moving the block right above withing the DirstateItem object. Doing so will help us getting rid of magic constant at the dirstatemap level. Differential Revision: https://phab.mercurial-scm.org/D11127

dirstate: drop a duplicated assert This very case is checked in 3 line above that one. Differential Revision: https://phab.mercurial-scm.org/D11126

dirstate: drop duplicated check This is covered by the `@requires_parents_change` decorator that this function use. Differential Revision: https://phab.mercurial-scm.org/D11125

dirstate: add a `set_possibly_dirty` in `fakedirstatewritetime` Differential Revision: https://phab.mercurial-scm.org/D11122

dirstatemap: use `set_possibly_dirty` in `clearambiguoustimes` lets put this new method to use Differential Revision: https://phab.mercurial-scm.org/D11121

dirstate-item: use `set_possibly_dirty` in `pure.pack_dirstate` Lets make use of the new function. Differential Revision: https://phab.mercurial-scm.org/D11120

dirstate-item: add a `set_possibly_dirty` method See inline documentation for details. The pushes the AMBIGUOUS_TIME implementation further down the line within the DirstateItem only. When this cleanup is done we will be able to stop using this representation internally. Differential Revision: https://phab.mercurial-scm.org/D11119

dirstate-item: add a `from_v1_data` constructor This class method is dedicated to building a DirstateItem from the data available in the "dirstate-v1" format. Since that format is frozen, this constructor will never change (unlike the `__init__` one). Differential Revision: https://phab.mercurial-scm.org/D11118

dirstate-item: use an explicit __init__ function instead of the attrs one For now, this is not doing anything special, however we plan to make it different in the future. So we start simple. Differential Revision: https://phab.mercurial-scm.org/D11117

dirstate: deprecated `drop` outside of `update/merge` All core users have been migrated. Differential Revision: https://phab.mercurial-scm.org/D11116

dirstate: deprecated `remove` outside of `update/merge` All core users have been migrated. Differential Revision: https://phab.mercurial-scm.org/D11115

dirstate: deprecate the `add` method outside of update/merge context All core users have been updated. Differential Revision: https://phab.mercurial-scm.org/D11114

upgrade: avoid a traceback in case of unrecognized revlog Without this, in case of revlog name not matching any know pattern we would get a traceback. Raising a clear error seems simpler. Differential Revision: https://phab.mercurial-scm.org/D11202

walk: no longer ignore revlogs of files starting with `undo.` (issue6542) Changeset 0b569c75d180 introduced new code in store.walk to filter out undo files left behind by the transaction. However doing so is also filtering out legitimate revlog file starting with `undo.` So this changeset is mostly rolling back that change and adding tests tests to catch this kind of error in the future. As a result we the transaction undo files a considered again by various code (in practice mostly persistent nodemap related). We either live with it (low inconvenient) or explicitly work around it for now. This should be good enough to no longer block the 5.9rc release with this issue. We shall build something cleaner within the 6.0 cycle. Differential Revision: https://phab.mercurial-scm.org/D11201

largefiles: replace use of synclfdirstate with drop Replace calling a complex function with a simple dedicated one Differential Revision: https://phab.mercurial-scm.org/D11113

testing: do not stop waiting if timeout is 0 (issue6541) Differential Revision: https://phab.mercurial-scm.org/D11102

largefile: use `parentchange` markcommitted Differential Revision: https://phab.mercurial-scm.org/D11112

largefile: use `set_untracked` in the override of copy/rename This is the new shiny API Differential Revision: https://phab.mercurial-scm.org/D11111

largefile: use `set_untracked` in the `forget` override This is the new shinny API. Differential Revision: https://phab.mercurial-scm.org/D11110

largefile: directly use set_untracked() for removing files This is new shiny API. Differential Revision: https://phab.mercurial-scm.org/D11109

largefile: use `parentchange` during rollback rollback is updating parent without touching the working copy. It should wrapped in a `parentchange` context. Differential Revision: https://phab.mercurial-scm.org/D11108

largefile: consider `updatelfiles` as a `parentchange` This is not strictly a `parentchange`, however this is still some internal dirstate adjustment as "similar" enough that it seems find to do so. Differential Revision: https://phab.mercurial-scm.org/D11107

largefile: use `set_tracked` in the `hg copy` overwrite This is the new shiny API. Differential Revision: https://phab.mercurial-scm.org/D11106

largefile: use parentchange during mergerecordupdates This match what core is doing around recordupdates. Differential Revision: https://phab.mercurial-scm.org/D11105

largefile: use `set_tracked` in the `hg add` overwrite This is the new shiny API. Differential Revision: https://phab.mercurial-scm.org/D11104

largefile: synchronise the dirstate within a `parentchange` context Semantically these changes are made to adjust the dirstate after a commit, so it should be in a `parentchange` context. Differential Revision: https://phab.mercurial-scm.org/D11103

upgrade_utils: fix crash for removecldeltachain action This fixes issue6383. Differential Revision: https://phab.mercurial-scm.org/D10914

windows: enforce the module policy to be c for test This is the same as what we do with the linux test. Differential Revision: https://phab.mercurial-scm.org/D11100

packaging: add command line flag to add extra vars to pyoxidizer This felt simpler than the previous incarnation of injecting content into the WiX build. I decided the easiest way to pass an arbitrary map into the process was some json - I may regret this, but time will tell on that. Differential Revision: https://phab.mercurial-scm.org/D11093

pyoxidizer: add hooks to inject extra python packages and install files We need this type of hook to inject our internal extension and resource files at Google. Presumably this could be useful to others, so instead of trying to carry an internal patch we've done this in a modular way that should be of value upstream. I'm extremely puzzled by the behavior of glob() on Windows, and I'll be filing at least one (probably two) bugs upstream about it. Differential Revision: https://phab.mercurial-scm.org/D11092

doc: wrap the help text for the purge extension Not sure why `black` or some other checker didn't complain about this. Differential Revision: https://phab.mercurial-scm.org/D11036

dummyssh: call python script with python We don't want windows to grab whatever python got configured to run .py file. We need the python used during the test to be used. Differential Revision: https://phab.mercurial-scm.org/D11083

dummyssh: use subprocess instead of os.call Welcome in 2020 Differential Revision: https://phab.mercurial-scm.org/D11082

run-test: rework the redirection script for python on windows This should get use something something overall simpler and clearer. Especially, we now have a `python.exe` script (even if by default Windows has `C:\Python27\python.exe` hardcoded in the register to open .py file) Differential Revision: https://phab.mercurial-scm.org/D11081

run-test: act that we requires MSYS to run the test on Windows If this ever change we can drop that constrains, however for now it seems clear to be clear about this limitation. Differential Revision: https://phab.mercurial-scm.org/D11080

sigpipe-remote: display more information about the non-py3 python If we are running the wrong things, it is useful to know which wrong thing we are running. Differential Revision: https://phab.mercurial-scm.org/D11079

sigpipe-remote: check for python version earlier in the script The should abort early, using the wrong python is a pretty bad sign. Differential Revision: https://phab.mercurial-scm.org/D11078

check-code: add a rules to catch os.path.abspath All previous usages have been migrated. So let us add a check-code rules to catch future usages. We restrict it to mercurial/ and hgext/ because multiple other script never depends on Mercurial modules. Differential Revision: https://phab.mercurial-scm.org/D11072

windows: use abspath in convert.subversion We replace `os.path.abspath` with `util.abspath`. This should solve more "drive capitalization" issue on Windows. Differential Revision: https://phab.mercurial-scm.org/D11071

windows: use abspath in convert.git We replace `os.path.abspath` with `util.abspath`. This should solve more "drive capitalization" issue on Windows. Differential Revision: https://phab.mercurial-scm.org/D11070

windows: use abspath in convert.bzr We replace `os.path.abspath` with `util.abspath`. This should solve more "drive capitalization" issue on Windows. Differential Revision: https://phab.mercurial-scm.org/D11069

windows: use abspath in the git extension We replace `os.path.abspath` with `util.abspath`. This should solve more "drive capitalization" issue on Windows. Differential Revision: https://phab.mercurial-scm.org/D11068

windows: use abspath in url We replace `os.path.abspath` with `util.abspath`. This should solve more "drive capitalization" issue on Windows. Differential Revision: https://phab.mercurial-scm.org/D11067

windows: use abspath in subrepoutil We replace `os.path.abspath` with `util.abspath`. This should solve more "drive capitalization" issue on Windows. Differential Revision: https://phab.mercurial-scm.org/D11066

windows: use abspath in subrepo We replace `os.path.abspath` with `util.abspath`. This should solve more "drive capitalization" issue on Windows. Differential Revision: https://phab.mercurial-scm.org/D11065

windows: use abspath in hgwebdir We replace `os.path.abspath` with `util.abspath`. This should solve more "drive capitalization" issue on Windows. Differential Revision: https://phab.mercurial-scm.org/D11064

windows: use abspath in mercurial/hg.py We replace `os.path.abspath` with `util.abspath`. This should solve more "drive capitalization" issue on Windows. Differential Revision: https://phab.mercurial-scm.org/D11063

windows: use abspath in extensions We replace `os.path.abspath` with `util.abspath`. This should solve more "drive capitalization" issue on Windows. Differential Revision: https://phab.mercurial-scm.org/D11062

windows: use abspath in commandserver We replace `os.path.abspath` with `util.abspath`. This should solve more "drive capitalization" issue on Windows. Differential Revision: https://phab.mercurial-scm.org/D11061

windows: use abspath in chgserver We replace `os.path.abspath` with `util.abspath`. This should solve more "drive capitalization" issue on Windows. Differential Revision: https://phab.mercurial-scm.org/D11060

windows: introduce a `util.abspath` to replace os.path.abspath This will let us mitigate the drive letter capitalization hell. See inline comment for details. Differential Revision: https://phab.mercurial-scm.org/D11059

windows: enforce upper case drive letter for getcwd in mercurial too This is affecting code that checks if a pull/push destination is the same as a configured one. For example the one creating divergent bookmark. Doing this fixes will help fixing `test-bookmarks.t` and `test-bookflow.t` on Windows. However, we also need to fix `abspath` invocation. Differential Revision: https://phab.mercurial-scm.org/D11058

hg-ssh: normalize the drive of the current working directory on windows For some reason os.getcwd() can return either `c:` or `C:`. We normalize this to `C:` and the like. This fix `test-ssh-bundle1.t` on windows. Differential Revision: https://phab.mercurial-scm.org/D11042

run-tests: enforce the drive letter from `getcwd` to upper case For some reason os.getcwd() can return either `c:` or `C:` depending of which binary you used on Windows. We normalize this to `C:` and the like. This fix `test-run-tests.t` on windows as the drive letter in "$TESTTMP" was "wrongly" set to 'c:/' if the test path wasn't explicitly specified. Differential Revision: https://phab.mercurial-scm.org/D11035

sigpipe-remote: simply delegate pipe forwarding to subprocess we can kill Instead of using sophisticated logics with thread a non blocking pipes, we simply spawn two new process in charge of reading the pipe and sending the result to the client. When it is time to cut the pipe we violently kill them without any remorse. This close the pipe regardless of any in progress `os.read` call. Ironically this is the very same things as what the initial shell setup was doing, but in Python. This makes the test pass run properly on Windows. This also reveal that the Windows behavior is broken as the transaction is not properly rollback. However this is an adventure for another time. Making the test behave properly was enough effort. Differential Revision: https://phab.mercurial-scm.org/D11087

sigpipe-remote: verify the script is IO are unbuffered We don't want to get stuck anywhere by buffers. Differential Revision: https://phab.mercurial-scm.org/D11086

sigpipe-remote: check environment variable sooner Lets make sure we have everything before doing anything. Differential Revision: https://phab.mercurial-scm.org/D11085

test-sigpipe: run the hook using `sh` This make the hook properly run on Windows. Differential Revision: https://phab.mercurial-scm.org/D11084

windows: add windows behavior on broken pager Apparently, Windows has "better" behavior than Unix in this case. This is an edge case that led me down a rabbit hole, only to find a bug in the Python documentation... I am not planning on trying to reproduce the same behavior on Unix systems since it's not really useful, but other people are welcome to! Differential Revision: https://phab.mercurial-scm.org/D11000

mergestate: use dirstate.update_file API in recordupdates() Let's use the new single function to replace multiple different functions. This is part of dirstate API refactor to make it more clearer and generic so that it can be replicated in Rust. Differential Revision: https://phab.mercurial-scm.org/D11077

dirstate: add dedicated function for updating data of a file `dirstate.normal()` is too generic to be a user facing function for that. This is a part of effort to refactor dirstate APIs and make them clearer. Differential Revision: https://phab.mercurial-scm.org/D11076

dirstate: add a `update_file` function This function is the other side of the `set_tracked`/`set_untracked` API revamp. It is to be used when the dirstate is changing its parents during and update or a merge. It states all the information we know about the file so that the dirstate can update its internal data. Unlike the `set_tracked`/`set_untracked` it has not regards for the information previously tracked in the tristate. Differential Revision: https://phab.mercurial-scm.org/D11075

resolve: use the `parentchange` context manager to apply merge action In an ideal world, we would not have to do that. However, we are miles away from being ready to not have to do it. So we add this context manager alongside a long comment. This will help use to get to the point were have two distinct API with strict rules about when to call them. Differential Revision: https://phab.mercurial-scm.org/D11074

sparse: adjust the temporary includes within a `parentchange` context This is related to dirstate adjustment. Differential Revision: https://phab.mercurial-scm.org/D11033

amend: adjust the dirstate within a `parentchange` context The adjustment in the direct consequence of the amend and the associated parents change. Differential Revision: https://phab.mercurial-scm.org/D11032

dirstate: use the right internal API in a test script The goal of the script it to perform arbitrary internal operation to create incorrect state, so lets make it clear. Differential Revision: https://phab.mercurial-scm.org/D11031

sparse: clear rules in the context of a `parentchanges` context This is the same logic as the change we did for narrow. Differential Revision: https://phab.mercurial-scm.org/D11029

sparse: make sure we adjust the dirstate at the same time as the parent This is more correct and help our API split. Differential Revision: https://phab.mercurial-scm.org/D11028

narrow: update narrow spec within a dirstate.parentchange context Even if the parents does not changes, the parents' content we consider is changed. So this seems legitimate. Differential Revision: https://phab.mercurial-scm.org/D11027

revert: use `set_untracked` instead of `drop` when applicable This is the new shiny API. Differential Revision: https://phab.mercurial-scm.org/D11026

revert: use `set_untracked` when performing a revert This is the new shiny API. Differential Revision: https://phab.mercurial-scm.org/D11022

mq: use `set_untracked` in `qrename` This is the new shiny API. Differential Revision: https://phab.mercurial-scm.org/D11021

context: use `dirstate.set_untracked` in `context.forget` This is the new shiny API. Differential Revision: https://phab.mercurial-scm.org/D11020

dirstate: add a `set_untracked` method for "hg remove"-like usage This is a step further toward clarifying the semantic of various dirstate call. See the justification for adding `set_tracked` for details. Differential Revision: https://phab.mercurial-scm.org/D11019

mq: use `set_tracked` in `qrename` This is the new shiny API. Differential Revision: https://phab.mercurial-scm.org/D11018

mq: update the dirstate and its parent within a `parentchange` context This is more correct, and move our plan of separated API for different dirstate usage forward. note: maybe the `parentchange` context manager should replace the dirstateguard entirely ? (in this case we should probably deprecated dirstateguard). Differential Revision: https://phab.mercurial-scm.org/D11017

context: use `dirstate.set_tracked` for `revert` This is the new shiny API. Differential Revision: https://phab.mercurial-scm.org/D11016

context: use `dirstate.set_tracked` in context.copy This is the new shiny API. Differential Revision: https://phab.mercurial-scm.org/D11015

context: use `dirstate.set_tracked` in `context.add` This is the new shiny API. Differential Revision: https://phab.mercurial-scm.org/D11014

dirstate: add a `set_tracked` method for "hg add"-like usage This is a step further toward clarifying the semantic of various dirstate call. Having a dedicated function comes with a couple of benefits: 1) we can move duplicated logic about how to handle the previous state within the dirstate. Since we are sure this is always called in the same situation, we can implement that logic once in the dirstate. 2) having a dedicated method for this case unlock also having a dedicated method for the other case and recording more information at that time. All this leading having more code within the dirstate and higher level API that are less error prone. Differential Revision: https://phab.mercurial-scm.org/D11013

dirstate: add a function to update tracking status while "moving" parents The `scmutil.dirstateparent` is moving the dirstate parent without touching the working copy. It is used by history-rewriting operations like amending of folding. The function was directly doing the "low level" computation and dirstate change. All that logic belong to the dirstate and should be moved there. For this purpose we introduce a new function that does just that and use it. Differential Revision: https://phab.mercurial-scm.org/D11012

dirstate: introduce an internal `_drop` method We want to split current user of `dirstate.drop` between `hg rm`-like cases and update of the dirstate coming from update/merge. To do this we will introduce new API. The first step is to introduces an internal function that these new API migh use (or not use) to distinct between the migrated users and the others. Differential Revision: https://phab.mercurial-scm.org/D11023

dirstate: introduce an internal `_remove` method We want to split current user of `dirstate.remove` between `hg rm`-like cases and update of the dirstate coming from update/merge. To do this we will introduce new API. The first step is to introduces an internal function that these new API migh use (or not use) to distinct between the migrated users and the others. Differential Revision: https://phab.mercurial-scm.org/D11011

dirstate: introduce an internal `_add` method We want to split current user of `dirstate.add` between `hg add`-like cases and update of the dirstate coming from update/merge. To do this we will introduce new API. The first step is to introduces an internal function that these new API migh use (or not use) to distinct between the migrated users and the others. Differential Revision: https://phab.mercurial-scm.org/D11010

run-tests: rely on an actual executable in PATH instead of alias for `hg` The alias approach is poorly inherited by other process that the test might spawn. To solve this we use the same approach as for `python`/`python3` we write an executable file explicitly. Doing this fixes `which hg` invocation that now returns the same location as `hg`. Using chg server side has some minor effect on some stdout/stderr ordering when using `chg` as the server too. Differential Revision: https://phab.mercurial-scm.org/D11053

tests: blacklist a handful of test with `rhg` or `chg` The use of `alias` to enforce `chg` and `rhg` means we are actually using a mix of `rhg`/`chg` and `hg` when calling `hg` in the test. Fixing this breaks various tests. This would be a large detour to fix that. I am disabling them for now with an appropriate comment. We would hopefully get back to them by the 5.8 release. Differential Revision: https://phab.mercurial-scm.org/D11052

run-tests: use more explicit signaling for `chg` Using a dedicated variable is clearer and less fragile. It cannot hurt. Differential Revision: https://phab.mercurial-scm.org/D11051

run-tests: drop the `rhg` flag for `hghave.py` if unset This seems cleaner. Differential Revision: https://phab.mercurial-scm.org/D11050

run-tests: introduce a `HGTEST_REAL_HG` variable for test It turns out that currently, `hg` and `which hg` can point to different things because `hg` is an alias… This is annoying because script and pieces of test are unknowingly using the wrong `hg`. We will fix it in another changeset. However some test actually need to use a real `hg` binary and not some `chg` or `rhg` equivalent. So we introduce a new variable with the right value and we put it to us in the appropriate location. Differential Revision: https://phab.mercurial-scm.org/D11049

run-test: clarify the error with a bad --with-hg is passed This helped me to understand what was going on when I got into trouble. Differential Revision: https://phab.mercurial-scm.org/D11046

windows: make sure we fully read and cleanly close the connection Maybe this will prevent server on Windows to sometimes complains about the client closing the connection too soon. So we make sure we read everything and we officially close the connection. Hopefully Windows will be happier and the test will stop being flaky. Differential Revision: https://phab.mercurial-scm.org/D11073

test-bookmark: clarify the path involved in a pull creating divergence This make the test easier to follow. Especially because this part is currently failing on windows. Differential Revision: https://phab.mercurial-scm.org/D11056

test-removeemptydirs: adjust to Windows behavior for the `histedit` case Windows stay in the "same" directory and the warning message is not applicable. This make `tests/test-removeemptydirs.t` pass on Windows. Differential Revision: https://phab.mercurial-scm.org/D11055

test-removeemptydirs: clarify the state of things in the `histedit` case This make the test easier to follow and will help to compare with the behavior on windows. Differential Revision: https://phab.mercurial-scm.org/D11054

test-infinitepush: drop unused helper They are unused since de4c2f3af97f. We drop them for clarity. Differential Revision: https://phab.mercurial-scm.org/D11048

run-tests: setup "correct" python earlier I don't see any good reason to wait. This result in slightly simpler code. Differential Revision: https://phab.mercurial-scm.org/D11047

run-tests: cleanup windows generation of the python / python3 pointeur They are no need to generate then in RUNTESTDIR and we don't need a python2 executable. (because not test ever use `python2` explicitly). This is probably still not perfect, but at least it is better. Differential Revision: https://phab.mercurial-scm.org/D11045

run-tests: always define a custom-bin directory We are always generating `python` / `python3` executable and we will soon often generate a `hg` executable. So let's always make it a thing. This will help us to make sure we have the appropriate binary available to all script. On both linux and Windows. Differential Revision: https://phab.mercurial-scm.org/D11044

run-tests: explicitly track that `pythondir` was inferred If we want to know that something happened, we should simply track that this things happened. This will be useful to start using `tmpbindir` more in a different form. Differential Revision: https://phab.mercurial-scm.org/D11043

run-tests: use a global WINDOWS constant instead of multiple tests This should make the code clearer. This required the adjustement of a hack in the code testing this code. Differential Revision: https://phab.mercurial-scm.org/D11041

test-run-test: use explicit variables to clarify paths Since $TESTTMP and $TESTDIR are defined for both the main scope and the test run during the test, the situation get confusing. To work around that, we define some new, explicit, variables with the values we needs to keep track of. Differential Revision: https://phab.mercurial-scm.org/D11034

run-tests: also catch double-escapes for $TESTTMP On Windows the `$TESTTMP` string "repr" is different than its value, because `\` become `\\`. We introduce a new `$STR_REPR_TESTTMP` substitution to match that too. Differential Revision: https://phab.mercurial-scm.org/D11001

ci-windows: introduce manual windows CI Note: most of the foundational work of this commit was done by Matt Harbison, but enough has changed that I don't feel comfortable sending this patch as his. In our quest to remove Python 2 from Mercurial, we need to get Windows CI going to check that Python 3 support is up to an acceptable standard. This is the first step: adding a manual step to run a full Windows CI, with a certain definition of "full" since some things are not installed yet, like SVN. Differential Revision: https://phab.mercurial-scm.org/D11025

dirstate-item: deprecate tuple access on the class This should help us to catch and update the last user of this, especially in extensions. People will need to run the test with --pure to actually catch it, but this is better than nothing. Differential Revision: https://phab.mercurial-scm.org/D10992

dirstate-item: use the v1_serialization method in debugstate I am assuming the debug command are looking into the serialized format so I am using the `v1_` variants. This assumption might be wrong. Differential Revision: https://phab.mercurial-scm.org/D10988

dirstate-item: also build DistateItem in dirstate.directories() The rust code was building tuple. Differential Revision: https://phab.mercurial-scm.org/D10983

parsers: move DirstateItem to attr.s This is the followup I requested on D10949 - it was easy, so I just did it. Differential Revision: https://phab.mercurial-scm.org/D11030

cmdutil: fix newandmodified file accounting for --interactive commits `originalchunks` is a misleading name, because it only contains header objects, which are flattened to selected hunks by the filter function. As such, `chunks not in originalchunks` is always True and misleading, because hunk objects never compare equal to header objects. This change fixes the internal naming and removes the useless parameter from the method. This change also fixes issue6533, by considering the filtered headers, rather than the hunks, when determining new and modified files. If a file is renamed + edited, and the edited hunks are deselected (but the file is not), the filtered chunks will contain a header for the file (because it's .special()) but but no hunks. Differential Revision: https://phab.mercurial-scm.org/D10936

test-run-tests: add missing backslash for Windows Differential Revision: https://phab.mercurial-scm.org/D11004

windows: use cpu-intensive task instead of real time in test Windows profiling breaks when using real time instead of cpu, but we need the extension function to show up, so make it do something instead of waiting. I've also duplicated the test cases so that explicit real and CPU times are tested (on platforms other than Windows) Differential Revision: https://phab.mercurial-scm.org/D11003

windows: use shell function instead of variable substitution This makes it compatible with Windows Differential Revision: https://phab.mercurial-scm.org/D11002

compat: normalise path before comparison in revlog splitting test This fixes the test on Window. Differential Revision: https://phab.mercurial-scm.org/D10999

windows: replicate the normalizing behavior of os.environ On Windows, `os.environ` normalizes environment variables to uppercase. Our current bytes-based environ substitution object is a simple dict, so we add the normalization behavior. This fixes test-http-peer.t on Windows. Differential Revision: https://phab.mercurial-scm.org/D10998

encoding: move case-related utils up This will be useful for the next commit that needs this code earlier. Differential Revision: https://phab.mercurial-scm.org/D11024

dirstate: drop the last explicit `state` usage in the module The internal detail of the `state` bytes is not longer exposed to the dirstate layer. Differential Revision: https://phab.mercurial-scm.org/D10977

test: use a python script in `test-transaction-rollback-on-sigpipe.t` This still does not work on Windows, but at least this is a python script now. Differential Revision: https://phab.mercurial-scm.org/D10947

test: make sure we hit the SIGPIPE in test-transaction-rollback-on-sigpipe With a coming change, we might not hit the sig pipe without issue and extra line of output. We do this early to make the next change clearer. Differential Revision: https://phab.mercurial-scm.org/D10946

test: clarify some output in `test-transaction-rollback-on-sigpipe` We don't want a dirty transaction to remain, but need to check the transaction was aborted. Otherwise it is easy to have no abandoned transaction if the transaction succeed. Differential Revision: https://phab.mercurial-scm.org/D10945

tests: add test case for issue 6262 If you split a commit, prune one of the sides, and then try to revive that commit by adding a successor, it will incorrectly be considered divergence. Differential Revision: https://phab.mercurial-scm.org/D10916

rewriteutil: look up common predecessor on unfiltered repo Before this patch, the code looking for divergence could crash when tried to look up a common predecessor in the filtered repo. This patch fixes that by looking up the common predecessor in an unfiltered repo. Differential Revision: https://phab.mercurial-scm.org/D10917

tests: demonstrate crash when common predecessor of divergence is hidden Differential Revision: https://phab.mercurial-scm.org/D11009

shallowutil: dedent code after the previous change Differential Revision: https://phab.mercurial-scm.org/D11008

shallowutil: narrow scope of try/except block This will make this code easier to understand in the future. Differential Revision: https://phab.mercurial-scm.org/D11007

remotefilelog: stop using RuntimeError for control flow We introduce a new exception to handle the various failure categories, rather than relying on RuntimeError. Differential Revision: https://phab.mercurial-scm.org/D11006

remotefilelog: tweak corrupt cache test to grep more flexibly I'm about to fix the abuse of RuntimeError here, which breaks the test if I don't do this. Differential Revision: https://phab.mercurial-scm.org/D11005

dirstatenonnormalcheck: fix some bytes formating on python3 Passing any object to `%s` no longer works, we need to explicitely convert the representation to bytes. Differential Revision: https://phab.mercurial-scm.org/D10993

dirstate-item: use the properties in `contrib/dirstatenonnormalcheck.py` Differential Revision: https://phab.mercurial-scm.org/D10991

dirstate-item: use the properties in largefiles I am now sure what large file is doing with then, but at least it is no longer using the tuple API to access data. Differential Revision: https://phab.mercurial-scm.org/D10990

dirstate-item: use the `state` property in debugpathcomplete Ideally we would use narrower property, but this is a good start. Differential Revision: https://phab.mercurial-scm.org/D10989

dirstate-item: use the properties in fakedirstatewritetime Differential Revision: https://phab.mercurial-scm.org/D10987

dirstate-item: use the properties in dirstatemap Differential Revision: https://phab.mercurial-scm.org/D10986

dirstate-item: use the properties in pathutil Differential Revision: https://phab.mercurial-scm.org/D10985

dirstate-item: use need_delay when packing dirstate Differential Revision: https://phab.mercurial-scm.org/D10984

dirstate-item: rename the class to DirstateItem The object is no longer a tuple, so it seems clearer to rename it (and its associated method) Differential Revision: https://phab.mercurial-scm.org/D10982

dirstate-entry: add a `mtime` property This is clearer than "tuple" indexing. Differential Revision: https://phab.mercurial-scm.org/D10981