dirstatemap: temporarily use `from_v1_data` in `addfile` We are about to change the `__init__` for `DirstateItem`. To make the transition easier, we move existing caller to `DirstateItem.from_v1_data`. Differential Revision: https://phab.mercurial-scm.org/D11318

dirstate-item: fix the declaration of the Cext `from_v1_meth` This method is apparently not called from anywhere since the declaration was garbage. We will start calling it in the next changeset. Differential Revision: https://phab.mercurial-scm.org/D11317

dirstate-item: fix Cext declaration of dm_nonnormal and dm_otherparent These are property, not method. Differential Revision: https://phab.mercurial-scm.org/D11316

dirstatemap: replace `removefile` by an explicit `entry.set_untracked()` All the other caller goes through `reset_state`, so we can safely have an explicit method on `DirstateItem` object. This means that all the logic to preserve the previous state (from p2, merged, etc) is now properly encapsulated within the DirstateItem. This pave the way to using different storage for these information. Differential Revision: https://phab.mercurial-scm.org/D11315

dirstate: forward `remove` call to newer `API` The `_remove` method was only called in the deprecated `remove` function. We merge the two and express it in terms of call to new API methods. Differential Revision: https://phab.mercurial-scm.org/D11314

branching: merge stable into default

store: return just one filename in walk functions Various walk functions return `(revlog_type, decoded, encoded)` where decoded could be None. But no-one cares about `encoded` and expects `unencoded` to be present, except verify (because this can only happen with old repo formats). Simplify all this by either failing outright if a decoding a filename fails (instead of almost certainly failing with a type error due to treating None as a bytes), or skipping the filename but providing in an out argument for hg verify. Differential Revision: https://phab.mercurial-scm.org/D11248

tests: rename test-clone-uncompressed.t as clone --uncompressed is deprecated in favor of --stream Differential Revision: https://phab.mercurial-scm.org/D11237

debugrebuildfncache: add a cheaper option to rebuild the fncache On my repository, debugrebuildfncache takes 5-10min with the lock. With the flag added in this commit, it takes 10s. The tradeoff is that it only recovers from certain kinds of corruptions. It is intended to to recover faster from fncaches broken by a revlog split during a transaction that ends up being rolled back. Differential Revision: https://phab.mercurial-scm.org/D11265

test: reduce noise, so the important bits stand out Differential Revision: https://phab.mercurial-scm.org/D11264

wireprotov1peer: update all rpcs to use the new batchable scheme If desired, we could keep the future class and the function that upgrades an old style rpc instead of a new style, for extensions. Differential Revision: https://phab.mercurial-scm.org/D11212

wireprotov1peer: simplify the way batchable rpcs are defined The scheme with futures/generator is confusing due to the way communication is done by side effects, especially with two different "future" objects. Just returning a request and a function to read the response is easier to understand. There are tests failures with the largefiles extension due to it aliasing one rpc to another one, which gets fixed in the next commit. Differential Revision: https://phab.mercurial-scm.org/D11211

dirstate: directly call the dirstatemap in `set_untracked` This function is only called in two places: the deprecated "remove" method and in the new `set_untracked` method. So we simply inline the appropriate content in `set_untracked`, paving the way to dropping the deprecated code and its associated function. Differential Revision: https://phab.mercurial-scm.org/D11313

resources: narrow the try:except clause to minimum Otherwise this mind hides other import or attribute errors. Differential Revision: https://phab.mercurial-scm.org/D11312

merge with stable

hgwebdir: avoid systematic full garbage collection Forcing a systematic full garbage collection upon each request can serioulsy harm performance. This is reported as https://bz.mercurial-scm.org/show_bug.cgi?id=6075 With this change we're performing the full collection according to a new setting, `experimental.web.full-garbage-collection-rate`. The default value is 1, which doesn't change the behavior and will allow us to test on real use cases. If the value is 0, no full garbage collection occurs. Regardless of the value of the setting, a partial garbage collection still occurs upon each request (not attempting to collect objects from the oldest generation). This should be enough to take care of reference cycles that have been created by the last request (assessment of this requires changing the setting, not to be 1). In my experience chasing memory leaks in Mercurial servers, the full collection never reclaimed any memory, but this is with Python 3 and biased towards small repositories. On the other hand, as explained in the Python developer docs [1], frequent full collections are very harmful in terms of performance if lots of objects survive the collection, and hence stay in the oldest generation. Note that `gc.collect()` is indeed trying to collect the oldest generation [2]. This happens usually in two cases: - unwanted lingering objects (i.e., an actual memory leak that the GC cannot do anything about). Sadly, we have lots of those these days. - desireable long-term objects, typically in caches (not inner caches carried by repositories, which should be collected with them). This is a subject of interest for the Heptapod project. In short, the flat rate that this change still permits is probably a bad idea in most cases, and the default value can be tweaked later on (or even be set to 0) according to experiments in the wild. The test is inspired from test-hgwebdir-paths.py [1] https://devguide.python.org/garbage_collector/#collecting-the-oldest-generation [2] https://docs.python.org/3/library/gc.html#gc.collect Differential Revision: https://phab.mercurial-scm.org/D11204

obsolete: disable other evolution config options if createmarkers is off We used to raise an abort in this case, but recent changes to local clone command (377d8fc20e34) resulted in destrepo both caring about experimental.evolution config options and not initializing extensions. So imagine if you had evolve and allowdivergence enabled in your ~/.hgrc. Local clone stopped working after 377d8fc20e34 because evolve sets experimental.evolution=all, but only on srcrepo, for destrepo the extension is not initialized. It's possible to make local cloning work by initializing extensions for destrepo in some cases, but in other cases (e.g. allowdivergence in ~/.hgrc, evolve extension in original-repo/.hg/hgrc) it would still fail. In a discussion with Pierre-Yves David it was decided to simply force other evolution options to be false if createmarkers is not enabled. Differential Revision: https://phab.mercurial-scm.org/D11223

fix: use obsolete.isenabled() to check for experimental.allowdivergence Now that obsolete.isenabled() can also check if divergence is allowed, let's use it for consistency. Other experimental.evolution options are already checked via this function. Differential Revision: https://phab.mercurial-scm.org/D11222

rebase: use obsolete.isenabled() to check for experimental.allowdivergence Now that obsolete.isenabled() can also check if divergence is allowed, let's use it for consistency. Other experimental.evolution options are already checked via this function. Differential Revision: https://phab.mercurial-scm.org/D11221

typing: add several assertions to dirstatemap to appease pytype I think it's been mentioned in IRC that these can't be None in this case. This fixes: File "/mnt/c/Users/Matt/hg/mercurial/dirstatemap.py", line 213, in addfile: unsupported operand type(s) for &: 'None' and 'int' [unsupported-operands] No attribute '__and__' on None or '__rand__' on int Called from (traceback): line 290, in reset_state File "/mnt/c/Users/Matt/hg/mercurial/dirstatemap.py", line 214, in addfile: unsupported operand type(s) for &: 'None' and 'int' [unsupported-operands] No attribute '__and__' on None or '__rand__' on int Called from (traceback): line 290, in reset_state Differential Revision: https://phab.mercurial-scm.org/D11235

tests: allow Google's internal builds of clang-format to be used These builds do not actually include any Google-specific formatting changes; the only reason they don't include the LLVM version number is due to a decision to elide the version number from *all* LLVM/clang projects. For most builds of clang-format, even "unofficial" ones, the LLVM version will be displayed; example: ``` clang-format version 14.0.0 (https://github.com/llvm/llvm-project.git 1830ec94ac022ae0b6d6876fc2251e6b91e5931e) ``` The Google-internal build looks like this: ``` clang-format version google3-trunk (1830ec94ac022ae0b6d6876fc2251e6b91e5931e) ``` Differential Revision: https://phab.mercurial-scm.org/D10538

Added signature for changeset 6ee0244fc1cf

Added tag 5.9.3 for changeset 6ee0244fc1cf

relnotes: update release notes for upcoming 5.9.3 Differential Revision: https://phab.mercurial-scm.org/D11720

heptapod-ci: actually give pytest more time before timeout `HGTEST_TIMEOUT` is overridden by `HGTEST_SLOWTIMEOUT` for tests marked as slow, which `test-check-pytype.t` is. So this whole time the timeout was 1500s (or 25 minutes), which is unfortunately not long enough for a *lot* of the times it's run on the CI. Differential Revision: https://phab.mercurial-scm.org/D11717

tests: better determinism in test-chg.t chg tests fail pretty often with "Sample count: *" line disappearing. It disappears because the sample count is zero, in which case a custom message is printed. This commit makes the test succeed in that case. Differential Revision: https://phab.mercurial-scm.org/D11716

python: compatibility for python 3.11 (issue6604) The `unittest._TextTestResult` alias has been removed. The "new" name has been available since 3.2, and we only support 3.5.3+. Differential Revision: https://phab.mercurial-scm.org/D11690

pyoxidizer: force pip to not use pep517 in order to be able to install hg Mercurial is not (yet) a pep517 package, but the presence of a pyproject.toml file tells newer-ish versions of pip that it should be one. This is related to 58fe6d127a01, and fixes pyoxidizer builds for the Heptapod CI. Differential Revision: https://phab.mercurial-scm.org/D11710

merge-halt: fix issue with merge.on-failure=halt breaking unshelve Differential Revision: https://phab.mercurial-scm.org/D11706

merge-halt: demonstrate unshelve issue with merge.on-failure=halt Differential Revision: https://phab.mercurial-scm.org/D11705

rust-nodemap: backed out mitigation for issue 6554 This is a backout of changeset 3fffb48539ee. Issue 6554 is now considered solved, hence its mitigation has to be removed, if only for its performance cost. Differential Revision: https://phab.mercurial-scm.org/D11703

rewrite: fix issue6599 Explanation inside. Differential Revision: https://phab.mercurial-scm.org/D11685

Added signature for changeset 750920b18aaa

Added tag 5.9.2 for changeset 750920b18aaa

windows-ci: temporarily allow Windows jobs to fail We have unfortunately ran out of free credit on the runners we were using from OVH for the Windows CI. We will be disabling the two remaining ones on the 30th of September, hence we need the CI to pass even if Windows jobs cannot start as a temporary measure. Hopefully we can find another way of getting Windows runners soon. Differential Revision: https://phab.mercurial-scm.org/D11499

manifestlog: also monitor `00manifest.n` when applicable This let the locarepo's file cache detect outdated nodemap docket and reload the manifestlog after `localrepo.invalidate` when applicable. The same problem than issue6554 could affect the Manifest too. Differential Revision: https://phab.mercurial-scm.org/D11483

changelog: also monitor `00changelog.n` when applicable (issue6554) This let the locarepo's file cache detect outdated nodemap docket and reload the changelog after `localrepo.invalidate` when applicable. Differential Revision: https://phab.mercurial-scm.org/D11482

persistent-nodemap: introduce a test to highlight possible race Weakness in the current file caching of the changelog means that a writer can end up using an outdated docket. This might result in "committed" persistent-nodemap data from a previous writer to be overwritten by a later writer. This break the strong "append only" assumption of the persistent nodemap and can result in confused reader. The race windows are quite narrow. See the test documentation for details. The issues is fixed in the next changeset. Differential Revision: https://phab.mercurial-scm.org/D11481

test: enable share-safe in test-persistent-nodemap We will use some share in this test and I would rather have them safe. Differential Revision: https://phab.mercurial-scm.org/D11480

filecache: abstract the fetching of the list of tracked file We will need it for a coming fix that will requires to check a variable list of file for the changelog. Differential Revision: https://phab.mercurial-scm.org/D11479

persistent-nodemap: use quiet upgrade in tests This make them less verbose while keeping the important information in the checked output. Differential Revision: https://phab.mercurial-scm.org/D11478

persistent-nodemap: fix a typo in a test comment oops. Differential Revision: https://phab.mercurial-scm.org/D11477

patchbomb: fix traceback on outgoing when not specifying revisions A user reported that `hg email -n -o` raised with the following traceback: ``` ** unknown exception encountered, please report by visiting ** https://mercurial-scm.org/wiki/BugTracker ** Python 3.8.12 (default, Sep 13 2021, 22:12:54) [GCC 7.5.0] ** Mercurial Distributed SCM (version 5.9.1) ** Extensions loaded: patchbomb Traceback (most recent call last): File "/usr/pkg/bin/hg", line 61, in <module> dispatch.run() File "/usr/pkg/lib/python3.8/site-packages/mercurial/dispatch.py", line 144, in run status = dispatch(req) File "/usr/pkg/lib/python3.8/site-packages/mercurial/dispatch.py", line 250, in dispatch status = _rundispatch(req) File "/usr/pkg/lib/python3.8/site-packages/mercurial/dispatch.py", line 294, in _rundispatch ret = _runcatch(req) or 0 File "/usr/pkg/lib/python3.8/site-packages/mercurial/dispatch.py", line 470, in _runcatch return _callcatch(ui, _runcatchfunc) File "/usr/pkg/lib/python3.8/site-packages/mercurial/dispatch.py", line 480, in _callcatch return scmutil.callcatch(ui, func) File "/usr/pkg/lib/python3.8/site-packages/mercurial/scmutil.py", line 153, in callcatch return func() File "/usr/pkg/lib/python3.8/site-packages/mercurial/dispatch.py", line 460, in _runcatchfunc return _dispatch(req) File "/usr/pkg/lib/python3.8/site-packages/mercurial/dispatch.py", line 1273, in _dispatch return runcommand( File "/usr/pkg/lib/python3.8/site-packages/mercurial/dispatch.py", line 918, in runcommand ret = _runcommand(ui, options, cmd, d) File "/usr/pkg/lib/python3.8/site-packages/mercurial/dispatch.py", line 1285, in _runcommand return cmdfunc() File "/usr/pkg/lib/python3.8/site-packages/mercurial/dispatch.py", line 1271, in <lambda> d = lambda: util.checksignature(func)(ui, *args, **strcmdopt) File "/usr/pkg/lib/python3.8/site-packages/mercurial/util.py", line 1886, in check return func(*args, **kwargs) File "/usr/pkg/lib/python3.8/site-packages/hgext/patchbomb.py", line 817, in email revs = _getoutgoing(repo, dest, revs) File "/usr/pkg/lib/python3.8/site-packages/hgext/patchbomb.py", line 537, in _getoutgoing safe_paths = [urlutil.hidepassword(p.rawloc) for p in paths] File "/usr/pkg/lib/python3.8/site-packages/hgext/patchbomb.py", line 537, in <listcomp> safe_paths = [urlutil.hidepassword(p.rawloc) for p in paths] File "/usr/pkg/lib/python3.8/site-packages/mercurial/utils/urlutil.py", line 501, in get_push_paths msg %= dest TypeError: %b requires a bytes-like object, or an object that implements __bytes__, not 'NoneType' ``` This is due to patchbomb passing `[None]` in the `dests` parameter of the multi-paths outgoing function, we fix this in this change and add a non-regression test. Differential Revision: https://phab.mercurial-scm.org/D11456

pep-517: remove the `build-backend` key to allow for users to build extensions PEP 517 changed how projects should define their build dependencies. The presence of a `pyproject.toml` file changes the behavior of `pip` to conform to PEP 517. Since we haven't updated the `setup.py` file yet (I'm not even sure we're not an edge case which would make it harder/impossible, I have not tried yet), this is a workaround to issue6589: it allows users to pass `--no-use-pep517` to pip to revert to the legacy installer. The build backend is simply assumed to be the `build_meta:__legacy__` one. Differential Revision: https://phab.mercurial-scm.org/D11393

tests: make removeemptydirs more portable The behavior of pwd change from one system (i.e. default shell/sh) from one system to another so we use fuzzy matching for the output we do not care and we use narrower command call to highlight what we are trying to test here. Differential Revision: https://phab.mercurial-scm.org/D11449

check-code: make it possible to ignore the PWD check in some situation The check-code script does not motivate the banning of $PWD so I am not sure what it is about. And I will needs to use the env variable in the next tests. So I am adding a way to make an exception. Differential Revision: https://phab.mercurial-scm.org/D11450

relnotes: update next Differential Revision: https://phab.mercurial-scm.org/D11448

relnotes: add release notes for 5.9 Differential Revision: https://phab.mercurial-scm.org/D11447

narrow: fix commits of empty files The problem is that when committing a new file with empty contents (or in general empty file with filelog p1 = -1), hg commit with narrow doesn't create a filelog revision at all, which causes failures in further commands. The problem seems to be that: - hg thinks that instead of creating a new filelog revision, it can use the filelog's p1 (the nullrev) - because it thinks the file contents is the same in that revision and in p1 - because `narrowfilelog.cmp(nullrev, b'')` is True (unlike with `filelog.cmp`) It's not clear to me which `cmp` behaves better. But I think it makes sense to change the commit code to not to "reuse" the null rev when adding an empty file with filelog p1 == filelog p2 == -1. This is consistent with never writing the null rev in the manifest, which `hg verify` claims is an invariant: ``` inside/c@4: manifest refers to unknown revision 000000000000 ``` Differential Revision: https://phab.mercurial-scm.org/D11400

narrow: show repo corruption when commiting empty files Differential Revision: https://phab.mercurial-scm.org/D11399

filemerge: be more strict when detecting conflict markers, add `|` markers I received a user complaint about detecting a line that contained 78 `=` followed by `*/` as a conflict marker. We'll never generate that, we generate 7 identical characters and either the end of the line, or a space. Let's be explicit about detecting exactly what we produce to reduce the chances of a false positive. While we're here, add `|||||||` as a detected conflict marker (generated with the `keep-merge3` style conflicts). Differential Revision: https://phab.mercurial-scm.org/D11391

Added signature for changeset 86a60679cf61

Added tag 5.9.1 for changeset 86a60679cf61

fix: again allow formatting the working copy while merging I forgot about unfinished merges (I think I was thinking only about unfinished merge conflicts) when I wrote https://phab.mercurial-scm.org/D11209. As a coworker (hg contributor dploch) reported to me, this led to `hg fix --working-dir` failing when you have an uncommitted merge. The fix is trivial: just move the assertion to just before the call to `scmutil.movedirstate()` where it actually matters. Differential Revision: https://phab.mercurial-scm.org/D11376

pyoxidizer: add arch to PyOxidizer MSIs This is the proper filename pattern. This filename format wasn't properly implemented when PyOxidizer took over Python 3 MSI generation in 603efb3845ba. Differential Revision: https://phab.mercurial-scm.org/D11359

pyoxidizer: pass arch to WiXInstaller() We just upgraded Windows automation to PyOxidizer 0.17.0. This version of PyOxidizer changed the behavior of WiX installers so installer architecture defaulted to "x64" (before it defaulted to the target of the pyoxidizer.exe binary). To allow controlling the architecture of the installer, the `arch` argument was added to `WiXInstaller`. This commit passes that argument in. Differential Revision: https://phab.mercurial-scm.org/D11358

contrib: install PyOxidizer 0.17.0 This pulls in some changes we want to improve Windows MSI installers. This will need separate enhancements to the pyoxidizer.bzl. But those will be handled in a separate changeset. Differential Revision: https://phab.mercurial-scm.org/D11357

packaging: reference proper output directory 9438e9b7321a changed the name of the PyOxidizer target, which changed the name of the output directory. The code changed by this patch wasn't properly updated by that changeset. This resulted in a run-time failure due to trying to read from a non-existent directory. This change should fix the building of Python 3 Inno installers. Differential Revision: https://phab.mercurial-scm.org/D11356

packaging: pass extra_pyoxidizer_vars only to pyoxidizer Before, we would attempt to call a function (build_installer_py2exe) that didn't accept this keyword argument. This was preventing the building of py2exe installers. Differential Revision: https://phab.mercurial-scm.org/D11355

hg: don't attempt to extend `sys.path` with the user site without `APPDATA` This variable is created by the system and *should* be available, but test-lfs-bundle.t has a test where it is explicitly unset. It wasn't caught before because prior to 95af358fcdfe, it was limited to the py2exe binary. As a precaution, fix both that and the pyoxidizer case that was causing the test to fail. Differential Revision: https://phab.mercurial-scm.org/D11354