tests: run python script through quoted interpreter instead of directly This helps Windows when python is installed to %PROGRAMFILES%. Differential Revision: https://phab.mercurial-scm.org/D10634

tests: ensure `$PYTHON` is quoted for Windows Global installs of python3 go into "Program Files", and tons of tests fail with mysterious errors if this isn't quoted. Most of this is a followup to 0826d684a1b5, but a some of these were existing issues. Shebang lines are ignored because quoting breaks direct execution- these will need to be launched indirectly with the quoted `$PYTHON` command. Differential Revision: https://phab.mercurial-scm.org/D10633

black: make codebase compatible with black v21.4b2 and v20.8b1 I don't know what exact version of black made it care about these whitespace differences, but this is the version I got when I just installed it with `pip3 install black`. I'm intentionally not increasing the version of black required, as I don't want to force everyone to upgrade their version of black, and these fixes are backwards compatible with black v20.8b1. If there are more issues in the future and this becomes a maintenance burden I may do so in a future change. Tested with both versions of black (I got the older version via `pip3 install black==20.8b1`) Differential Revision: https://phab.mercurial-scm.org/D10539

debugcommands: fix some plural-agreements I noticed Differential Revision: https://phab.mercurial-scm.org/D10669

Added signature for changeset 067f2c53fb24

Added tag 5.8 for changeset 067f2c53fb24

phabricator: adapt to the new `urlutil.url()` API This avoids a bunch of deprecation warnings in the tests. Differential Revision: https://phab.mercurial-scm.org/D10541

extensions: ignore exceptions from an extension's `getversion()` method This method is usually called when there's a stacktrace being generated, or with `hg version -v`. Raising another exception risks mangling the bug report info. I hit this issue when trying to add the method to the keyring extension to report the version of the extension and the underlying module, and ran into demandimport issues prior to py3.8. It seems like a wise thing to do anyway, though unfortunately there's no convenient `ui` object around to issue a warning. Use 'unknown' to signal that it tried to report a version and failed, unlike the default case of printing nothing. Differential Revision: https://phab.mercurial-scm.org/D10540

git: ensure all dirstate state values are bytes I'm not sure how this particular git status occurs, but after the fallout of issue 6510 and getting into the issue 6511 state where `git status` shows the files as modified in both the "to be committed" and "not staged" lists, `hg diff` was crashing in `workingctx.__contains__()`. Differential Revision: https://phab.mercurial-scm.org/D10532

tests: synchronize the git and Mercurial username The problem with the default name of "test" set by the test runner is the stringutil methods are unable to split out separate user and email addresses that git wants. This means the username is recorded in git as "test <test>". Amending a commit with that user ends up trying to use "<test>" as the person field for the new commit, and the git library complains about the angle brackets. We should probably abort with a clearer message any time this bad form is used with the git extension. One of the commit dates is tweaked to recreate the ambiguous hash prefix from before. Differential Revision: https://phab.mercurial-scm.org/D10531

git: initialize `extra` to have at least the branch name for nullid Otherwise, this crashes trying to convert to local encoding: ... File "/mnt/c/Users/Matt/hg/mercurial/logcmdutil.py", line 333, in _show branch = ctx.branch() File "/mnt/c/Users/Matt/hg/mercurial/context.py", line 675, in branch return encoding.tolocal(self._changeset.extra.get(b"branch")) File "/mnt/c/Users/Matt/hg/mercurial/encoding.py", line 181, in tolocal if isasciistr(s): TypeError: a bytes-like object is required, not 'NoneType' This was originally reported to the thg bug tracker. https://foss.heptapod.net/mercurial/tortoisehg/thg/-/issues/5629 Differential Revision: https://phab.mercurial-scm.org/D10528

git: consistently use str for parents when rebuilding the index database The tests show no changes, but when these values are overwritten shortly after when a git commit object is available, that uses str. It seems better to use that for consistency. It does materially affect the database though, because the old value stored was `X'3030..3030'` and is now '00..00' when the changelog table is dumped in sqlite3. There is one query that specifies the parents, but it passes the non null hashes as str, so it worked as expected. That likely explains the lack of test changes. Differential Revision: https://phab.mercurial-scm.org/D10527

git: pass the correct type to the sqlite3 query for baselog.hasnode() It looks like this function is mostly used in exchange (which isn't supported now) and histedit (which crashes). But I did verify in a REPL that passing bytes to `db.execute()` fails to find the row, whereas passing str works. Differential Revision: https://phab.mercurial-scm.org/D10526

git: use the correct type for stopping changelog.revs() The `tip` function returns a binary node, but the database is expecting an int. Differential Revision: https://phab.mercurial-scm.org/D10525

git: use the proper filenode for deleted files in changelog.changelogrevision Inline printing shows it still doesn't find the removed file in the test, but it is at least a str instead of bytes like the immediate query before it that does find files. Looking at the database, it doesn't look like the remove was recorded. Additionally, `hg log -r 'removes("re:.*")'` stacktraces. Differential Revision: https://phab.mercurial-scm.org/D10524

tests: add coverage for git.changelog.headrevs(...) Differential Revision: https://phab.mercurial-scm.org/D10523

git: make changelog.tiprev() return int instead of tuple (issue6510) Differential Revision: https://phab.mercurial-scm.org/D10522

git: fix partial node matching Looks like everything is meant to be a str here. Differential Revision: https://phab.mercurial-scm.org/D10521

tests: stabilize test-git-interop.t for some versions of git Not sure why this differs on my system, but it's likely due to running 2.17.1 on Ubuntu 18.04. Differential Revision: https://phab.mercurial-scm.org/D10520

tests: bump default timeout to 360s A number of tests hit or almost hit the default limit even on modern hardware. While the tests are ideally split into smaller pieces, that's non-trivial work. HyperThreading and similar technologies can trigger this often, even without any other load on the machine. Differential Revision: https://phab.mercurial-scm.org/D10534

relnotes: copy contents of "next" to "5.8" and clear "next" The same procedure as every year^Wcycle. Unlike previous cycles, I haven't actually recorded the copy because we don't want to propagate further changes in "next" (from the stable branch) to "5.8". Differential Revision: https://phab.mercurial-scm.org/D10500

dirstateguard: use mktemp-like functionality to generate the backup filenames Previously these were generated with names like: `dirstate.backup.commit.<memory address of dirstateguard>` This could cause problems if two hg commands ran at the same time that used the same memory address, (which is apparently not uncommon if chg is involved), as memory addresses are not unique across processes. This issue was reported in the post-review comments on http://phab.mercurial-scm.org/D9952. Differential Revision: https://phab.mercurial-scm.org/D10504

repoview: separate concerns in _filteredrepotypes comment The cited issue in Python bugtracker is closed, but hasn't been fixed. We've been able to use the attached example and reproduce it with Python 3.9. The point where it turns from needless stress on the GC to the an actual leak is when one factors in the fact that the GC was before Python 3.4 unable to collect some types (see PEP 442). Note that even with Python 2.7, the simple example of cycles due to __mro__ are collectable. This was seen again with the example attached on the CPython issue.

repoview: fix memory leak of filtered repo classes The leak occurs in long-running server processes with extensions, and is measured at 110kB per request. Before this change, the contents of the `_filteredrepotypes` cache are not properly garbage collected, despite it begin a `WeakKeyDictionary`. Extensions have a tendency to generate a new repository class for each `localrepo` instantiation. Server processes based on `hgwebdir_mod` will instantiate a new `localrepo` for each HTTP request that involves a repository. As a result, with a testing process that repeatedly opens a repository with several extensions activated (`topic` notably among them), we see a steady increase in resident memory of 110kB per repository instantiation before this change. This is also true, if we call `gc.collect()` at each instantiation, like `hgwebdir_mod` does, or not. The cause of the leak is that the *values* aren't weak references. This change uses `weakref.ref` for the values, and this makes in our measurements the resident size increase drop to 5kB per repository instantiation, with no explicit call of `gc.collect()` at all. There is currently no reason to believe that this remaining leak of 5kB is related to or even due to Mercurial core. We've also seen evidence that `ui.ui` instances weren't properly garbage collected before the change (with the change, they are). This could explain why the figures are relatively high. In theory, the collection of weak references could lead to much more misses in the cache, so we measured the impact on the original case that was motivation for introducing that cache in 7e89bd0cfb86 (see also issue5043): `hg convert` of the mozilla-central repository. The bad news here is that there is a major memory leak there, both with and without the present changeset. There were no more cache misses, and we could see no more memory leak with this change: the resident size after importing roughly 100000 changesets was at 12.4GB before, and 12.5GB after. The small increase is mentioned for completeness only, and we believe that it should be ignored, at least as long as the main leak isn't fixed. At less than 1% of the main leak, even finding out whether it is merely noise would be wasteful. Original context where this was spotted and first mitigated: https://foss.heptapod.net/heptapod/heptapod/-/issues/466 The leak reduction was also obtained in Heptapod inner HTTP server, which amounts to the same as `hgwebdir_mod` for these questions. The measurements done with Python 3.9, similar figures seen with 3.8. More work on our side would be needed to give measurements with 2.7, because of testing server process does not support it.

repoview: style change in newtype() cache handling This way of writing it does not change the logic at all, but is more fit for the change we want to make in the next changeset. If anything, that's one dict lookup less in the hot path, but that should be non measurable.

tests: don't hard-code /bin/bash From Thomas Klausner in pkgsrc. Differential Revision: https://phab.mercurial-scm.org/D10507

lint: fix failing code check 8d2b62d716b0 introduced code that `test-check-code.t` wasn't happy about, this fixes it. Differential Revision: https://phab.mercurial-scm.org/D10506

typing: enable pytype processing on mercurial/upgrade_utils/actions.py This was the suggested workaround for the pytype bug that caused this file to be processed forever. Differential Revision: https://phab.mercurial-scm.org/D10460

incoming: use bytes for an error message Caught by pytype. Differential Revision: https://phab.mercurial-scm.org/D10461

merge: stable heads I forgot to pull before rolling rc1, so we just have a couple of patches that missed the rc1 train. Mea culpa.