Matt Harbison <matt_harbison@yahoo.com> [Sun, 02 May 2021 19:54:08 -0400] rev 47034
tests: invoke some shell scripts through the shell interpreter for Windows
Otherwise, Windows was prompting what program to use to open the file (or just
opening it if there was a file association configured).
Differential Revision: https://phab.mercurial-scm.org/D10635
Matt Harbison <matt_harbison@yahoo.com> [Sat, 01 May 2021 13:04:56 -0400] rev 47033
tests: run python script through quoted interpreter instead of directly
This helps Windows when python is installed to %PROGRAMFILES%.
Differential Revision: https://phab.mercurial-scm.org/D10634
Matt Harbison <matt_harbison@yahoo.com> [Sat, 01 May 2021 00:41:43 -0400] rev 47032
tests: ensure `$PYTHON` is quoted for Windows
Global installs of python3 go into "Program Files", and tons of tests fail with
mysterious errors if this isn't quoted. Most of this is a followup to
0826d684a1b5, but a some of these were existing issues. Shebang lines are
ignored because quoting breaks direct execution- these will need to be launched
indirectly with the quoted `$PYTHON` command.
Differential Revision: https://phab.mercurial-scm.org/D10633
Kyle Lippincott <spectral@google.com> [Fri, 30 Apr 2021 16:13:02 -0700] rev 47031
black: make codebase compatible with black v21.4b2 and v20.8b1
I don't know what exact version of black made it care about these whitespace
differences, but this is the version I got when I just installed it with
`pip3 install black`.
I'm intentionally not increasing the version of black required, as I don't want
to force everyone to upgrade their version of black, and these fixes are
backwards compatible with black v20.8b1. If there are more issues in the future
and this becomes a maintenance burden I may do so in a future change.
Tested with both versions of black (I got the older version via
`pip3 install black==20.8b1`)
Differential Revision: https://phab.mercurial-scm.org/D10539
Augie Fackler <augie@google.com> [Tue, 04 May 2021 13:28:42 -0400] rev 47030
debugcommands: fix some plural-agreements I noticed
Differential Revision: https://phab.mercurial-scm.org/D10669
Pulkit Goyal <7895pulkit@gmail.com> [Mon, 03 May 2021 22:53:02 +0530] rev 47029
Added signature for changeset 067f2c53fb24
Pulkit Goyal <7895pulkit@gmail.com> [Mon, 03 May 2021 22:52:56 +0530] rev 47028
Added tag 5.8 for changeset 067f2c53fb24
Matt Harbison <matt_harbison@yahoo.com> [Sat, 01 May 2021 00:28:39 -0400] rev 47027
phabricator: adapt to the new `urlutil.url()` API
This avoids a bunch of deprecation warnings in the tests.
Differential Revision: https://phab.mercurial-scm.org/D10541
Matt Harbison <matt_harbison@yahoo.com> [Fri, 30 Apr 2021 17:36:09 -0400] rev 47026
extensions: ignore exceptions from an extension's `getversion()` method
This method is usually called when there's a stacktrace being generated, or with
`hg version -v`. Raising another exception risks mangling the bug report info.
I hit this issue when trying to add the method to the keyring extension to
report the version of the extension and the underlying module, and ran into
demandimport issues prior to py3.8. It seems like a wise thing to do anyway,
though unfortunately there's no convenient `ui` object around to issue a
warning. Use 'unknown' to signal that it tried to report a version and failed,
unlike the default case of printing nothing.
Differential Revision: https://phab.mercurial-scm.org/D10540
Matt Harbison <matt_harbison@yahoo.com> [Wed, 28 Apr 2021 17:05:32 -0400] rev 47025
git: ensure all dirstate state values are bytes
I'm not sure how this particular git status occurs, but after the fallout of
issue 6510 and getting into the issue 6511 state where `git status` shows the
files as modified in both the "to be committed" and "not staged" lists,
`hg diff` was crashing in `workingctx.__contains__()`.
Differential Revision: https://phab.mercurial-scm.org/D10532
Matt Harbison <matt_harbison@yahoo.com> [Wed, 28 Apr 2021 10:29:45 -0400] rev 47024
tests: synchronize the git and Mercurial username
The problem with the default name of "test" set by the test runner is the
stringutil methods are unable to split out separate user and email addresses
that git wants. This means the username is recorded in git as "test <test>".
Amending a commit with that user ends up trying to use "<test>" as the person
field for the new commit, and the git library complains about the angle
brackets. We should probably abort with a clearer message any time this bad
form is used with the git extension.
One of the commit dates is tweaked to recreate the ambiguous hash prefix from
before.
Differential Revision: https://phab.mercurial-scm.org/D10531
Matt Harbison <matt_harbison@yahoo.com> [Tue, 27 Apr 2021 19:38:19 -0400] rev 47023
git: initialize `extra` to have at least the branch name for nullid
Otherwise, this crashes trying to convert to local encoding:
...
File "/mnt/c/Users/Matt/hg/mercurial/logcmdutil.py", line 333, in _show
branch = ctx.branch()
File "/mnt/c/Users/Matt/hg/mercurial/context.py", line 675, in branch
return encoding.tolocal(self._changeset.extra.get(b"branch"))
File "/mnt/c/Users/Matt/hg/mercurial/encoding.py", line 181, in tolocal
if isasciistr(s):
TypeError: a bytes-like object is required, not 'NoneType'
This was originally reported to the thg bug tracker.
https://foss.heptapod.net/mercurial/tortoisehg/thg/-/issues/5629
Differential Revision: https://phab.mercurial-scm.org/D10528
Matt Harbison <matt_harbison@yahoo.com> [Tue, 27 Apr 2021 18:39:59 -0400] rev 47022
git: consistently use str for parents when rebuilding the index database
The tests show no changes, but when these values are overwritten shortly after
when a git commit object is available, that uses str. It seems better to use
that for consistency.
It does materially affect the database though, because the old value stored was
`X'3030..3030'` and is now '00..00' when the changelog table is dumped in
sqlite3. There is one query that specifies the parents, but it passes the non
null hashes as str, so it worked as expected. That likely explains the lack of
test changes.
Differential Revision: https://phab.mercurial-scm.org/D10527
Matt Harbison <matt_harbison@yahoo.com> [Tue, 27 Apr 2021 17:54:08 -0400] rev 47021
git: pass the correct type to the sqlite3 query for baselog.hasnode()
It looks like this function is mostly used in exchange (which isn't supported
now) and histedit (which crashes). But I did verify in a REPL that passing
bytes to `db.execute()` fails to find the row, whereas passing str works.
Differential Revision: https://phab.mercurial-scm.org/D10526
Matt Harbison <matt_harbison@yahoo.com> [Tue, 27 Apr 2021 17:11:55 -0400] rev 47020
git: use the correct type for stopping changelog.revs()
The `tip` function returns a binary node, but the database is expecting an int.
Differential Revision: https://phab.mercurial-scm.org/D10525
Matt Harbison <matt_harbison@yahoo.com> [Tue, 27 Apr 2021 16:58:59 -0400] rev 47019
git: use the proper filenode for deleted files in changelog.changelogrevision
Inline printing shows it still doesn't find the removed file in the test, but it
is at least a str instead of bytes like the immediate query before it that does
find files. Looking at the database, it doesn't look like the remove was
recorded. Additionally, `hg log -r 'removes("re:.*")'` stacktraces.
Differential Revision: https://phab.mercurial-scm.org/D10524
Matt Harbison <matt_harbison@yahoo.com> [Tue, 27 Apr 2021 12:59:17 -0400] rev 47018
tests: add coverage for git.changelog.headrevs(...)
Differential Revision: https://phab.mercurial-scm.org/D10523
Matt Harbison <matt_harbison@yahoo.com> [Tue, 27 Apr 2021 12:31:30 -0400] rev 47017
git: make changelog.tiprev() return int instead of tuple (issue6510)
Differential Revision: https://phab.mercurial-scm.org/D10522
Matt Harbison <matt_harbison@yahoo.com> [Tue, 27 Apr 2021 00:26:12 -0400] rev 47016
git: fix partial node matching
Looks like everything is meant to be a str here.
Differential Revision: https://phab.mercurial-scm.org/D10521
Matt Harbison <matt_harbison@yahoo.com> [Tue, 27 Apr 2021 00:23:05 -0400] rev 47015
tests: stabilize test-git-interop.t for some versions of git
Not sure why this differs on my system, but it's likely due to running 2.17.1 on
Ubuntu 18.04.
Differential Revision: https://phab.mercurial-scm.org/D10520
Joerg Sonnenberger <joerg@bec.de> [Fri, 30 Apr 2021 02:25:10 +0200] rev 47014
tests: bump default timeout to 360s
A number of tests hit or almost hit the default limit even on modern
hardware. While the tests are ideally split into smaller pieces, that's
non-trivial work. HyperThreading and similar technologies can trigger
this often, even without any other load on the machine.
Differential Revision: https://phab.mercurial-scm.org/D10534
Martin von Zweigbergk <martinvonz@google.com> [Tue, 20 Apr 2021 08:28:11 -0700] rev 47013
relnotes: copy contents of "next" to "5.8" and clear "next"
The same procedure as every year^Wcycle.
Unlike previous cycles, I haven't actually recorded the copy because
we don't want to propagate further changes in "next" (from the stable
branch) to "5.8".
Differential Revision: https://phab.mercurial-scm.org/D10500
Kyle Lippincott <spectral@google.com> [Tue, 20 Apr 2021 13:01:47 -0700] rev 47012
dirstateguard: use mktemp-like functionality to generate the backup filenames
Previously these were generated with names like:
`dirstate.backup.commit.<memory address of dirstateguard>`
This could cause problems if two hg commands ran at the same time that used the
same memory address, (which is apparently not uncommon if chg is involved), as
memory addresses are not unique across processes.
This issue was reported in the post-review comments on
http://phab.mercurial-scm.org/D9952.
Differential Revision: https://phab.mercurial-scm.org/D10504
Georges Racinet <georges.racinet@octobus.net> [Sat, 24 Apr 2021 16:30:05 +0200] rev 47011
repoview: separate concerns in _filteredrepotypes comment
The cited issue in Python bugtracker is closed, but hasn't been
fixed. We've been able to use the attached example and reproduce
it with Python 3.9.
The point where it turns from needless stress on the GC to
the an actual leak is when one factors in the fact that the GC
was before Python 3.4 unable to collect some types (see PEP 442).
Note that even with Python 2.7, the simple example of cycles
due to __mro__ are collectable. This was seen again with the
example attached on the CPython issue.
Georges Racinet <georges.racinet@octobus.net> [Fri, 23 Apr 2021 18:30:53 +0200] rev 47010
repoview: fix memory leak of filtered repo classes
The leak occurs in long-running server processes with
extensions, and is measured at 110kB per request.
Before this change, the contents of the `_filteredrepotypes`
cache are not properly garbage collected, despite it begin
a `WeakKeyDictionary`.
Extensions have a tendency to generate a new repository class
for each `localrepo` instantiation. Server processes based
on `hgwebdir_mod` will instantiate a new `localrepo` for each
HTTP request that involves a repository.
As a result, with a testing process that repeatedly opens a
repository with several extensions activated
(`topic` notably among them), we see a steady increase in
resident memory of 110kB per repository instantiation before this
change. This is also true, if we call `gc.collect()` at each
instantiation, like `hgwebdir_mod` does, or not.
The cause of the leak is that the *values* aren't weak references.
This change uses `weakref.ref` for the values, and this makes
in our measurements the resident size increase drop to 5kB per
repository instantiation, with no explicit call of `gc.collect()`
at all.
There is currently no reason to believe that this remaining leak
of 5kB is related to or even due to Mercurial core.
We've also seen evidence that `ui.ui` instances weren't properly
garbage collected before the change (with the change, they are).
This could explain why the figures are relatively high.
In theory, the collection of weak references could lead to
much more misses in the cache, so we measured the impact on
the original case that was motivation for introducing that cache
in 7e89bd0cfb86 (see also issue5043): `hg convert` of the
mozilla-central repository. The bad news here is that there is a
major memory leak there, both with and without the present changeset.
There were no more cache misses, and we could see no
more memory leak with this change: the resident size after importing
roughly 100000 changesets was at 12.4GB before, and 12.5GB after.
The small increase is mentioned for completeness only, and we
believe that it should be ignored, at least as long as the main
leak isn't fixed. At less than 1% of the main leak, even finding out
whether it is merely noise would be wasteful.
Original context where this was spotted and first mitigated:
https://foss.heptapod.net/heptapod/heptapod/-/issues/466
The leak reduction was also obtained in Heptapod inner HTTP server,
which amounts to the same as `hgwebdir_mod` for these questions.
The measurements done with Python 3.9, similar figures seen with 3.8.
More work on our side would be needed to give measurements with 2.7,
because of testing server process does not support it.
Georges Racinet <georges.racinet@octobus.net> [Sat, 24 Apr 2021 15:46:39 +0200] rev 47009
repoview: style change in newtype() cache handling
This way of writing it does not change the logic at all,
but is more fit for the change we want to make in the
next changeset.
If anything, that's one dict lookup less in the hot path,
but that should be non measurable.
Joerg Sonnenberger <joerg@bec.de> [Thu, 22 Apr 2021 02:57:30 +0200] rev 47008
tests: don't hard-code /bin/bash
From Thomas Klausner in pkgsrc.
Differential Revision: https://phab.mercurial-scm.org/D10507
Raphaël Gomès <rgomes@octobus.net> [Wed, 21 Apr 2021 17:59:14 +0200] rev 47007
lint: fix failing code check
8d2b62d716b0 introduced code that `test-check-code.t` wasn't happy about, this
fixes it.
Differential Revision: https://phab.mercurial-scm.org/D10506
Matt Harbison <matt_harbison@yahoo.com> [Wed, 14 Apr 2021 09:49:36 -0400] rev 47006
typing: enable pytype processing on mercurial/upgrade_utils/actions.py
This was the suggested workaround for the pytype bug that caused this file to be
processed forever.
Differential Revision: https://phab.mercurial-scm.org/D10460
Matt Harbison <matt_harbison@yahoo.com> [Sun, 18 Apr 2021 00:56:09 -0400] rev 47005
incoming: use bytes for an error message
Caught by pytype.
Differential Revision: https://phab.mercurial-scm.org/D10461
Augie Fackler <augie@google.com> [Wed, 21 Apr 2021 10:58:42 -0400] rev 47004
merge: stable heads
I forgot to pull before rolling rc1, so we just have a couple of
patches that missed the rc1 train. Mea culpa.
Augie Fackler <raf@durin42.com> [Wed, 21 Apr 2021 10:49:29 -0400] rev 47003
Added signature for changeset 8d2b62d716b0
Augie Fackler <raf@durin42.com> [Wed, 21 Apr 2021 10:49:28 -0400] rev 47002
Added tag 5.8rc1 for changeset 8d2b62d716b0
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 20 Apr 2021 16:31:13 +0200] rev 47001
relnote: mention a nodemap bug fixes
The fix was in a3720569a43f.
Differential Revision: https://phab.mercurial-scm.org/D10499
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 20 Apr 2021 16:29:05 +0200] rev 47000
relnote: document the use of persistent-nodemap by default for Rust build
Differential Revision: https://phab.mercurial-scm.org/D10498
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 20 Apr 2021 16:18:43 +0200] rev 46999
relnode: document the move to `zstd` as default compression
Differential Revision: https://phab.mercurial-scm.org/D10497
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 20 Apr 2021 16:12:19 +0200] rev 46998
relnote: clarify the backward compatibility entry about p1/p2 swap
The previous message might read a bit scary. So we clarify that the nodeid are
not affected.
Differential Revision: https://phab.mercurial-scm.org/D10496
Matt Harbison <matt_harbison@yahoo.com> [Tue, 20 Apr 2021 23:31:36 -0400] rev 46997
mail: add a TODO about proper mbox locking
Per @nbjoerg in IRC, the mbox isn't locked correctly.
Differential Revision: https://phab.mercurial-scm.org/D10505
Joerg Sonnenberger <joerg@bec.de> [Tue, 20 Apr 2021 20:35:46 +0200] rev 46996
packaging: extract pre-computed version when running from plain tarball
The tarballs created by setup.py are not including the usual archive
markers that `hg archive` leaves, so the rewrite of the version number
computation actually left the empty version string around. This meant
that installations from PyPI would use 0.0.0 as version string.
Differential Revision: https://phab.mercurial-scm.org/D10502
Augie Fackler <raf@durin42.com> [Tue, 20 Apr 2021 11:12:20 -0400] rev 46995
Added signature for changeset f67b8946bb1b
Augie Fackler <raf@durin42.com> [Tue, 20 Apr 2021 11:12:19 -0400] rev 46994
Added tag 5.8rc0 for changeset f67b8946bb1b
Augie Fackler <augie@google.com> [Tue, 20 Apr 2021 11:01:06 -0400] rev 46993
merge: default into stable for 5.8 rc
Matt Harbison <matt_harbison@yahoo.com> [Mon, 19 Apr 2021 20:38:52 -0400] rev 46992
mail: force SMTPException to bytes before wrapping in error.Abort
Pytype wasn't complaining about this for some reason, but PyCharm was and we
already handle another instance of this exception the same way in this function.
Differential Revision: https://phab.mercurial-scm.org/D10473
Matt Harbison <matt_harbison@yahoo.com> [Mon, 19 Apr 2021 17:26:57 -0400] rev 46991
mail: split out the SMTP login to allow the keyring extension to wrap it
The keyring extension only needs to tweak this tiny section of the larger
function. But without any place to intercept the username/password fetching, it
copy/pasted the entire function, and has grown a bunch of compatibility hacks to
support older versions of Mercurial as well.
Differential Revision: https://phab.mercurial-scm.org/D10471
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 20 Apr 2021 04:27:03 +0200] rev 46990
store: exclude `undo.` nodemap's file from `walk`
There are "temporary" local file that we should not be transfered by `walk` user
like local clone and stream clone.
This fix the small issue that the new tests highlighted.
Differential Revision: https://phab.mercurial-scm.org/D10482
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 20 Apr 2021 05:08:55 +0200] rev 46989
streamclone: treat volatile file as "fullfile"
The nodemap related file might change (or get deleted) during the stream clone
in a way incompatible with the streaming process.
So we introduce a new flag for this type of file and integrate it with the
existing `revlog_type` field recently added to `store.walk` returns.
We use that new flat to dispatch such file to the existing mechanism for "atomic
replacement" file for the nodemap docket and datafile. This fix the bugs we have
been adding tests for.
Strictly speaking, the nodemap datafile is happened only a could maybe be used
in a slightly more efficient way, however this is good enough for now.
Differential Revision: https://phab.mercurial-scm.org/D10481
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 19 Apr 2021 20:47:46 +0200] rev 46988
nodemap: add a test about nodemap "vacuum" during stream clone
When the nodemap "vacuum" it generate a new datafile, with a new unique name and
delete the old one. This confuse the stream clone code and create the same kind
of crash we were seeing in the previous patch.
We build a test case where this happens. The next changeset will contains a fix
We can also notice that we are, wrongfully exchanging `undo.*` files that the
client will not be able to use. This will be dealt with soon.
Differential Revision: https://phab.mercurial-scm.org/D10480
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 19 Apr 2021 20:24:13 +0200] rev 46987
nodemap: add a test about racy commit during stream clone
That test show that the resulting client nodemap is different from the server
one. This happens because the server one transferred a corrupted node map. The
data file match the pre-commit content while the docket has post commit content.
As the result the nodemap was detected invalid and recomputed.
When running without the rust implementation, the code is also generating a new
datafile unconditionally, This mean the older file is no longer there are
transfer time, resulting in a crash.
We will fix this issue later, but we start with writing tests highlighting the
issue.
Differential Revision: https://phab.mercurial-scm.org/D10479
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 19 Apr 2021 19:12:28 +0200] rev 46986
streamclone: remove sleep based "synchronisation" in tests
Sleep based test synchronisation does not work.
Variation in machine performance and load can make the two process miss their
windows. Instead we migrate to explicit signaling through the file system as
other tests file are using.
Differential Revision: https://phab.mercurial-scm.org/D10478
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 19 Apr 2021 19:10:49 +0200] rev 46985
testing: add a `write_file` function
The function is kinda trivial, but having a simple function avoid silly mistake.
Differential Revision: https://phab.mercurial-scm.org/D10477
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 19 Apr 2021 19:10:18 +0200] rev 46984
testing: add a utility function to wait for file create
This is similar to `tests/testlib/wait-on-file`, but for the python code
Differential Revision: https://phab.mercurial-scm.org/D10476
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 19 Apr 2021 19:09:18 +0200] rev 46983
streamclone: check the errors log in tests
Code can be buggy, in that case having error message tend to help.
Differential Revision: https://phab.mercurial-scm.org/D10475
Matt Harbison <matt_harbison@yahoo.com> [Tue, 20 Apr 2021 00:23:56 -0400] rev 46982
mail: use a context manager when writing to mbox
Differential Revision: https://phab.mercurial-scm.org/D10484
Martin von Zweigbergk <martinvonz@google.com> [Mon, 19 Apr 2021 21:31:24 -0700] rev 46981
errors: remove unnecessary varargs handling from OutOfBandError
In my recent D10465, I moved some code over from scmutil into
`OutOfBandError.__init__`. The code was written to deal with an
arbitrary number of `message` arguments to the constructor. It turns
out that we only ever pass 0 or 1. Given that, let's simplify it.
Differential Revision: https://phab.mercurial-scm.org/D10483
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 19 Apr 2021 11:10:16 +0200] rev 46980
phab-refresh: use a special comment on the stable branch
Phabricator make it hard to spot the branch information. As a result patch
intended for stable are consistently queued for default.
We add a special command and flashy GIF to try to attract reviewer attention on
the fact patch are intended for stable.
Differential Revision: https://phab.mercurial-scm.org/D10464
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 19 Apr 2021 11:04:11 +0200] rev 46979
phab-refresh: extract the comment in a variable
This make the script more readable and it is about to become more complex.
Differential Revision: https://phab.mercurial-scm.org/D10463
Martin von Zweigbergk <martinvonz@google.com> [Mon, 19 Apr 2021 16:53:45 -0700] rev 46978
tests: make test-archive.t pass on py3.9 (issue6504)
Something got stricter at parsing URL query parameters and now the
parameters need to be separated by "&"; ";" is no longer allowed. See
issue6504 for details.
Differential Revision: https://phab.mercurial-scm.org/D10472
Martin von Zweigbergk <martinvonz@google.com> [Mon, 19 Apr 2021 12:52:23 -0700] rev 46977
errors: raise RemoteError in some places in exchange.py
This is part of https://www.mercurial-scm.org/wiki/ErrorCategoriesPlan
Differential Revision: https://phab.mercurial-scm.org/D10467
Martin von Zweigbergk <martinvonz@google.com> [Mon, 19 Apr 2021 09:37:01 -0700] rev 46976
errors: introduce a class for remote errors
Having an exception for remote errors makes it much easier to exit
with the right detailed exit code.
Differential Revision: https://phab.mercurial-scm.org/D10466
Martin von Zweigbergk <martinvonz@google.com> [Mon, 19 Apr 2021 10:49:15 -0700] rev 46975
errors: make OutOfBandError extend Abort
I'm about to create a new `RemoteError` exception and make
`OutOfBandError` extend it. This patch prepares for that.
Differential Revision: https://phab.mercurial-scm.org/D10465