Matt Harbison <matt_harbison@yahoo.com> [Mon, 05 Apr 2021 23:54:54 -0400] rev 46894
tests: skip test-git-interop.t on Windows
Casefolding isn't handled in dirstate yet, triggering a bunch of assertions.
But while this is more correctly `no-icasefs`, it's more likely to get attention
if someone sees it. I'd just rather not have it adding to the noise on Windows
for now.
Differential Revision: https://phab.mercurial-scm.org/D10312
Matt Harbison <matt_harbison@yahoo.com> [Mon, 05 Apr 2021 13:02:51 -0400] rev 46893
contrib: restore the `hg fix` configuration in the examples
After decc3bd3f20d, running `black` will DTRT, but running `hg fix` did nothing
(unless the example config file was %included, in which case it truncated the
file instead of formatting it). I'm not sure why that was happening, but let's
not leave a code shredder laying around.
Differential Revision: https://phab.mercurial-scm.org/D10311
Valentin Gatien-Baron <vgatien-baron@janestreet.com> [Wed, 31 Mar 2021 17:54:02 -0400] rev 46892
blackbox: fix type error on log rotation on read-only filesystem
Grepping around, the code uses either encoding.strtolocal or
stringutil.forcebytestr in this situation. No idea which is best.
Differential Revision: https://phab.mercurial-scm.org/D10293
Simon Sapin <simon.sapin@octobus.net> [Thu, 08 Apr 2021 14:38:27 +0200] rev 46891
rust: Remove use of `py.eval()`
The previous Rust code allocated an intermediate `Vec`, converted that
to a Python list, then used `eval` to run Python code that converts that
list to a Python set.
rust-cpython exposes Rust bindings for Python sets, let’s use that instead
to construct a set directly.
Differential Revision: https://phab.mercurial-scm.org/D10328
Simon Sapin <simon.sapin@octobus.net> [Thu, 08 Apr 2021 21:46:54 +0200] rev 46890
rust: Remove the compile-time 'dirstate-tree' feature flag
This code has compiler errors since it is not built on CI and nobody has been
working on it for some time.
We (Octobus) are still pursuing status optimizations based on a tree data
structure for the dirstate, but upcoming patches will use a run-time opt-in
instead of compile-time, so that at least corresponding Rust code keeps
compiling when other changes are made.
Differential Revision: https://phab.mercurial-scm.org/D10329
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> [Sun, 13 Sep 2020 22:14:25 -0400] rev 46889
procutil: avoid using os.fork() to implement runbgcommand
We ran into the following deadlock:
- some command creates an ssh peer, then raises without explicitly
closing the peer (hg id + extension in our case)
- dispatch catches the exception, calls ui.log('commandfinish', ..)
(the sshpeer is still not closed), which calls logtoprocess, which
calls procutil.runbgcommand.
- in the child of runbgcommand's fork(), between the fork and the
exec, the opening of file descriptors triggers a gc which runs the
destructor for sshpeer, which waits on ssh's stderr being closed,
which never happens since ssh's stderr is held open by the parent of
the fork where said destructor hasn't run
Remotefilelog appears to have a hack around this deadlock as well.
I don't know if there's more subtlety to it, because even though the
problem is determistic, it is very fragile, so I didn't manage to
reduce it.
I can imagine three ways of tackling this problem:
1. don't run any python between fork and exec in runbgcommand
2. make the finalizer harmless after the fork
3. close the peer without relying on gc behavior
This commit goes with 1, as forking without exec'ing is tricky in
general in a language with gc finalizers. And maybe it's better in the
presence of rust threads. A future commit will try 2 or 3.
Performance wise: at low memory usage, it's an improvement. At higher
memory usage, it's about 2x faster than before when ensurestart=True,
but 2x slower when ensurestart=False. Not sure if that matters. The
reason for that last bit is that the subprocess.Popen always waits for
the execve to finish, and at high memory usage, execve is slow because
it deallocates the large page table. Numbers and script:
before after
mem=1.0GB, ensurestart=True 52.1ms 26.0ms
mem=1.0GB, ensurestart=False 14.7ms 26.0ms
mem=0.5GB, ensurestart=True 23.2ms 11.2ms
mem=0.5GB, ensurestart=False 6.2ms 11.3ms
mem=0.2GB, ensurestart=True 15.7ms 7.4ms
mem=0.2GB, ensurestart=False 4.3ms 8.1ms
mem=0.0GB, ensurestart=True 2.3ms 0.7ms
mem=0.0GB, ensurestart=False 0.8ms 0.8ms
import time
for memsize in [1_000_000_000, 500_000_000, 250_000_000, 0]:
mem = 'a' * memsize
for ensurestart in [True, False]:
now = time.time()
n = 100
for i in range(n):
procutil.runbgcommand([b'true'], {}, ensurestart=ensurestart)
after = time.time()
ms = (after - now) / float(n) * 1000
print(f'mem={memsize / 1e9:.1f}GB, ensurestart={ensurestart} -> {ms:.1f}ms')
Differential Revision: https://phab.mercurial-scm.org/D9019