Thu, 08 Apr 2021 21:46:54 +0200 rust: Remove the compile-time 'dirstate-tree' feature flag
Simon Sapin <simon.sapin@octobus.net> [Thu, 08 Apr 2021 21:46:54 +0200] rev 46890
rust: Remove the compile-time 'dirstate-tree' feature flag This code has compiler errors since it is not built on CI and nobody has been working on it for some time. We (Octobus) are still pursuing status optimizations based on a tree data structure for the dirstate, but upcoming patches will use a run-time opt-in instead of compile-time, so that at least corresponding Rust code keeps compiling when other changes are made. Differential Revision: https://phab.mercurial-scm.org/D10329
Sun, 13 Sep 2020 22:14:25 -0400 procutil: avoid using os.fork() to implement runbgcommand
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> [Sun, 13 Sep 2020 22:14:25 -0400] rev 46889
procutil: avoid using os.fork() to implement runbgcommand We ran into the following deadlock: - some command creates an ssh peer, then raises without explicitly closing the peer (hg id + extension in our case) - dispatch catches the exception, calls ui.log('commandfinish', ..) (the sshpeer is still not closed), which calls logtoprocess, which calls procutil.runbgcommand. - in the child of runbgcommand's fork(), between the fork and the exec, the opening of file descriptors triggers a gc which runs the destructor for sshpeer, which waits on ssh's stderr being closed, which never happens since ssh's stderr is held open by the parent of the fork where said destructor hasn't run Remotefilelog appears to have a hack around this deadlock as well. I don't know if there's more subtlety to it, because even though the problem is determistic, it is very fragile, so I didn't manage to reduce it. I can imagine three ways of tackling this problem: 1. don't run any python between fork and exec in runbgcommand 2. make the finalizer harmless after the fork 3. close the peer without relying on gc behavior This commit goes with 1, as forking without exec'ing is tricky in general in a language with gc finalizers. And maybe it's better in the presence of rust threads. A future commit will try 2 or 3. Performance wise: at low memory usage, it's an improvement. At higher memory usage, it's about 2x faster than before when ensurestart=True, but 2x slower when ensurestart=False. Not sure if that matters. The reason for that last bit is that the subprocess.Popen always waits for the execve to finish, and at high memory usage, execve is slow because it deallocates the large page table. Numbers and script: before after mem=1.0GB, ensurestart=True 52.1ms 26.0ms mem=1.0GB, ensurestart=False 14.7ms 26.0ms mem=0.5GB, ensurestart=True 23.2ms 11.2ms mem=0.5GB, ensurestart=False 6.2ms 11.3ms mem=0.2GB, ensurestart=True 15.7ms 7.4ms mem=0.2GB, ensurestart=False 4.3ms 8.1ms mem=0.0GB, ensurestart=True 2.3ms 0.7ms mem=0.0GB, ensurestart=False 0.8ms 0.8ms import time for memsize in [1_000_000_000, 500_000_000, 250_000_000, 0]: mem = 'a' * memsize for ensurestart in [True, False]: now = time.time() n = 100 for i in range(n): procutil.runbgcommand([b'true'], {}, ensurestart=ensurestart) after = time.time() ms = (after - now) / float(n) * 1000 print(f'mem={memsize / 1e9:.1f}GB, ensurestart={ensurestart} -> {ms:.1f}ms') Differential Revision: https://phab.mercurial-scm.org/D9019
Thu, 08 Apr 2021 18:43:08 -0400 share: store relative share paths with '/' separators
Matt Harbison <matt_harbison@yahoo.com> [Thu, 08 Apr 2021 18:43:08 -0400] rev 46888
share: store relative share paths with '/' separators I created a relative share in Windows and tried to use it in WSL, and it failed: abort: .hg/sharedpath points to nonexistent directory /mnt/c/Users/Matt/hg-review/.hg/..\..\hg\.hg Use `normpath` on the read side so that the code has the usual Windows style paths it always had (I don't think that matters much), but it also eliminates the directory escaping path components in the case where the path is printed. This will not fix repositories that have already been created, but it's trivial enough to hand edit the file to correct it. Differential Revision: https://phab.mercurial-scm.org/D10330
(0) -30000 -10000 -3000 -1000 -300 -100 -30 -10 -3 +3 +10 +30 +100 +300 +1000 +3000 tip