Thu, 25 Mar 2021 20:22:00 -0400 util: fix the signature for the pypy override of sortdict.update()
Matt Harbison <matt_harbison@yahoo.com> [Thu, 25 Mar 2021 20:22:00 -0400] rev 46900
util: fix the signature for the pypy override of sortdict.update() PyCharm flagged this as not matching the base class signature. Not sure if there was anything supplying these extra arguments though. Differential Revision: https://phab.mercurial-scm.org/D10275
Thu, 25 Mar 2021 18:59:14 -0400 typing: add type hints to mercurial/error.py
Matt Harbison <matt_harbison@yahoo.com> [Thu, 25 Mar 2021 18:59:14 -0400] rev 46899
typing: add type hints to mercurial/error.py The only slightly unusual things here are that `location` is passed to `ParseError` and both bytes and an int (so this accepts both), and the message passed `ProgrammingError` is immediately converted to str. Therefore it is typed as `AnyStr`, because there are a couple of instances that are already passed as str. There are a couple of places where bytes are being passed to builtin exceptions that might need to be converted to str. Differential Revision: https://phab.mercurial-scm.org/D10274
Fri, 19 Mar 2021 00:36:26 -0400 tests: add a (very slow) test that executes pytype
Matt Harbison <matt_harbison@yahoo.com> [Fri, 19 Mar 2021 00:36:26 -0400] rev 46898
tests: add a (very slow) test that executes pytype This is an updated form of D7295, and completes successfully with pytype 2021.03.22. The 5 or so crashes that were mostly in the hgweb files seems to have been fixed in 2021.03.10. Differential Revision: https://phab.mercurial-scm.org/D10237
Tue, 06 Apr 2021 10:38:27 +0200 upgrade: do not hardcore file extension of revlogs
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 06 Apr 2021 10:38:27 +0200] rev 46897
upgrade: do not hardcore file extension of revlogs This logic already lives inside the `store` module. So lets reuse it instead. Differential Revision: https://phab.mercurial-scm.org/D10317
Tue, 06 Apr 2021 10:38:11 +0200 upgrade: take advantage of the new information returned by `store.walk`
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 06 Apr 2021 10:38:11 +0200] rev 46896
upgrade: take advantage of the new information returned by `store.walk` Before this change the upgrade code had to analyse filename to process them directly. Lets keep that logic private to the store and more to a more robust explicit approach. Differential Revision: https://phab.mercurial-scm.org/D10316
Tue, 06 Apr 2021 10:38:03 +0200 store: also return some information about the type of file `walk` found
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 06 Apr 2021 10:38:03 +0200] rev 46895
store: also return some information about the type of file `walk` found We start returning of 4th information in the `store.walk` return tuple: the type of the file. This will make it easier for caller to determine which kind of file they are looking at. This should especically help with the `upgrade-repo` code that has to do a lot of fragile index's file name comparison. Differential Revision: https://phab.mercurial-scm.org/D10315
Mon, 05 Apr 2021 23:54:54 -0400 tests: skip test-git-interop.t on Windows
Matt Harbison <matt_harbison@yahoo.com> [Mon, 05 Apr 2021 23:54:54 -0400] rev 46894
tests: skip test-git-interop.t on Windows Casefolding isn't handled in dirstate yet, triggering a bunch of assertions. But while this is more correctly `no-icasefs`, it's more likely to get attention if someone sees it. I'd just rather not have it adding to the noise on Windows for now. Differential Revision: https://phab.mercurial-scm.org/D10312
Mon, 05 Apr 2021 13:02:51 -0400 contrib: restore the `hg fix` configuration in the examples
Matt Harbison <matt_harbison@yahoo.com> [Mon, 05 Apr 2021 13:02:51 -0400] rev 46893
contrib: restore the `hg fix` configuration in the examples After decc3bd3f20d, running `black` will DTRT, but running `hg fix` did nothing (unless the example config file was %included, in which case it truncated the file instead of formatting it). I'm not sure why that was happening, but let's not leave a code shredder laying around. Differential Revision: https://phab.mercurial-scm.org/D10311
Wed, 31 Mar 2021 17:54:02 -0400 blackbox: fix type error on log rotation on read-only filesystem
Valentin Gatien-Baron <vgatien-baron@janestreet.com> [Wed, 31 Mar 2021 17:54:02 -0400] rev 46892
blackbox: fix type error on log rotation on read-only filesystem Grepping around, the code uses either encoding.strtolocal or stringutil.forcebytestr in this situation. No idea which is best. Differential Revision: https://phab.mercurial-scm.org/D10293
Thu, 08 Apr 2021 14:38:27 +0200 rust: Remove use of `py.eval()`
Simon Sapin <simon.sapin@octobus.net> [Thu, 08 Apr 2021 14:38:27 +0200] rev 46891
rust: Remove use of `py.eval()` The previous Rust code allocated an intermediate `Vec`, converted that to a Python list, then used `eval` to run Python code that converts that list to a Python set. rust-cpython exposes Rust bindings for Python sets, let’s use that instead to construct a set directly. Differential Revision: https://phab.mercurial-scm.org/D10328
Thu, 08 Apr 2021 21:46:54 +0200 rust: Remove the compile-time 'dirstate-tree' feature flag
Simon Sapin <simon.sapin@octobus.net> [Thu, 08 Apr 2021 21:46:54 +0200] rev 46890
rust: Remove the compile-time 'dirstate-tree' feature flag This code has compiler errors since it is not built on CI and nobody has been working on it for some time. We (Octobus) are still pursuing status optimizations based on a tree data structure for the dirstate, but upcoming patches will use a run-time opt-in instead of compile-time, so that at least corresponding Rust code keeps compiling when other changes are made. Differential Revision: https://phab.mercurial-scm.org/D10329
Sun, 13 Sep 2020 22:14:25 -0400 procutil: avoid using os.fork() to implement runbgcommand
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> [Sun, 13 Sep 2020 22:14:25 -0400] rev 46889
procutil: avoid using os.fork() to implement runbgcommand We ran into the following deadlock: - some command creates an ssh peer, then raises without explicitly closing the peer (hg id + extension in our case) - dispatch catches the exception, calls ui.log('commandfinish', ..) (the sshpeer is still not closed), which calls logtoprocess, which calls procutil.runbgcommand. - in the child of runbgcommand's fork(), between the fork and the exec, the opening of file descriptors triggers a gc which runs the destructor for sshpeer, which waits on ssh's stderr being closed, which never happens since ssh's stderr is held open by the parent of the fork where said destructor hasn't run Remotefilelog appears to have a hack around this deadlock as well. I don't know if there's more subtlety to it, because even though the problem is determistic, it is very fragile, so I didn't manage to reduce it. I can imagine three ways of tackling this problem: 1. don't run any python between fork and exec in runbgcommand 2. make the finalizer harmless after the fork 3. close the peer without relying on gc behavior This commit goes with 1, as forking without exec'ing is tricky in general in a language with gc finalizers. And maybe it's better in the presence of rust threads. A future commit will try 2 or 3. Performance wise: at low memory usage, it's an improvement. At higher memory usage, it's about 2x faster than before when ensurestart=True, but 2x slower when ensurestart=False. Not sure if that matters. The reason for that last bit is that the subprocess.Popen always waits for the execve to finish, and at high memory usage, execve is slow because it deallocates the large page table. Numbers and script: before after mem=1.0GB, ensurestart=True 52.1ms 26.0ms mem=1.0GB, ensurestart=False 14.7ms 26.0ms mem=0.5GB, ensurestart=True 23.2ms 11.2ms mem=0.5GB, ensurestart=False 6.2ms 11.3ms mem=0.2GB, ensurestart=True 15.7ms 7.4ms mem=0.2GB, ensurestart=False 4.3ms 8.1ms mem=0.0GB, ensurestart=True 2.3ms 0.7ms mem=0.0GB, ensurestart=False 0.8ms 0.8ms import time for memsize in [1_000_000_000, 500_000_000, 250_000_000, 0]: mem = 'a' * memsize for ensurestart in [True, False]: now = time.time() n = 100 for i in range(n): procutil.runbgcommand([b'true'], {}, ensurestart=ensurestart) after = time.time() ms = (after - now) / float(n) * 1000 print(f'mem={memsize / 1e9:.1f}GB, ensurestart={ensurestart} -> {ms:.1f}ms') Differential Revision: https://phab.mercurial-scm.org/D9019
Thu, 08 Apr 2021 18:43:08 -0400 share: store relative share paths with '/' separators
Matt Harbison <matt_harbison@yahoo.com> [Thu, 08 Apr 2021 18:43:08 -0400] rev 46888
share: store relative share paths with '/' separators I created a relative share in Windows and tried to use it in WSL, and it failed: abort: .hg/sharedpath points to nonexistent directory /mnt/c/Users/Matt/hg-review/.hg/..\..\hg\.hg Use `normpath` on the read side so that the code has the usual Windows style paths it always had (I don't think that matters much), but it also eliminates the directory escaping path components in the case where the path is printed. This will not fix repositories that have already been created, but it's trivial enough to hand edit the file to correct it. Differential Revision: https://phab.mercurial-scm.org/D10330
Fri, 09 Apr 2021 12:02:51 +0200 unit-tests: Fix `cargo test` on 32-bit platforms
Simon Sapin <simon.sapin@octobus.net> [Fri, 09 Apr 2021 12:02:51 +0200] rev 46887
unit-tests: Fix `cargo test` on 32-bit platforms Fixes https://bz.mercurial-scm.org/show_bug.cgi?id=6506 This makes `IndexEntryBuilder::build`, which is only used in unit tests, use `u32` or `u64` instead of platform-dependent `usize` when packing binary data to be used at test input. To run Rust unit tests in 32-bit mode in a x86-64 environment, use: rustup target add i686-unknown-linux-gnu # Once (cd rust && cargo test --target i686-unknown-linux-gnu) Differential Revision: https://phab.mercurial-scm.org/D10351
Fri, 09 Apr 2021 08:46:40 -0700 rename: add --forget option and stop suggesting `hg revert` for undoing
Martin von Zweigbergk <martinvonz@google.com> [Fri, 09 Apr 2021 08:46:40 -0700] rev 46886
rename: add --forget option and stop suggesting `hg revert` for undoing Differential Revision: https://phab.mercurial-scm.org/D10355
Fri, 09 Apr 2021 11:32:19 -0400 win32: enable legacy I/O mode to fix missing pager output on Windows with py3
Matt Harbison <matt_harbison@yahoo.com> [Fri, 09 Apr 2021 11:32:19 -0400] rev 46885
win32: enable legacy I/O mode to fix missing pager output on Windows with py3 The equivalent interpreter option is set by wrapper.exe, but this *.bat file is what gets installed in a venv. Without this mode, any command that spins up a pager has no output, unless the pager is explicitly disabled. The variable is set inside the `setlocal` scope to keep it from leaking into the environment after the bat file exits. We should probably still figure out how to ship a compiled hg.exe when installing with `pip`, because the binary does other things like enable long filename support. But this avoids the dangerous and confusing lack of output in the meantime. Differential Revision: https://phab.mercurial-scm.org/D10354
Thu, 14 Jan 2021 04:58:20 +0100 persistent-nodemap: enable the feature by default when using Rust
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 14 Jan 2021 04:58:20 +0100] rev 46884
persistent-nodemap: enable the feature by default when using Rust As discussed at the 5.6 sprint, we can make it enabled by default, but only for Rust installation. Differential Revision: https://phab.mercurial-scm.org/D9765
Fri, 09 Apr 2021 03:36:24 +0200 persistent-nodemap: disable it for test-wireproto-content-redirects.t
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 09 Apr 2021 03:36:24 +0200] rev 46883
persistent-nodemap: disable it for test-wireproto-content-redirects.t This is not relevant for the test and this will avoid a lot of variations. Differential Revision: https://phab.mercurial-scm.org/D10338
Fri, 09 Apr 2021 03:30:18 +0200 persistent-nodemap: disable it for test-wireproto-command-capabilities.t
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 09 Apr 2021 03:30:18 +0200] rev 46882
persistent-nodemap: disable it for test-wireproto-command-capabilities.t This is not relevant for the test and this will avoid a lot of variations. Differential Revision: https://phab.mercurial-scm.org/D10337
Fri, 09 Apr 2021 03:36:07 +0200 persistent-nodemap: disable it unconditionally for test-wireproto-caching.t
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 09 Apr 2021 03:36:07 +0200] rev 46881
persistent-nodemap: disable it unconditionally for test-wireproto-caching.t This is not relevant for the test and this will avoid a lot of variations. Differential Revision: https://phab.mercurial-scm.org/D10336
Fri, 09 Apr 2021 03:51:25 +0200 persistent-nodemap: disable it unconditionally for test-ssh-proto-unbundle.t
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 09 Apr 2021 03:51:25 +0200] rev 46880
persistent-nodemap: disable it unconditionally for test-ssh-proto-unbundle.t This is not relevant for the test and this will avoid a lot of variations. Differential Revision: https://phab.mercurial-scm.org/D10335
Fri, 09 Apr 2021 03:51:35 +0200 persistent-nodemap: disable it unconditionally for test-ssh-proto.t
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 09 Apr 2021 03:51:35 +0200] rev 46879
persistent-nodemap: disable it unconditionally for test-ssh-proto.t This is not relevant for the test and this will avoid a lot of variations. Differential Revision: https://phab.mercurial-scm.org/D10334
Fri, 09 Apr 2021 04:13:07 +0200 persistent-nodemap: disable it unconditionally for test-share-safe.t
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 09 Apr 2021 04:13:07 +0200] rev 46878
persistent-nodemap: disable it unconditionally for test-share-safe.t This is not relevant for the test and this will avoid a lot of variations. Differential Revision: https://phab.mercurial-scm.org/D10333
Fri, 09 Apr 2021 03:44:45 +0200 persistent-nodemap: disable it unconditionally for test-http-protocol.t
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 09 Apr 2021 03:44:45 +0200] rev 46877
persistent-nodemap: disable it unconditionally for test-http-protocol.t This is not relevant for the test and this will avoid a lot of variations. Differential Revision: https://phab.mercurial-scm.org/D10332
Fri, 09 Apr 2021 03:15:07 +0200 persistent-nodemap: disable it unconditionally for test-http-bad-server.t
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 09 Apr 2021 03:15:07 +0200] rev 46876
persistent-nodemap: disable it unconditionally for test-http-bad-server.t This is not relevant for the test and this will avoid a lot of variations Differential Revision: https://phab.mercurial-scm.org/D10331
Thu, 08 Apr 2021 20:05:59 +0200 clang-format: run the formatter on mercurial/cext/revlog.c
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 08 Apr 2021 20:05:59 +0200] rev 46875
clang-format: run the formatter on mercurial/cext/revlog.c This fix `test-check-clang-format.t` that has been complaining for a while. Differential Revision: https://phab.mercurial-scm.org/D10327
Tue, 06 Apr 2021 18:55:19 +0200 revlog-compression: use zstd by default (if available)
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 06 Apr 2021 18:55:19 +0200] rev 46874
revlog-compression: use zstd by default (if available) As see in changeset bb271ec2fbfb, zstd is 20% to 50% faster for reading and writing. Use take advantage of the new config behavior to try zstd by default, falling back to zlib is zstd is not available on that plateform. Differential Revision: https://phab.mercurial-scm.org/D10326
Wed, 07 Apr 2021 13:31:15 +0200 rhg: make rhg recognise it supports zstd compression for revlogs
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 07 Apr 2021 13:31:15 +0200] rev 46873
rhg: make rhg recognise it supports zstd compression for revlogs It already did, but was not aware of it. Differential Revision: https://phab.mercurial-scm.org/D10324
(0) -30000 -10000 -3000 -1000 -300 -100 -50 -28 +28 +50 +100 +300 +1000 +3000 tip