nodemap: also warm manifest nodemap with other caches
The `hg debugupdatecache` command now also warm the persistent nodemap for the
manifest (when applicable).
Differential Revision: https://phab.mercurial-scm.org/D8411
nodemap: also use persistent nodemap for manifest
The manifest as a different usage pattern than the changelog. First, while the
lookup in changelog are not garanteed to match, the lookup in the manifest
nodemap come from changelog and will exist in the manifest. In addition, looking
up a manifest almost always result in unpacking a manifest an operation that
rarely come cheap.
Nevertheless, using a persistent nodemap provide a significant gain for some
operations.
For our measurementw, we use `hg cat --rev REV FILE` on the our reference
mozilla-try. On this repository the persistent nodemap cache is about 29 MB in
side for a total store side of 11,988 MB
File with large history (file: b2g/config/gaia.json, revision:
195a1146daa0)
no optimisation: 0.358s
using mmap for index: 0.297s (-0.061s)
persistent nodemap for changelog only: 0.275s (-0.024s)
persistent nodemap for manifest too: 0.258s (-0.017s)
File with small history (file: .hgignore, revision:
195a1146daa0)
no optimisation: 0.377s
using mmap for index: 0.296s (-0.061s)
persistent nodemap for changelog only: 0.274s (-0.022s)
persistent nodemap for manifest too: 0.257s (-0.017s)
Same file but using a revision (
8ba995b74e18) with a smaller manifest (3944829
bytes vs 10 bytes)
no optimisation: 0.192s (-0.185s)
using mmap for index: 0.131s (-0.061s)
persistent nodemap for changelog only: 0.106s (-0.025s)
persistent nodemap for manifest too: 0.087s (-0.019s)
Differential Revision: https://phab.mercurial-scm.org/D8410
nodemap: create files in the repository used in the test
We need a manifest with more content to test persistent nodemap for manifest.
This change the repository content and affect all the hashes.
Differential Revision: https://phab.mercurial-scm.org/D8409
rust-matchers: add timing tracing to regex compilation
This might be useful to diagnose later performance issues or just to show
the difference between engines.
Differential Revision: https://phab.mercurial-scm.org/D8498
url: fix a bytes vs str crash in processing proxy headers (
issue6249)
I have no idea how to make a test for this, so if somebody knows, feel free to
add one or follow up on this. The bug reporter reported that it worked for
them, so there may not be other hidden issues here.
Differential Revision: https://phab.mercurial-scm.org/D8485
pullbundles: use unfiltered repo for head/base matching
The unfiltered view works even when changeset transistion from draft to
hidden phase. The normal visibility is already ensured by discovery as
invisible heads would have been filtered out before. Skipping the
filtering has a positive impact on performance, too.
Differential Revision: https://phab.mercurial-scm.org/D8481
procutil: always waiting on child processes to prevent zombies with 'hg serve'
When runbgcommand is invoked by an extension with ensurestart=False, we never
called waitpid - which is fine in most cases, except if that's happening on a
command server (e.g. chg), in which case the child defunct process will just
sit there for as long as the server is running.
The actual semantics of SIGCHLD signal handling is a lot more complex than
it seems, and the POSIX standard *seems* to read that it's ignored by default
and everything would just work without the waitpid if we're not listening for
it, but the truth is that it's only ignored if we *explicitly* set it to
SIG_IGN. We further cannot set it to SIG_IGN or to a catch-all handler across
all of 'hg serve', because Python's suprocess.Popen relies on that signal,
and a few specific parts of hg also set custom handlers, so instead we wait
for specific PIDs in dedicated threads.
I did a poor-man's benchmark of the thread creation and it seems to take
about 1ms, which is way better than the 20+ms from ensurestart=True.
Differential Revision: https://phab.mercurial-scm.org/D8497
tests: use regular POSIX shell
wait-on-file requires one POSIX extension (sleep with non-integral
argument), but it doesn't require any bash extensions, so just require a
normal POSIX shell. While here, use consistent formatting without
redundant ;
Differential Revision: https://phab.mercurial-scm.org/D8500
rust-regex: increase the DFA size limit for the `regex` crate
`re2`'s DFA limit is already increased in `rust/hg-core/src/re2/rust_re2.cpp`,
the same has to be done for the `regex` crate.
Big repositories with big `.hgignore`s will sometimes hit this limit and face
extreme performance regressions (I've seen one take *minutes* for `hg status`).
Differential Revision: https://phab.mercurial-scm.org/D8499