Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 13 Mar 2024 12:02:06 +0100] rev 51599
tags-cache: directly perform a monimal walk for hgtagsfnodescache warming
We do something narrower than the path retrieving data. So lets use dedicated
code instead.
This provides further useful speedup:
### data-env-vars.name = mozilla-try-2023-03-22-zstd-sparse-revlog
# benchmark.name = hg.debug.debug-update-cache
# bin-env-vars.hg.flavor = default
# bin-env-vars.hg.py-re2-module = default
# benchmark.variants.pre-state = warm
before-this-series: 19.947581
skip-fnode-filter: 18.916804 (-5.17%, -1.03)
use-rev-num: 17.493725 (-12.30%, -2.45)
this-changesets: 15.919466 (-20.19%, -4.03)
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 13 Mar 2024 11:51:11 +0100] rev 51598
tags-cache: directly operate on rev-num warming hgtagsfnodescache
Not having to goes through nodeid speed up things notably.
### data-env-vars.name = mozilla-try-2023-03-22-zstd-sparse-revlog
# benchmark.name = hg.debug.debug-update-cache
# bin-env-vars.hg.flavor = default
# bin-env-vars.hg.py-re2-module = default
# benchmark.variants.pre-state = warm
before-this-series: 19.947581
before-this-changes: 18.916804 (-5.17%, -1.03)
this-changesets: 17.493725 (-12.30%, -2.45)
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 13 Mar 2024 11:38:28 +0100] rev 51597
tags-cache: skip the filternode step if we are not going to use it
When warming the hgtagsfnodescache, we don't need the actual result, so we can
simply skip the part that "filter" fnode we read from the cache. So provide a
quite visible speed up to the top level `hg debugupdatecache` function.
### data-env-vars.name = mozilla-try-2023-03-22-zstd-sparse-revlog
# benchmark.name = hg.debug.debug-update-cache
# bin-env-vars.hg.flavor = default
# bin-env-vars.hg.py-re2-module = default
# benchmark.variants.pre-state = warm
before: 19.947581
after: 18.916804 (-5.17%, -1.03)
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 13 Mar 2024 11:34:21 +0100] rev 51596
tags-cache: add a dedicated warm cache function to hgtagsfnodescache
Having a dedicated API point will help to optimize that specific usage. Right
doing a full phases weam takes a long time, even when the cache is already
filled.
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 09 Apr 2024 22:37:15 +0200] rev 51595
outgoing: add a simple fastpath when there is no common
This further speed up case like `hg bundle --all` for larger repository.
### data-env-vars.name = mozilla-try-2023-03-22-zstd-sparse-revlog
# benchmark.name = hg.command.bundle
# bin-env-vars.hg.flavor = default
# bin-env-vars.hg.py-re2-module = default
# benchmark.variants.revs = all
# benchmark.variants.type = none-streamv2
before: 316.749699
after: 311.165461 (-1.76%, -5.58)
There is further work to be done in this area like not doing any outgoing
computation in the stream case for example. however the recent changes already
gives use a large win for a small amount of local work.
### benchmark.name = hg.command.bundle
# bin-env-vars.hg.flavor = default
# bin-env-vars.hg.py-re2-module = default
# benchmark.variants.revs = all
# benchmark.variants.type = none-streamv2
## data-env-vars.name = mercurial-public-2024-03-22-zstd-sparse-revlog
pre-%ln-change: 1.263859
the-%ln-change: 0.700229 (-44.60%, -0.56)
prev-changeset: 0.496050 (-60.75%, -0.77)
this-changeset: 0.495243 (-60.81%, -0.77)
## data-env-vars.name = tryton-public-2024-03-22-zstd-sparse-revlog
pre-%ln-change: 2.975765
the-%ln-change: 1.870798 (-37.13%, -1.10)
prev-changeset: 1.461583 (-50.88%, -1.51)
this-changeset: 1.469185 (-50.63%, -1.51)
## data-env-vars.name = pypy-2024-03-22-zstd-sparse-revlog
pre-%ln-change: 4.540080
the-%ln-change: 3.401700 (-25.07%, -1.14)
prev-changeset: 2.915810 (-35.78%, -1.62)
this-changeset: 2.911643 (-35.87%, -1.63)
## data-env-vars.name = heptapod-public-2024-03-25-zstd-sparse-revlog
pre-%ln-change: 10.138396
the-%ln-change: 7.750458 (-23.55%, -2.39)
prev-changeset: 6.665565 (-34.25%, -3.47)
this-changeset: 6.672078 (-34.19%, -3.47)
## data-env-vars.name = mozilla-try-2023-03-22-zstd-sparse-revlog
pre-%ln-change: 399.484481
the-%ln-change: 346.508952 (-13.26%, -52.98)
prev-changeset: 316.749699 (-20.71%, -82.73)
this-changeset: 311.165461 (-22.11%, -88.32)
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 09 Apr 2024 22:36:35 +0200] rev 51594
outgoing: rework the handling of the `missingroots` case to be faster
The previous implementation was slow, to the point it was taking a significant
amount of `hg bundle --type none-streamv2` call. We rework the code to compute
the same value much faster, making the operation disappear from the `hg bundle
--type none-streamv2` profile. Someone would remark that producing a streamclone
does not requires an `outgoing` object. However that is a matter for another
day. There is other user of `missingroots` (non stream `hg bundle` call for
example), and they will also benefit from this rework.
We implement an old TODO in the process, directly computing the missing and
common attribute as we have most element at hand already.
### benchmark.name = hg.command.bundle
# bin-env-vars.hg.flavor = default
# bin-env-vars.hg.py-re2-module = default
# benchmark.variants.revs = all
# benchmark.variants.type = none-streamv2
## data-env-vars.name = heptapod-public-2024-03-25-zstd-sparse-revlog
before: 7.750458
after: 6.665565 (-14.00%, -1.08)
## data-env-vars.name = mercurial-public-2024-03-22-zstd-sparse-revlog
before: 0.700229
after: 0.496050 (-29.16%, -0.20)
## data-env-vars.name = mozilla-try-2023-03-22-zstd-sparse-revlog
before: 346.508952
after: 316.749699 (-8.59%, -29.76)
## data-env-vars.name = pypy-2024-03-22-zstd-sparse-revlog
before: 3.401700
after: 2.915810 (-14.28%, -0.49)
## data-env-vars.name = tryton-public-2024-03-22-zstd-sparse-revlog
before: 1.870798
after: 1.461583 (-21.87%, -0.41)
note: this whole `missingroots` of outgoing has a limited number of callers and
could likely be replace by something simpler (like taking an explicit
"missing_revs" set for example). However this is a wider change and we focus on
a small impact, quick rework that does not change the API for now.