Matt Harbison <matt_harbison@yahoo.com> [Mon, 07 Oct 2024 13:19:16 -0400] rev 51983
tests: skip `test-wsgicgi.t` on MSYS
The test is attempting to set `PATH_INFO="/rev/\xe2\x80\x94"` into the
environment, which it does. The problem is that when MSYS sees a leading '/' in
an environment variable, it thinks it's a unix filesystem path, so it "helpfully"
prepends the Windows path to the MSYS root directory before running a non-MSYS
process. hgweb would then split this value on '/', so it would get 'C:' instead
of 'rev', and return a 400 since that isn't a valid web command.
I tried generating a *.bat file, but had trouble running that via `cmd.exe`
inside the test. I also tried generating an equivalent *.py launcher that would
set the environment variables itself. But there is no `os.environb` on Windows,
and the value was getting mangled when put into the script. So, I give up. If
it's encoding stuff on Windows, it's probably broken.
Joerg Sonnenberger <joerg@bec.de> [Thu, 27 Jun 2024 03:32:52 +0200] rev 51982
clonebundles: allow manifest to specify sha256 digest of bundles
paugier <pierre.augier@univ-grenoble-alpes.fr> [Thu, 03 Oct 2024 14:45:01 +0200] rev 51981
install: add long_description_content_type
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 09 Oct 2024 10:56:51 +0200] rev 51980
headrevs: replace a boolean match with a if/else
I missed that while doing a previous cleanup.
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 27 Sep 2024 03:55:40 +0200] rev 51979
head-revs: add a native implementation of the `stop_rev` parameter
This does not add too much complexity to the native code and help with
branchmap v3 performance.
Note that the final conversion of the heads from native-code to Python is still
too costly, especially in Rust. In addition the current caching around headrevs
is too simple and fragile. However these are an unrelated problem.
### benchmark.name = hg.command.unbundle
# bin-env-vars.hg.py-re2-module = default
# benchmark.variants.
issue6528 = disabled
# benchmark.variants.resource-usage = default
# benchmark.variants.reuse-external-delta-parent = yes
# benchmark.variants.revs = any-1-extra-rev
# benchmark.variants.source = unbundle
# benchmark.variants.validate = default
# benchmark.variants.verbosity = quiet
## data-env-vars.name = netbeans-2018-08-01-zstd-sparse-revlog
# bin-env-vars.hg.flavor = default
branch-v2: 0.233711 ~~~~~
branch-v3 before: 0.239857 (+2.63%, +0.01)
branch-v3 after: 0.239558 (+2.50%, +0.01)
# bin-env-vars.hg.flavor = rust
branch-v2: 0.235230 ~~~~~
branch-v3 before: 0.240972 (+2.44%, +0.01)
branch-v3 after: 0.239917 (+1.99%, +0.00)
## data-env-vars.name = netbeans-2018-08-01-ds2-pnm
# bin-env-vars.hg.flavor = rust
branch-v2: 0.255586 ~~~~~
branch-v3 before: 0.268560 (+5.08%, +0.01)
branch-v3 after: 0.262261 (+2.61%, +0.01)
## data-env-vars.name = mozilla-central-2024-03-22-zstd-sparse-revlog
# bin-env-vars.hg.flavor = default
branch-v2: 0.339010 ~~~~~
branch-v3 before: 0.349389 (+3.06%, +0.01)
branch-v3 after: 0.348247 (+2.72%, +0.01)
# bin-env-vars.hg.flavor = rust
branch-v2: 0.346525 ~~~~~
branch-v3 before: 0.355661 (+2.64%, +0.01)
branch-v3 after: 0.350906 (+1.26%, +0.00)
## data-env-vars.name = mozilla-central-2024-03-22-ds2-pnm
# bin-env-vars.hg.flavor = rust
branch-v2: 0.380202 ~~~~~
branch-v3 before: 0.408851 (+7.54%, +0.03)
branch-v3 after: 0.406511 (+6.92%, +0.03)
## data-env-vars.name = mozilla-unified-2024-03-22-zstd-sparse-revlog
# bin-env-vars.hg.flavor = default
branch-v2: 0.412165 ~~~~~
branch-v3 before: 0.427782 (+3.79%, +0.02)
branch-v3 after: 0.422595 (+2.53%, +0.01)
# bin-env-vars.hg.flavor = rust
branch-v2: 0.412397 ~~~~~
branch-v3 before: 0.422354 (+2.41%, +0.01)
branch-v3 after: 0.421079 (+2.11%, +0.01)
## data-env-vars.name = mozilla-unified-2024-03-22-ds2-pnm
# bin-env-vars.hg.flavor = rust
branch-v2: 0.429501 ~~~~~
branch-v3 before: 0.443197 (+3.19%, +0.01)
branch-v3 after: 0.449432 (+4.64%, +0.02)
## data-env-vars.name = mozilla-try-2024-03-26-zstd-sparse-revlog
# bin-env-vars.hg.flavor = default
branch-v2: 3.403171 ~~~~~
branch-v3 before: 3.819477 (+12.23%, +0.42)
branch-v3 after: 3.658482 (+7.50%, +0.26)
# bin-env-vars.hg.flavor = rust
branch-v2: 3.454876 ~~~~~
branch-v3 before: 3.590284 (+3.92%, +0.14)
branch-v3 after: 3.545843 (+2.63%, +0.09)
## data-env-vars.name = mozilla-try-2024-03-26-ds2-pnm
# bin-env-vars.hg.flavor = rust
branch-v2: 3.465435 ~~~~~
branch-v3 before: 3.633278 (+4.84%, +0.17)
branch-v3 after: 3.556074 (+2.62%, +0.09)
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 26 Sep 2024 01:52:09 +0200] rev 51978
head-revs: move hg-core's inner_headrevsfiltered closer to inner_headrevs
This is pure code movement, it make a coming changesets significantly clearer.
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 03 Sep 2024 11:11:17 +0200] rev 51977
branchmap-v3: introduce a "stop_rev" argument to `headsrevs`
The `headsrevs` method of the revlog already have a `revs` argument to compute
the headrevs of a limited set of heads. However, it disable the use of the
native compiled code to compute the heads, which slows down the branchmap v3
code a lot.
The branchmap v3 usage is actually quite constrained as we will always only
ignores a part at the top of the graph. So we could be significantly faster.
We start by making small change to the python side to improve the situation and
introduce the new API. More collaboration with the native code are coming later.
This massively speedup operation and close most of the remaining gaps between
branchmap-v3 and branchmap-v2. especially on repository with many revs like
mozilla-try. A small overhead remains mostly because the `headrevs` logic
currently has some inefficiently. We will look into them from there.
### benchmark.name = hg.command.unbundle
# bin-env-vars.hg.py-re2-module = default
# benchmark.variants.
issue6528 = disabled
# benchmark.variants.resource-usage = default
# benchmark.variants.reuse-external-delta-parent = yes
# benchmark.variants.revs = any-1-extra-rev
# benchmark.variants.source = unbundle
# benchmark.variants.validate = default
# benchmark.variants.verbosity = quiet
## data-env-vars.name = netbeans-2018-08-01-zstd-sparse-revlog
# bin-env-vars.hg.flavor = default
branch-v2: 0.233711 ~~~~~
branch-v3 before: 0.368769 (+57.79%, +0.14)
branch-v3 after: 0.239857 (+2.63%, +0.01)
# bin-env-vars.hg.flavor = rust
branch-v2: 0.235230 ~~~~~
branch-v3 before: 0.372460 (+58.34%, +0.14)
branch-v3 after: 0.240972 (+2.44%, +0.01)
## data-env-vars.name = netbeans-2018-08-01-ds2-pnm
# bin-env-vars.hg.flavor = rust
branch-v2: 0.255586 ~~~~~
branch-v3 before: 0.318907 (+24.78%, +0.06)
branch-v3 after: 0.268560 (+5.08%, +0.01)
## data-env-vars.name = mozilla-central-2024-03-22-zstd-sparse-revlog
# bin-env-vars.hg.flavor = default
branch-v2: 0.339010 ~~~~~
branch-v3 before: 0.349752 (+3.17%, +0.01)
branch-v3 after: 0.349389 (+3.06%, +0.01)
# bin-env-vars.hg.flavor = rust
branch-v2: 0.346525 ~~~~~
branch-v3 before: 0.354300 (+2.24%, +0.01)
branch-v3 after: 0.355661 (+2.64%, +0.01)
## data-env-vars.name = mozilla-central-2024-03-22-ds2-pnm
# bin-env-vars.hg.flavor = rust
branch-v2: 0.380202 ~~~~~
branch-v3 before: 0.396293 (+4.23%, +0.02)
branch-v3 after: 0.408851 (+7.54%, +0.03)
## data-env-vars.name = mozilla-unified-2024-03-22-zstd-sparse-revlog
# bin-env-vars.hg.flavor = default
branch-v2: 0.412165 ~~~~~
branch-v3 before: 0.424769 (+3.06%, +0.01)
branch-v3 after: 0.427782 (+3.79%, +0.02)
# bin-env-vars.hg.flavor = rust
branch-v2: 0.412397 ~~~~~
branch-v3 before: 0.421796 (+2.28%, +0.01)
branch-v3 after: 0.422354 (+2.41%, +0.01)
## data-env-vars.name = mozilla-unified-2024-03-22-ds2-pnm
# bin-env-vars.hg.flavor = rust
branch-v2: 0.429501 ~~~~~
branch-v3 before: 0.443849 (+3.34%, +0.01)
branch-v3 after: 0.443197 (+3.19%, +0.01)
## data-env-vars.name = mozilla-try-2024-03-26-zstd-sparse-revlog
# bin-env-vars.hg.flavor = default
branch-v2: 3.403171 ~~~~~
branch-v3 before: 6.234055 (+83.18%, +2.83)
branch-v3 after: 3.819477 (+12.23%, +0.42)
# bin-env-vars.hg.flavor = rust
branch-v2: 3.454876 ~~~~~
branch-v3 before: 6.307813 (+82.58%, +2.85)
branch-v3 after: 3.590284 (+3.92%, +0.14)
## data-env-vars.name = mozilla-try-2024-03-26-ds2-pnm
# bin-env-vars.hg.flavor = rust
branch-v2: 3.465435 ~~~~~
branch-v3 before: 5.176076 (+49.36%, +1.71)
branch-v3 after: 3.633278 (+4.84%, +0.17)
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 03 Sep 2024 02:13:03 +0200] rev 51976
branchmap-v3: filter topo heads using node for performance reason
The branchmap currently contains heads as nodeid. If we build a set of revnum
with the topological heads, we need to turn the nodeid in the branchmap to revnum to
be able to check if they are topo-heads. That nodeid → revnum lookup is
"expensive" and adds up to something noticeable if you do it hundreds of thousand
of time.
Instead we turn all the topo-heads revnums into nodes and build a set. So we can
directly test membership of the nodeids stored in the branchmap. That is much
faster.
Ideally we would have revnum in the branchmap and could directly test revnum
against a revnum set and that would be even faster. However that's an adventure
for another time.
Without this change, the branchmap format "v3" was significantly slower than the
"v2" format. With this changes, some of that gap is recovered
With rust + persistent nodemap, this overhead was smaller because the extra
lookup did not had to to build the nodemap from scratch.
In addition the mozilla-unified repository is able to use the "pure_top" mode of
branchmap v3, so it was not really affected by this.
Future changeset will work of the remaining of the performance gap.
### benchmark.name = hg.command.unbundle
# bin-env-vars.hg.py-re2-module = default
# benchmark.variants.
issue6528 = disabled
# benchmark.variants.resource-usage = default
# benchmark.variants.reuse-external-delta-parent = yes
# benchmark.variants.revs = any-1-extra-rev
# benchmark.variants.source = unbundle
# benchmark.variants.validate = default
# benchmark.variants.verbosity = quiet
## data-env-vars.name = netbeans-2018-08-01-zstd-sparse-revlog
# bin-env-vars.hg.flavor = default
branch-v2: 0.233711 ~~~~~
branch-v3 before: 0.380994 (+63.02%, +0.15)
branch-v3 after: 0.368769 (+57.79%, +0.14)
# bin-env-vars.hg.flavor = rust
branch-v2: 0.235230 ~~~~~
branch-v3 before: 0.385060 (+63.70%, +0.15)
branch-v3 after: 0.372460 (+58.34%, +0.14)
## data-env-vars.name = netbeans-2018-08-01-ds2-pnm
# bin-env-vars.hg.flavor = rust
branch-v2: 0.255586 ~~~~~
branch-v3 before: 0.317524 (+24.23%, +0.06)
branch-v3 after: 0.318907 (+24.78%, +0.06)
## data-env-vars.name = mozilla-central-2024-03-22-zstd-sparse-revlog
# bin-env-vars.hg.flavor = default
branch-v2: 0.339010 ~~~~~
branch-v3 before: 0.410007 (+20.94%, +0.07)
branch-v3 after: 0.349752 (+3.17%, +0.01)
# bin-env-vars.hg.flavor = rust
branch-v2: 0.346525 ~~~~~
branch-v3 before: 0.410428 (+18.44%, +0.06)
branch-v3 after: 0.354300 (+2.24%, +0.01)
## data-env-vars.name = mozilla-central-2024-03-22-ds2-pnm
# bin-env-vars.hg.flavor = rust
branch-v2: 0.380202 ~~~~~
branch-v3 before: 0.393871 (+3.60%, +0.01)
branch-v3 after: 0.396293 (+4.23%, +0.02)
## data-env-vars.name = mozilla-unified-2024-03-22-zstd-sparse-revlog
# bin-env-vars.hg.flavor = default
branch-v2: 0.412165 ~~~~~
branch-v3 before: 0.438105 (+6.29%, +0.03)
branch-v3 after: 0.424769 (+3.06%, +0.01)
# bin-env-vars.hg.flavor = rust
branch-v2: 0.412397 ~~~~~
branch-v3 before: 0.438405 (+6.31%, +0.03)
branch-v3 after: 0.421796 (+2.28%, +0.01)
## data-env-vars.name = mozilla-unified-2024-03-22-ds2-pnm
# bin-env-vars.hg.flavor = rust
branch-v2: 0.429501 ~~~~~
branch-v3 before: 0.452692 (+5.40%, +0.02)
branch-v3 after: 0.443849 (+3.34%, +0.01)
## data-env-vars.name = mozilla-try-2024-03-26-zstd-sparse-revlog
# bin-env-vars.hg.flavor = default
branch-v2: 3.403171 ~~~~~
branch-v3 before: 6.562345 (+92.83%, +3.16)
branch-v3 after: 6.234055 (+83.18%, +2.83)
# bin-env-vars.hg.flavor = rust
branch-v2: 3.454876 ~~~~~
branch-v3 before: 6.160248 (+78.31%, +2.71)
branch-v3 after: 6.307813 (+82.58%, +2.85)
## data-env-vars.name = mozilla-try-2024-03-26-ds2-pnm
# bin-env-vars.hg.flavor = rust
branch-v2: 3.465435 ~~~~~
branch-v3 before: 5.381648 (+55.30%, +1.92)
branch-v3 after: 5.176076 (+49.36%, +1.71)