view tests/test-cache-abuse.t @ 51977:42a116f1cdc1

branchmap-v3: introduce a "stop_rev" argument to `headsrevs` The `headsrevs` method of the revlog already have a `revs` argument to compute the headrevs of a limited set of heads. However, it disable the use of the native compiled code to compute the heads, which slows down the branchmap v3 code a lot. The branchmap v3 usage is actually quite constrained as we will always only ignores a part at the top of the graph. So we could be significantly faster. We start by making small change to the python side to improve the situation and introduce the new API. More collaboration with the native code are coming later. This massively speedup operation and close most of the remaining gaps between branchmap-v3 and branchmap-v2. especially on repository with many revs like mozilla-try. A small overhead remains mostly because the `headrevs` logic currently has some inefficiently. We will look into them from there. ### benchmark.name = hg.command.unbundle # bin-env-vars.hg.py-re2-module = default # benchmark.variants.issue6528 = disabled # benchmark.variants.resource-usage = default # benchmark.variants.reuse-external-delta-parent = yes # benchmark.variants.revs = any-1-extra-rev # benchmark.variants.source = unbundle # benchmark.variants.validate = default # benchmark.variants.verbosity = quiet ## data-env-vars.name = netbeans-2018-08-01-zstd-sparse-revlog # bin-env-vars.hg.flavor = default branch-v2: 0.233711 ~~~~~ branch-v3 before: 0.368769 (+57.79%, +0.14) branch-v3 after: 0.239857 (+2.63%, +0.01) # bin-env-vars.hg.flavor = rust branch-v2: 0.235230 ~~~~~ branch-v3 before: 0.372460 (+58.34%, +0.14) branch-v3 after: 0.240972 (+2.44%, +0.01) ## data-env-vars.name = netbeans-2018-08-01-ds2-pnm # bin-env-vars.hg.flavor = rust branch-v2: 0.255586 ~~~~~ branch-v3 before: 0.318907 (+24.78%, +0.06) branch-v3 after: 0.268560 (+5.08%, +0.01) ## data-env-vars.name = mozilla-central-2024-03-22-zstd-sparse-revlog # bin-env-vars.hg.flavor = default branch-v2: 0.339010 ~~~~~ branch-v3 before: 0.349752 (+3.17%, +0.01) branch-v3 after: 0.349389 (+3.06%, +0.01) # bin-env-vars.hg.flavor = rust branch-v2: 0.346525 ~~~~~ branch-v3 before: 0.354300 (+2.24%, +0.01) branch-v3 after: 0.355661 (+2.64%, +0.01) ## data-env-vars.name = mozilla-central-2024-03-22-ds2-pnm # bin-env-vars.hg.flavor = rust branch-v2: 0.380202 ~~~~~ branch-v3 before: 0.396293 (+4.23%, +0.02) branch-v3 after: 0.408851 (+7.54%, +0.03) ## data-env-vars.name = mozilla-unified-2024-03-22-zstd-sparse-revlog # bin-env-vars.hg.flavor = default branch-v2: 0.412165 ~~~~~ branch-v3 before: 0.424769 (+3.06%, +0.01) branch-v3 after: 0.427782 (+3.79%, +0.02) # bin-env-vars.hg.flavor = rust branch-v2: 0.412397 ~~~~~ branch-v3 before: 0.421796 (+2.28%, +0.01) branch-v3 after: 0.422354 (+2.41%, +0.01) ## data-env-vars.name = mozilla-unified-2024-03-22-ds2-pnm # bin-env-vars.hg.flavor = rust branch-v2: 0.429501 ~~~~~ branch-v3 before: 0.443849 (+3.34%, +0.01) branch-v3 after: 0.443197 (+3.19%, +0.01) ## data-env-vars.name = mozilla-try-2024-03-26-zstd-sparse-revlog # bin-env-vars.hg.flavor = default branch-v2: 3.403171 ~~~~~ branch-v3 before: 6.234055 (+83.18%, +2.83) branch-v3 after: 3.819477 (+12.23%, +0.42) # bin-env-vars.hg.flavor = rust branch-v2: 3.454876 ~~~~~ branch-v3 before: 6.307813 (+82.58%, +2.85) branch-v3 after: 3.590284 (+3.92%, +0.14) ## data-env-vars.name = mozilla-try-2024-03-26-ds2-pnm # bin-env-vars.hg.flavor = rust branch-v2: 3.465435 ~~~~~ branch-v3 before: 5.176076 (+49.36%, +1.71) branch-v3 after: 3.633278 (+4.84%, +0.17)
author Pierre-Yves David <pierre-yves.david@octobus.net>
date Tue, 03 Sep 2024 11:11:17 +0200
parents 0f26ee69cf36
children
line wrap: on
line source

Enable obsolete markers

  $ cat >> $HGRCPATH << EOF
  > [experimental]
  > evolution.createmarkers=True
  > [phases]
  > publish=False
  > EOF

Build a repo with some cacheable bits:

  $ hg init a
  $ cd a

  $ echo a > a
  $ hg ci -qAm0
  $ hg tag t1
  $ hg book -i bk1

  $ hg branch -q b2
  $ hg ci -Am1
  $ hg tag t2

  $ echo dumb > dumb
  $ hg ci -qAmdumb
  $ hg debugobsolete b1174d11b69e63cb0c5726621a43c859f0858d7f
  1 new obsolescence markers
  obsoleted 1 changesets

  $ hg phase -pr t1
  $ hg phase -fsr t2

Make a helper function to check cache damage invariants:

- command output shouldn't change
- cache should be present after first use
- corruption/repair should be silent (no exceptions or warnings)
- cache should survive deletion, overwrite, and append
- unreadable / unwriteable caches should be ignored
- cache should be rebuilt after corruption

  $ damage() {
  >  CMD=$1
  >  CACHE=.hg/cache/$2
  >  CLEAN=$3
  >  hg $CMD > before
  >  test -f $CACHE || echo "not present"
  >  echo bad > $CACHE
  >  test -z "$CLEAN" || $CLEAN
  >  hg $CMD > after
  >  "$RUNTESTDIR/pdiff" before after || echo "*** overwrite corruption"
  >  echo corruption >> $CACHE
  >  test -z "$CLEAN" || $CLEAN
  >  hg $CMD > after
  >  "$RUNTESTDIR/pdiff" before after || echo "*** append corruption"
  >  rm $CACHE
  >  mkdir $CACHE
  >  test -z "$CLEAN" || $CLEAN
  >  hg $CMD > after
  >  "$RUNTESTDIR/pdiff" before after || echo "*** read-only corruption"
  >  test -d $CACHE || echo "*** directory clobbered"
  >  rmdir $CACHE
  >  test -z "$CLEAN" || $CLEAN
  >  hg $CMD > after
  >  "$RUNTESTDIR/pdiff" before after || echo "*** missing corruption"
  >  test -f $CACHE || echo "not rebuilt"
  > }

Beat up tags caches:

  $ damage "tags --hidden" tags2
  $ damage tags tags2-visible
  $ damage "tag -f t3" hgtagsfnodes1
  1 new orphan changesets
  1 new orphan changesets
  1 new orphan changesets
  1 new orphan changesets
  1 new orphan changesets

Beat up branch caches:

  $ damage branches branch2-base "rm .hg/cache/branch2-[vs]*"
  $ damage branches branch2-served "rm .hg/cache/branch2-[bv]*"
  $ damage branches branch2-visible
  $ damage "log -r branch(.)" rbc-names-v2
  $ damage "log -r branch(default)" rbc-names-v2
  $ damage "log -r branch(b2)" rbc-revs-v2

We currently can't detect an rbc cache with unknown names:

  $ damage "log -qr branch(b2)" rbc-names-v2
  --- before	* (glob)
  +++ after	* (glob)
  @@ -1,8 +?,0 @@ (glob)
  -2:5fb7d38b9dc4
  -3:60b597ffdafa
  -4:b1174d11b69e
  -5:6354685872c0
  -6:5ebc725f1bef
  -7:7b76eec2f273
  -8:ef3428d9d644
  -9:ba7a936bc03c
  *** append corruption