view tests/test-url-download.t @ 51977:42a116f1cdc1

branchmap-v3: introduce a "stop_rev" argument to `headsrevs`

The `headsrevs` method of the revlog already have a `revs` argument to compute
the headrevs of a limited set of heads. However, it disable the use of the
native compiled code to compute the heads, which slows down the branchmap v3
code a lot.

The branchmap v3 usage is actually quite constrained as we will always only
ignores a part at the top of the graph. So we could be significantly faster.

We start by making small change to the python side to improve the situation and
introduce the new API. More collaboration with the native code are coming later.

This massively speedup operation and close most of the remaining gaps between
branchmap-v3 and branchmap-v2. especially on repository with many revs like
mozilla-try. A small overhead remains mostly because the `headrevs` logic
currently has some inefficiently. We will look into them from there.

### benchmark.name                                 = hg.command.unbundle
  # bin-env-vars.hg.py-re2-module                  = default
  # benchmark.variants.issue6528                   = disabled
  # benchmark.variants.resource-usage              = default
  # benchmark.variants.reuse-external-delta-parent = yes
  # benchmark.variants.revs                        = any-1-extra-rev
  # benchmark.variants.source                      = unbundle
  # benchmark.variants.validate                    = default
  # benchmark.variants.verbosity                   = quiet

 ## data-env-vars.name                             = netbeans-2018-08-01-zstd-sparse-revlog
  # bin-env-vars.hg.flavor                         = default
branch-v2:        0.233711  ~~~~~
branch-v3 before: 0.368769  (+57.79%, +0.14)
branch-v3 after:  0.239857   (+2.63%, +0.01)
  # bin-env-vars.hg.flavor                         = rust
branch-v2:        0.235230  ~~~~~
branch-v3 before: 0.372460  (+58.34%, +0.14)
branch-v3 after:  0.240972   (+2.44%, +0.01)
 ## data-env-vars.name                             = netbeans-2018-08-01-ds2-pnm
  # bin-env-vars.hg.flavor                         = rust
branch-v2:        0.255586  ~~~~~
branch-v3 before: 0.318907  (+24.78%, +0.06)
branch-v3 after:  0.268560   (+5.08%, +0.01)

 ## data-env-vars.name                             = mozilla-central-2024-03-22-zstd-sparse-revlog
  # bin-env-vars.hg.flavor                         = default
branch-v2:        0.339010  ~~~~~
branch-v3 before: 0.349752   (+3.17%, +0.01)
branch-v3 after:  0.349389   (+3.06%, +0.01)
  # bin-env-vars.hg.flavor                         = rust
branch-v2:        0.346525  ~~~~~
branch-v3 before: 0.354300   (+2.24%, +0.01)
branch-v3 after:  0.355661   (+2.64%, +0.01)
 ## data-env-vars.name                             = mozilla-central-2024-03-22-ds2-pnm
  # bin-env-vars.hg.flavor                         = rust
branch-v2:        0.380202  ~~~~~
branch-v3 before: 0.396293   (+4.23%, +0.02)
branch-v3 after:  0.408851   (+7.54%, +0.03)

 ## data-env-vars.name                             = mozilla-unified-2024-03-22-zstd-sparse-revlog
  # bin-env-vars.hg.flavor                         = default
branch-v2:        0.412165  ~~~~~
branch-v3 before: 0.424769   (+3.06%, +0.01)
branch-v3 after:  0.427782   (+3.79%, +0.02)
  # bin-env-vars.hg.flavor                         = rust
branch-v2:        0.412397  ~~~~~
branch-v3 before: 0.421796   (+2.28%, +0.01)
branch-v3 after:  0.422354   (+2.41%, +0.01)
 ## data-env-vars.name                             = mozilla-unified-2024-03-22-ds2-pnm
  # bin-env-vars.hg.flavor                         = rust
branch-v2:        0.429501  ~~~~~
branch-v3 before: 0.443849   (+3.34%, +0.01)
branch-v3 after:  0.443197   (+3.19%, +0.01)

 ## data-env-vars.name                             = mozilla-try-2024-03-26-zstd-sparse-revlog
  # bin-env-vars.hg.flavor                         = default
branch-v2:        3.403171  ~~~~~
branch-v3 before: 6.234055  (+83.18%, +2.83)
branch-v3 after:  3.819477  (+12.23%, +0.42)
  # bin-env-vars.hg.flavor                         = rust
branch-v2:        3.454876  ~~~~~
branch-v3 before: 6.307813  (+82.58%, +2.85)
branch-v3 after:  3.590284   (+3.92%, +0.14)
 ## data-env-vars.name                             = mozilla-try-2024-03-26-ds2-pnm
  # bin-env-vars.hg.flavor                         = rust
branch-v2:        3.465435  ~~~~~
branch-v3 before: 5.176076  (+49.36%, +1.71)
branch-v3 after:  3.633278   (+4.84%, +0.17)

#require serve

  $ hg init server
  $ hg serve -R server -p $HGPORT -d --pid-file=hg1.pid -E ../error.log
  $ cat hg1.pid >> $DAEMON_PIDS

Check basic fetching

  $ hg debugdownload "http://localhost:$HGPORT/?cmd=lookup&key=tip"
  1 0000000000000000000000000000000000000000
  $ hg debugdownload  -o null.txt "http://localhost:$HGPORT/?cmd=lookup&key=null"
  $ cat null.txt
  1 0000000000000000000000000000000000000000

Check the request is made from the usual Mercurial logic
(rev details, give different content if the request has a Mercurial user agent)

  $ get-with-headers.py --headeronly "localhost:$HGPORT" "rev/tip" content-type
  200 Script output follows
  content-type: text/html; charset=ascii
  $ hg debugdownload "http://localhost:$HGPORT/rev/tip"
  
  # HG changeset patch
  # User 
  # Date 0 0
  # Node ID 0000000000000000000000000000000000000000
  
  
  
  

Check other kind of compatible url

  $ hg debugdownload ./null.txt
  1 0000000000000000000000000000000000000000

  $ cat ../error.log

Test largefile URL
------------------

  $ cat << EOF >> $HGRCPATH
  > [extensions]
  > largefiles=
  > EOF

  $ killdaemons.py
  $ rm -f error.log hg1.pid
  $ hg serve -R server -p $HGPORT -d --pid-file=hg1.pid -E error.log
  $ cat hg1.pid >> $DAEMON_PIDS

  $ hg -R server debuglfput null.txt
  a57b57b39ee4dc3da1e03526596007f480ecdbe8

  $ hg debugdownload "largefile://a57b57b39ee4dc3da1e03526596007f480ecdbe8" --config paths.default=http://localhost:$HGPORT/
  1 0000000000000000000000000000000000000000

from within a repository

  $ hg clone http://localhost:$HGPORT/ client
  no changes found
  updating to branch default
  0 files updated, 0 files merged, 0 files removed, 0 files unresolved

  $ cd client
  $ hg path
  default = http://localhost:$HGPORT/
  $ hg debugdownload "largefile://a57b57b39ee4dc3da1e03526596007f480ecdbe8"
  1 0000000000000000000000000000000000000000
  $ cd ..

  $ cat error.log