tests/test-remotefilelog-clone.t

stream-clone: avoid opening a revlog in case we do not need it

Opening an revlog has a cost, especially if it is inline as we have to scan the
file and construct an index.

To prevent the associated slowdown, we just do a minimal scan to check that an
inline file is still inline, and simply stream the file without creating a
revlog when we can.

This provides a big boost compared to the previous changeset, even if the full
generation is still penalized by the initial gathering of information.

All benchmarks are run on linux with Python 3.10.7.
# benchmark.name             = hg.exchange.stream.generate
# benchmark.variants.version = v2

### Compared to the previous changesets

We get a large win all across the board!

# mercurial-2018-08-01-zstd-sparse-revlog
before:  0.250694 seconds
after:   0.105986 seconds  (-57.72%)
# pypy-2018-08-01-zstd-sparse-revlog
before:  3.885657 seconds
after:   1.709748 seconds  (-56.00%)
# netbeans-2018-08-01-zstd-sparse-revlog
before: 16.679371 seconds
after:   7.687469 seconds  (-53.91%)
# mozilla-central-2018-08-01-zstd-sparse-revlog
before: 38.575482 seconds
after:  17.520316 seconds  (-54.58%)
# mozilla-try-2019-02-18-zstd-sparse-revlog
before: 81.160994 seconds
after:  37.073753 seconds  (-54.32%)



### Compared to 6.4.3

We are still significantly slower than 6.4.3, the extra time is usually twice
slower than the extra time we observe on the locked section, which is a quite
interesting information.

Except for mercurial-central that is much faster. That discrepancy is not really
explained yet.

# mercurial-2018-08-01-zstd-sparse-revlog
6.4.3:  0.072560 seconds
after:  0.105986 seconds  (+46.07%) (- 0.03 seconds)
# pypy-2018-08-01-zstd-sparse-revlog
6.4.3:  1.211193 seconds
after:  1.709748 seconds  (+41.16%) (-0.45 seconds)
# netbeans-2018-08-01-zstd-sparse-revlog
6.4.3:  4.932843 seconds
after:  7.687469 seconds  (+55.84%) (-2.75 seconds)
# mozilla-central-2018-08-01-zstd-sparse-revlog
6.4.3: 34.012226 seconds
after: 17.520316 seconds  (-48.49%) (-16.49 seconds)
# mozilla-try-2019-02-18-zstd-sparse-revlog
6.4.3: 23.850555 seconds
after: 37.073753 seconds  (+55.44%) (+13.22 seconds)

#require no-windows

  $ . "$TESTDIR/remotefilelog-library.sh"

  $ hg init master
  $ cd master
  $ cat >> .hg/hgrc <<EOF
  > [remotefilelog]
  > server=True
  > EOF
  $ echo x > x
  $ hg commit -qAm x

  $ cd ..

# shallow clone from full

  $ hgcloneshallow ssh://user@dummy/master shallow --noupdate
  streaming all changes
  2 files to transfer, 227 bytes of data
  transferred 227 bytes in * seconds (*/sec) (glob)
  searching for changes
  no changes found
  $ cd shallow
  $ hg debugrequires
  dotencode
  dirstate-v2 (dirstate-v2 !)
  exp-remotefilelog-repo-req-1
  fncache
  generaldelta
  persistent-nodemap (rust !)
  revlog-compression-zstd (zstd !)
  revlogv1
  share-safe
  sparserevlog
  store

  $ hg update
  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
  1 files fetched over 1 fetches - (1 misses, 0.00% hit ratio) over *s (glob)

  $ cat x
  x

  $ ls .hg/store/data
  $ echo foo > f
  $ hg add f
  $ hg ci -m 'local content'
  $ ls .hg/store/data
  4a0a19218e082a343a1b17e5333409af9d98f0f5

  $ cd ..

# shallow clone from shallow

  $ hgcloneshallow ssh://user@dummy/shallow shallow2  --noupdate
  streaming all changes
  3 files to transfer, 564 bytes of data
  transferred 564 bytes in * seconds (*/sec) (glob)
  searching for changes
  no changes found
  $ cd shallow2
  $ hg debugrequires
  dotencode
  dirstate-v2 (dirstate-v2 !)
  exp-remotefilelog-repo-req-1
  fncache
  generaldelta
  persistent-nodemap (rust !)
  revlog-compression-zstd (zstd !)
  revlogv1
  share-safe
  sparserevlog
  store
  $ ls .hg/store/data
  4a0a19218e082a343a1b17e5333409af9d98f0f5

  $ hg update
  2 files updated, 0 files merged, 0 files removed, 0 files unresolved

  $ cat x
  x

  $ cd ..

# full clone from shallow

Note: the output to STDERR comes from a different process to the output on
STDOUT and their relative ordering is not deterministic. As a result, the test
was failing sporadically. To avoid this, we capture STDERR to a file and
check its contents separately.

  $ TEMP_STDERR=full-clone-from-shallow.stderr.tmp
  $ hg clone --noupdate ssh://user@dummy/shallow full 2>$TEMP_STDERR
  streaming all changes
  [100]
  $ cat $TEMP_STDERR
  remote: abort: Cannot clone from a shallow repo to a full repo.
  abort: pull failed on remote
  $ rm $TEMP_STDERR

# getbundle full clone

  $ printf '[server]\npreferuncompressed=False\n' >> master/.hg/hgrc
  $ hgcloneshallow ssh://user@dummy/master shallow3
  requesting all changes
  adding changesets
  adding manifests
  adding file changes
  added 1 changesets with 0 changes to 0 files
  new changesets b292c1e3311f
  updating to branch default
  1 files updated, 0 files merged, 0 files removed, 0 files unresolved

  $ ls shallow3/.hg/store/data
  $ hg debugrequires -R shallow3/
  dotencode
  dirstate-v2 (dirstate-v2 !)
  exp-remotefilelog-repo-req-1
  fncache
  generaldelta
  persistent-nodemap (rust !)
  revlog-compression-zstd (zstd !)
  revlogv1
  share-safe
  sparserevlog
  store