tests/test-narrow-clone-stream.t

store: introduce _matchtrackedpath() and use it to filter store files

This patch introduces a function to filter store files on the basis of the path
which they are tracking.

The function assumes that the entries can be of two types, 'meta/*' and 'data/*'
which means it will just work on revlog based storage and not with another
storage ways.

For the 'data/*' entries, we remove the 'data/' part and '.i/.d' part from the
beginning and the end then pass that to matcher.

For the 'meta/*' entries, we remove the 'meta/' and '/00manifest.(i/d)' part from
beginning and end then call matcher.visitdir() with it to make sure all the
parent directories are also downloaded.

Since the storage filtering for narrow stream clones is implemented with this
patch, we remove the un-implemented error message, add some more tests and add
the treemanifest case to tests too.

The tests demonstrate that it works correctly.

After this patch, we have now narrow stream clones working. Narrow stream clones
are a very important feature for large repositories who have good internet
connection because they use streamclones for cloning and if they do normal
narrow clone, that takes more time then a full streamclone. Also narrow-stream
clone will drastically speed up clone timings.

Differential Revision: https://phab.mercurial-scm.org/D5139

#testcases tree flat

Tests narrow stream clones

  $ . "$TESTDIR/narrow-library.sh"

#if tree
  $ cat << EOF >> $HGRCPATH
  > [experimental]
  > treemanifest = 1
  > EOF
#endif

Server setup

  $ hg init master
  $ cd master
  $ mkdir dir
  $ mkdir dir/src
  $ cd dir/src
  $ for x in `$TESTDIR/seq.py 20`; do echo $x > "f$x"; hg add "f$x"; hg commit -m "Commit src $x"; done

  $ cd ..
  $ mkdir tests
  $ cd tests
  $ for x in `$TESTDIR/seq.py 20`; do echo $x > "f$x"; hg add "f$x"; hg commit -m "Commit src $x"; done
  $ cd ../../..

Trying to stream clone when the server does not support it

  $ hg clone --narrow ssh://user@dummy/master narrow --noupdate --include "dir/src/f10" --stream
  streaming all changes
  remote: abort: server does not support narrow stream clones
  abort: pull failed on remote
  [255]

Enable stream clone on the server

  $ echo "[experimental.server]" >> master/.hg/hgrc
  $ echo "stream-narrow-clones=True" >> master/.hg/hgrc

Cloning a specific file when stream clone is supported

  $ hg clone --narrow ssh://user@dummy/master narrow --noupdate --include "dir/src/f10" --stream
  streaming all changes
  * files to transfer, * KB of data (glob)
  transferred * KB in * seconds (* */sec) (glob)

  $ cd narrow
  $ ls
  $ hg tracked
  I path:dir/src/f10

Making sure we have the correct set of requirements

  $ cat .hg/requires
  dotencode
  fncache
  generaldelta
  narrowhg-experimental
  revlogv1
  store
  treemanifest (tree !)

Making sure store has the required files

  $ ls .hg/store/
  00changelog.i
  00manifest.i
  data
  fncache
  meta (tree !)
  narrowspec
  undo
  undo.backupfiles
  undo.phaseroots

Checking that repository has all the required data and not broken

  $ hg verify
  checking changesets
  checking manifests
  checking directory manifests (tree !)
  crosschecking files in changesets and manifests
  checking files
  checked 40 changesets with 1 changes to 1 files