Mercurial > hg
annotate tests/test-narrow-clone-stream.t @ 40494:9aeb9e2d28a7
store: introduce _matchtrackedpath() and use it to filter store files
This patch introduces a function to filter store files on the basis of the path
which they are tracking.
The function assumes that the entries can be of two types, 'meta/*' and 'data/*'
which means it will just work on revlog based storage and not with another
storage ways.
For the 'data/*' entries, we remove the 'data/' part and '.i/.d' part from the
beginning and the end then pass that to matcher.
For the 'meta/*' entries, we remove the 'meta/' and '/00manifest.(i/d)' part from
beginning and end then call matcher.visitdir() with it to make sure all the
parent directories are also downloaded.
Since the storage filtering for narrow stream clones is implemented with this
patch, we remove the un-implemented error message, add some more tests and add
the treemanifest case to tests too.
The tests demonstrate that it works correctly.
After this patch, we have now narrow stream clones working. Narrow stream clones
are a very important feature for large repositories who have good internet
connection because they use streamclones for cloning and if they do normal
narrow clone, that takes more time then a full streamclone. Also narrow-stream
clone will drastically speed up clone timings.
Differential Revision: https://phab.mercurial-scm.org/D5139
author | Pulkit Goyal <pulkit@yandex-team.ru> |
---|---|
date | Wed, 17 Oct 2018 17:42:32 +0300 |
parents | af62936c2508 |
children | d6ec45b79277 |
rev | line source |
---|---|
40494
9aeb9e2d28a7
store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
40338
diff
changeset
|
1 #testcases tree flat |
9aeb9e2d28a7
store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
40338
diff
changeset
|
2 |
40338
af62936c2508
streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
diff
changeset
|
3 Tests narrow stream clones |
af62936c2508
streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
diff
changeset
|
4 |
af62936c2508
streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
diff
changeset
|
5 $ . "$TESTDIR/narrow-library.sh" |
af62936c2508
streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
diff
changeset
|
6 |
40494
9aeb9e2d28a7
store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
40338
diff
changeset
|
7 #if tree |
9aeb9e2d28a7
store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
40338
diff
changeset
|
8 $ cat << EOF >> $HGRCPATH |
9aeb9e2d28a7
store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
40338
diff
changeset
|
9 > [experimental] |
9aeb9e2d28a7
store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
40338
diff
changeset
|
10 > treemanifest = 1 |
9aeb9e2d28a7
store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
40338
diff
changeset
|
11 > EOF |
9aeb9e2d28a7
store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
40338
diff
changeset
|
12 #endif |
9aeb9e2d28a7
store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
40338
diff
changeset
|
13 |
40338
af62936c2508
streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
diff
changeset
|
14 Server setup |
af62936c2508
streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
diff
changeset
|
15 |
af62936c2508
streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
diff
changeset
|
16 $ hg init master |
af62936c2508
streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
diff
changeset
|
17 $ cd master |
af62936c2508
streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
diff
changeset
|
18 $ mkdir dir |
af62936c2508
streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
diff
changeset
|
19 $ mkdir dir/src |
af62936c2508
streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
diff
changeset
|
20 $ cd dir/src |
af62936c2508
streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
diff
changeset
|
21 $ for x in `$TESTDIR/seq.py 20`; do echo $x > "f$x"; hg add "f$x"; hg commit -m "Commit src $x"; done |
af62936c2508
streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
diff
changeset
|
22 |
af62936c2508
streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
diff
changeset
|
23 $ cd .. |
af62936c2508
streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
diff
changeset
|
24 $ mkdir tests |
af62936c2508
streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
diff
changeset
|
25 $ cd tests |
af62936c2508
streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
diff
changeset
|
26 $ for x in `$TESTDIR/seq.py 20`; do echo $x > "f$x"; hg add "f$x"; hg commit -m "Commit src $x"; done |
af62936c2508
streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
diff
changeset
|
27 $ cd ../../.. |
af62936c2508
streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
diff
changeset
|
28 |
af62936c2508
streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
diff
changeset
|
29 Trying to stream clone when the server does not support it |
af62936c2508
streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
diff
changeset
|
30 |
af62936c2508
streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
diff
changeset
|
31 $ hg clone --narrow ssh://user@dummy/master narrow --noupdate --include "dir/src/f10" --stream |
af62936c2508
streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
diff
changeset
|
32 streaming all changes |
af62936c2508
streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
diff
changeset
|
33 remote: abort: server does not support narrow stream clones |
af62936c2508
streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
diff
changeset
|
34 abort: pull failed on remote |
af62936c2508
streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
diff
changeset
|
35 [255] |
af62936c2508
streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
diff
changeset
|
36 |
af62936c2508
streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
diff
changeset
|
37 Enable stream clone on the server |
af62936c2508
streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
diff
changeset
|
38 |
40494
9aeb9e2d28a7
store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
40338
diff
changeset
|
39 $ echo "[experimental.server]" >> master/.hg/hgrc |
40338
af62936c2508
streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
diff
changeset
|
40 $ echo "stream-narrow-clones=True" >> master/.hg/hgrc |
af62936c2508
streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
diff
changeset
|
41 |
af62936c2508
streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
diff
changeset
|
42 Cloning a specific file when stream clone is supported |
af62936c2508
streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
diff
changeset
|
43 |
af62936c2508
streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
diff
changeset
|
44 $ hg clone --narrow ssh://user@dummy/master narrow --noupdate --include "dir/src/f10" --stream |
af62936c2508
streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
diff
changeset
|
45 streaming all changes |
40494
9aeb9e2d28a7
store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
40338
diff
changeset
|
46 * files to transfer, * KB of data (glob) |
9aeb9e2d28a7
store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
40338
diff
changeset
|
47 transferred * KB in * seconds (* */sec) (glob) |
9aeb9e2d28a7
store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
40338
diff
changeset
|
48 |
9aeb9e2d28a7
store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
40338
diff
changeset
|
49 $ cd narrow |
9aeb9e2d28a7
store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
40338
diff
changeset
|
50 $ ls |
9aeb9e2d28a7
store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
40338
diff
changeset
|
51 $ hg tracked |
9aeb9e2d28a7
store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
40338
diff
changeset
|
52 I path:dir/src/f10 |
9aeb9e2d28a7
store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
40338
diff
changeset
|
53 |
9aeb9e2d28a7
store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
40338
diff
changeset
|
54 Making sure we have the correct set of requirements |
9aeb9e2d28a7
store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
40338
diff
changeset
|
55 |
9aeb9e2d28a7
store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
40338
diff
changeset
|
56 $ cat .hg/requires |
9aeb9e2d28a7
store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
40338
diff
changeset
|
57 dotencode |
9aeb9e2d28a7
store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
40338
diff
changeset
|
58 fncache |
9aeb9e2d28a7
store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
40338
diff
changeset
|
59 generaldelta |
9aeb9e2d28a7
store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
40338
diff
changeset
|
60 narrowhg-experimental |
9aeb9e2d28a7
store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
40338
diff
changeset
|
61 revlogv1 |
9aeb9e2d28a7
store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
40338
diff
changeset
|
62 store |
9aeb9e2d28a7
store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
40338
diff
changeset
|
63 treemanifest (tree !) |
9aeb9e2d28a7
store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
40338
diff
changeset
|
64 |
9aeb9e2d28a7
store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
40338
diff
changeset
|
65 Making sure store has the required files |
9aeb9e2d28a7
store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
40338
diff
changeset
|
66 |
9aeb9e2d28a7
store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
40338
diff
changeset
|
67 $ ls .hg/store/ |
9aeb9e2d28a7
store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
40338
diff
changeset
|
68 00changelog.i |
9aeb9e2d28a7
store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
40338
diff
changeset
|
69 00manifest.i |
9aeb9e2d28a7
store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
40338
diff
changeset
|
70 data |
9aeb9e2d28a7
store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
40338
diff
changeset
|
71 fncache |
9aeb9e2d28a7
store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
40338
diff
changeset
|
72 meta (tree !) |
9aeb9e2d28a7
store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
40338
diff
changeset
|
73 narrowspec |
9aeb9e2d28a7
store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
40338
diff
changeset
|
74 undo |
9aeb9e2d28a7
store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
40338
diff
changeset
|
75 undo.backupfiles |
9aeb9e2d28a7
store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
40338
diff
changeset
|
76 undo.phaseroots |
9aeb9e2d28a7
store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
40338
diff
changeset
|
77 |
9aeb9e2d28a7
store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
40338
diff
changeset
|
78 Checking that repository has all the required data and not broken |
9aeb9e2d28a7
store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
40338
diff
changeset
|
79 |
9aeb9e2d28a7
store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
40338
diff
changeset
|
80 $ hg verify |
9aeb9e2d28a7
store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
40338
diff
changeset
|
81 checking changesets |
9aeb9e2d28a7
store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
40338
diff
changeset
|
82 checking manifests |
9aeb9e2d28a7
store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
40338
diff
changeset
|
83 checking directory manifests (tree !) |
9aeb9e2d28a7
store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
40338
diff
changeset
|
84 crosschecking files in changesets and manifests |
9aeb9e2d28a7
store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
40338
diff
changeset
|
85 checking files |
9aeb9e2d28a7
store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
40338
diff
changeset
|
86 checked 40 changesets with 1 changes to 1 files |