Mercurial > hg
annotate tests/test-pending.t @ 24735:07200e3332a1
tags: extract .hgtags filenodes cache to a standalone file
Resolution of .hgtags filenodes values has historically been a
performance pain point for large repositories, where reading individual
manifests can take over 100ms. Multiplied by hundreds or even thousands
of heads and resolving .hgtags filenodes becomes a performance issue.
This patch establishes a standalone cache file holding the .hgtags
filenodes for each changeset. After this patch, the .hgtags filenode
for any particular changeset should only have to be computed once
during the lifetime of the repository.
The introduced hgtagsfnodes1 cache file is modeled after the rev branch
cache: the cache is effectively an array of entries consisting of a
changeset fragment and the filenode for a revision. The file grows in
proportion to the length of the repository (24 bytes per changeset) and
is truncated when the repository is stripped. The file is not written
unless tag info is requested and tags have changed since last time.
This patch partially addresses issue4550. Future patches will split the
"tags" cache file into per-filter files and will refactor the cache
format to not capture the .hgtags fnodes, as these are now stored in
the hgtagsfnodes1 cache. This patch is capable of standing alone. We
should not have to wait on the tags cache filter split and format
refactor for this patch to land.
author | Gregory Szorc <gregory.szorc@gmail.com> |
---|---|
date | Wed, 15 Apr 2015 17:42:38 -0400 |
parents | 921458360270 |
children | 8678b1eafbcf |
rev | line source |
---|---|
13237
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
1 Verify that pending changesets are seen by pretxn* hooks but not by other |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
2 processes that access the destination repo while the hooks are running. |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
3 |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
4 The hooks (python and external) both reject changesets after some think time, |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
5 during which another process runs pull. Each hook creates a file ('notify') to |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
6 indicate to the controlling process that it is running; the process removes the |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
7 file to indicate the hook can terminate. |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
8 |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
9 init env vars |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
10 |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
11 $ d=`pwd` |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
12 $ maxwait=20 |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
13 |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
14 utility to run the test - start a push in the background and run pull |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
15 |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
16 $ dotest() { |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
17 > rm -f notify |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
18 > printf 'push '; hg -R child-push tip --template '{node}\n' |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
19 > hg -R child-push -q push > push.out 2>&1 & |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
20 > |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
21 > # wait for hook to create the notify file |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
22 > i=$maxwait |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
23 > while [ ! -f notify -a $i != 0 ]; do |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
24 > sleep 1 |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
25 > i=`expr $i - 1` |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
26 > done |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
27 > |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
28 > # run pull |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
29 > hg -R child-pull -q pull |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
30 > rc=$? |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
31 > |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
32 > # tell hook to finish; notify should exist. |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
33 > rm notify |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
34 > wait |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
35 > |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
36 > cat push.out |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
37 > printf 'pull '; hg -R child-pull tip --template '{node}\n' |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
38 > return $rc |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
39 > } |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
40 |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
41 python hook |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
42 |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
43 $ cat <<EOF > reject.py |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
44 > import os, time |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
45 > from mercurial import ui, localrepo |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
46 > def rejecthook(ui, repo, hooktype, node, **opts): |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
47 > ui.write('hook %s\\n' % repo['tip'].hex()) |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
48 > # create the notify file so caller knows we're running |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
49 > fpath = os.path.join('$d', 'notify') |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
50 > f = open(fpath, 'w') |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
51 > f.close() |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
52 > # wait for ack - caller should delete the notify file |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
53 > i = $maxwait |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
54 > while os.path.exists(fpath) and i > 0: |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
55 > time.sleep(1) |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
56 > i -= 1 |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
57 > return True # reject the changesets |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
58 > EOF |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
59 |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
60 external hook |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
61 |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
62 $ cat <<EOF > reject.sh |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
63 > printf 'hook '; hg tip --template '{node}\\n' |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
64 > # create the notify file so caller knows we're running |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
65 > fpath=$d/notify |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
66 > touch \$fpath |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
67 > # wait for ack - caller should delete the notify file |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
68 > i=$maxwait |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
69 > while [ -f \$fpath -a \$i != 0 ]; do |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
70 > sleep 1 |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
71 > i=\`expr \$i - 1\` |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
72 > done |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
73 > exit 1 # reject the changesets |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
74 > EOF |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
75 |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
76 create repos |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
77 |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
78 $ hg init parent |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
79 $ hg clone -q parent child-push |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
80 $ hg clone -q parent child-pull |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
81 $ echo a > child-push/a |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
82 $ hg -R child-push add child-push/a |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
83 $ hg -R child-push commit -m a -d '1000000 0' |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
84 |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
85 test python hook |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
86 |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
87 $ cat <<EOF > parent/.hg/hgrc |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
88 > [extensions] |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
89 > reject = $d/reject.py |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
90 > [hooks] |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
91 > pretxnchangegroup = python:reject.rejecthook |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
92 > EOF |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
93 |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
94 $ dotest |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
95 push 29b62aeb769fdf78d8d9c5f28b017f76d7ef824b |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
96 hook 29b62aeb769fdf78d8d9c5f28b017f76d7ef824b |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
97 transaction abort! |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
98 rollback completed |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
99 abort: pretxnchangegroup hook failed |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
100 pull 0000000000000000000000000000000000000000 |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
101 |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
102 test external hook |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
103 |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
104 $ cat <<EOF > parent/.hg/hgrc |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
105 > [hooks] |
16975
921458360270
tests: remove hghave system-sh from test-pending.t
Mads Kiilerich <mads@kiilerich.com>
parents:
16107
diff
changeset
|
106 > pretxnchangegroup = sh $d/reject.sh |
13237
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
107 > EOF |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
108 |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
109 $ dotest |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
110 push 29b62aeb769fdf78d8d9c5f28b017f76d7ef824b |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
111 hook 29b62aeb769fdf78d8d9c5f28b017f76d7ef824b |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
112 transaction abort! |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
113 rollback completed |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
114 abort: pretxnchangegroup hook exited with status 1 |
c046978cc0a9
tests: check visibility of pending changesets
John Coomes <john.coomes@oracle.com>
parents:
diff
changeset
|
115 pull 0000000000000000000000000000000000000000 |