streamclone: also stream caches to the client
When stream clone is used over bundle2, relevant cache files are also streamed.
This is expected to be a massive performance win for clone since no important
cache will have to be recomputed.
Some performance numbers:
(All times are wall-clock times in seconds, 2 attempts per case.)
# Mozilla-Central
## Clone over ssh over lan
V1 streaming: 234.3 239.6
V2 streaming: 248.4 243.7
## Clone over ssh over Internet
V1 streaming: 175.5 110.9
V2 streaming: 109.1 111.0
## Clone over HTTP over lan
V1 streaming: 105.3 105.6
V2 streaming: 112.7 111.4
## Clone over HTTP over internet
V1 streaming: 105.6 114.6
V2 streaming: 226.7 225.9
## Hg tags
V1 streaming (no cache): 1.084 1.071
V2 streaming (cache): 0.312 0.325
## Hg branches
V1 streaming (no cache): 14.047 14.148
V2 streaming (with cache): 0.312 0.333
# Pypy
## Clone over ssh over internet
V1 streaming: 29.4 30.1
V2 streaming: 31.2 30.1
## Clone over http over internet
V1 streaming: 29.7 29.7
V2 streaming: 75.2 72.9
(since ssh and lan are not affected, there seems to be an issue with how we
read/write the http stream on connection with latency, unrelated to the format)
## Hg tags
V1 streaming (no cache): 1.752 1.664
V2 streaming (with cache): 0.274 0.260
## Hg branches
V1 streaming (no cache): 4.469 4.728
V2 streaming (with cache): 0.318 0.321
# Private repository:
* 500K revision revisions
* 11K topological heads
* 28K branch heads
## hg tags
no cache: 1543.332
with cache: 4.900
## hg branches
no cache: 91.828
with cache: 2.955
Corrupt an hg repo with a pull started during an aborted commit
Create two repos, so that one of them can pull from the other one.
$ hg init source
$ cd source
$ touch foo
$ hg add foo
$ hg ci -m 'add foo'
$ hg clone . ../corrupted
updating to branch default
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ echo >> foo
$ hg ci -m 'change foo'
Add a hook to wait 5 seconds and then abort the commit
$ cd ../corrupted
$ echo "[hooks]" >> .hg/hgrc
$ echo 'pretxncommit = sh -c "sleep 5; exit 1"' >> .hg/hgrc
start a commit...
$ touch bar
$ hg add bar
$ hg ci -m 'add bar' &
... and start a pull while the commit is still running
$ sleep 1
$ hg pull ../source 2>/dev/null
pulling from ../source
transaction abort!
rollback completed
abort: pretxncommit hook exited with status 1
searching for changes
adding changesets
adding manifests
adding file changes
added 1 changesets with 1 changes to 1 files
new changesets 52998019f625
(run 'hg update' to get a working copy)
see what happened
$ wait
$ hg verify
checking changesets
checking manifests
crosschecking files in changesets and manifests
checking files
1 files, 2 changesets, 2 total revisions
$ cd ..