streamclone: also stream caches to the client
When stream clone is used over bundle2, relevant cache files are also streamed.
This is expected to be a massive performance win for clone since no important
cache will have to be recomputed.
Some performance numbers:
(All times are wall-clock times in seconds, 2 attempts per case.)
# Mozilla-Central
## Clone over ssh over lan
V1 streaming: 234.3 239.6
V2 streaming: 248.4 243.7
## Clone over ssh over Internet
V1 streaming: 175.5 110.9
V2 streaming: 109.1 111.0
## Clone over HTTP over lan
V1 streaming: 105.3 105.6
V2 streaming: 112.7 111.4
## Clone over HTTP over internet
V1 streaming: 105.6 114.6
V2 streaming: 226.7 225.9
## Hg tags
V1 streaming (no cache): 1.084 1.071
V2 streaming (cache): 0.312 0.325
## Hg branches
V1 streaming (no cache): 14.047 14.148
V2 streaming (with cache): 0.312 0.333
# Pypy
## Clone over ssh over internet
V1 streaming: 29.4 30.1
V2 streaming: 31.2 30.1
## Clone over http over internet
V1 streaming: 29.7 29.7
V2 streaming: 75.2 72.9
(since ssh and lan are not affected, there seems to be an issue with how we
read/write the http stream on connection with latency, unrelated to the format)
## Hg tags
V1 streaming (no cache): 1.752 1.664
V2 streaming (with cache): 0.274 0.260
## Hg branches
V1 streaming (no cache): 4.469 4.728
V2 streaming (with cache): 0.318 0.321
# Private repository:
* 500K revision revisions
* 11K topological heads
* 28K branch heads
## hg tags
no cache: 1543.332
with cache: 4.900
## hg branches
no cache: 91.828
with cache: 2.955
#require serve
$ hg init server
$ hg serve -R server -p $HGPORT -d --pid-file=hg1.pid -E ../error.log
$ cat hg1.pid >> $DAEMON_PIDS
Check basic fetching
$ hg debugdownload "http://localhost:$HGPORT/?cmd=lookup&key=tip"
1 0000000000000000000000000000000000000000
$ hg debugdownload -o null.txt "http://localhost:$HGPORT/?cmd=lookup&key=null"
$ cat null.txt
1 0000000000000000000000000000000000000000
Check the request is made from the usual Mercurial logic
(rev details, give different content if the request has a Mercurial user agent)
$ get-with-headers.py --headeronly "localhost:$HGPORT" "rev/tip" content-type
200 Script output follows
content-type: text/html; charset=ascii
$ hg debugdownload "http://localhost:$HGPORT/rev/tip"
# HG changeset patch
# User
# Date 0 0
# Node ID 0000000000000000000000000000000000000000
Check other kind of compatible url
$ hg debugdownload ./null.txt
1 0000000000000000000000000000000000000000
Test largefile URL
------------------
$ cat << EOF >> $HGRCPATH
> [extensions]
> largefiles=
> EOF
$ killdaemons.py
$ rm -f error.log hg1.pid
$ hg serve -R server -p $HGPORT -d --pid-file=hg1.pid -E error.log
$ cat hg1.pid >> $DAEMON_PIDS
$ hg -R server debuglfput null.txt
a57b57b39ee4dc3da1e03526596007f480ecdbe8
$ hg --traceback debugdownload "largefile://a57b57b39ee4dc3da1e03526596007f480ecdbe8" --config paths.default=http://localhost:$HGPORT/
1 0000000000000000000000000000000000000000
from within a repository
$ hg clone http://localhost:$HGPORT/ client
no changes found
updating to branch default
0 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ cd client
$ hg path
default = http://localhost:$HGPORT/
$ hg debugdownload "largefile://a57b57b39ee4dc3da1e03526596007f480ecdbe8"
1 0000000000000000000000000000000000000000
$ cd ..