view tests/test-mq-qclone-http.t @ 35767:5f5fb279fd39

streamclone: also stream caches to the client

When stream clone is used over bundle2, relevant cache files are also streamed.
This is expected to be a massive performance win for clone since no important
cache will have to be recomputed.

Some performance numbers:

(All times are wall-clock times in seconds, 2 attempts per case.)

# Mozilla-Central

## Clone over ssh over lan
V1 streaming: 234.3 239.6
V2 streaming: 248.4 243.7

## Clone over ssh over Internet
V1 streaming: 175.5 110.9
V2 streaming: 109.1 111.0

## Clone over HTTP over lan
V1 streaming: 105.3 105.6
V2 streaming: 112.7 111.4

## Clone over HTTP over internet
V1 streaming: 105.6 114.6
V2 streaming: 226.7 225.9

## Hg tags
V1 streaming (no cache): 1.084 1.071
V2 streaming (cache):    0.312 0.325

## Hg branches
V1 streaming (no cache):   14.047 14.148
V2 streaming (with cache):  0.312  0.333

# Pypy

## Clone over ssh over internet
V1 streaming: 29.4 30.1
V2 streaming: 31.2 30.1

## Clone over http over internet
V1 streaming: 29.7 29.7
V2 streaming: 75.2 72.9

(since ssh and lan are not affected, there seems to be an issue with how we
read/write the http stream on connection with latency, unrelated to the format)

## Hg tags
V1 streaming (no cache):   1.752 1.664
V2 streaming (with cache): 0.274 0.260

## Hg branches
V1 streaming (no cache):   4.469 4.728
V2 streaming (with cache): 0.318 0.321

# Private repository:
    * 500K revision revisions
    * 11K topological heads
    * 28K branch heads

## hg tags
no cache:   1543.332
with cache:    4.900

## hg branches
no cache:   91.828
with cache:  2.955

#require killdaemons

hide outer repo
  $ hg init

  $ echo "[extensions]" >> $HGRCPATH
  $ echo "mq=" >> $HGRCPATH
  $ mkdir webdir
  $ cd webdir
  $ hg init a
  $ hg --cwd a qinit -c
  $ echo a > a/a
  $ hg --cwd a ci -A -m a
  adding a
  $ echo b > a/b
  $ hg --cwd a addremove
  adding b
  $ hg --cwd a qnew -f b.patch
  $ hg --cwd a qcommit -m b.patch
  $ hg --cwd a log --template "{desc}\n"
  [mq]: b.patch
  a
  $ hg --cwd a/.hg/patches log --template "{desc}\n"
  b.patch
  $ root=`pwd`
  $ cd ..

test with recursive collection

  $ cat > collections.conf <<EOF
  > [paths]
  > /=$root/**
  > EOF
  $ hg serve -p $HGPORT -d --pid-file=hg.pid --webdir-conf collections.conf \
  >     -A access-paths.log -E error-paths-1.log
  $ cat hg.pid >> $DAEMON_PIDS
  $ get-with-headers.py localhost:$HGPORT '?style=raw'
  200 Script output follows
  
  
  /a/
  /a/.hg/patches/
  
  $ hg qclone http://localhost:$HGPORT/a b
  requesting all changes
  adding changesets
  adding manifests
  adding file changes
  added 2 changesets with 2 changes to 2 files
  new changesets cb9a9f314b8b:184916345baa
  requesting all changes
  adding changesets
  adding manifests
  adding file changes
  added 1 changesets with 3 changes to 3 files
  new changesets 4052ceaa8c4e
  updating to branch default
  3 files updated, 0 files merged, 0 files removed, 0 files unresolved
  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
  $ hg --cwd b log --template "{desc}\n"
  a
  $ hg --cwd b qpush -a
  applying b.patch
  now at: b.patch
  $ hg --cwd b log --template "{desc}\n"
  imported patch b.patch
  a

test with normal collection

  $ cat > collections1.conf <<EOF
  > [paths]
  > /=$root/*
  > EOF
  $ hg serve -p $HGPORT1 -d --pid-file=hg.pid --webdir-conf collections1.conf \
  >     -A access-paths.log -E error-paths-1.log
  $ cat hg.pid >> $DAEMON_PIDS
  $ get-with-headers.py localhost:$HGPORT1 '?style=raw'
  200 Script output follows
  
  
  /a/
  /a/.hg/patches/
  
  $ hg qclone http://localhost:$HGPORT1/a c
  requesting all changes
  adding changesets
  adding manifests
  adding file changes
  added 2 changesets with 2 changes to 2 files
  new changesets cb9a9f314b8b:184916345baa
  requesting all changes
  adding changesets
  adding manifests
  adding file changes
  added 1 changesets with 3 changes to 3 files
  new changesets 4052ceaa8c4e
  updating to branch default
  3 files updated, 0 files merged, 0 files removed, 0 files unresolved
  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
  $ hg --cwd c log --template "{desc}\n"
  a
  $ hg --cwd c qpush -a
  applying b.patch
  now at: b.patch
  $ hg --cwd c log --template "{desc}\n"
  imported patch b.patch
  a

test with old-style collection

  $ cat > collections2.conf <<EOF
  > [collections]
  > $root=$root
  > EOF
  $ hg serve -p $HGPORT2 -d --pid-file=hg.pid --webdir-conf collections2.conf \
  >     -A access-paths.log -E error-paths-1.log
  $ cat hg.pid >> $DAEMON_PIDS
  $ get-with-headers.py localhost:$HGPORT2 '?style=raw'
  200 Script output follows
  
  
  /a/
  /a/.hg/patches/
  
  $ hg qclone http://localhost:$HGPORT2/a d
  requesting all changes
  adding changesets
  adding manifests
  adding file changes
  added 2 changesets with 2 changes to 2 files
  new changesets cb9a9f314b8b:184916345baa
  requesting all changes
  adding changesets
  adding manifests
  adding file changes
  added 1 changesets with 3 changes to 3 files
  new changesets 4052ceaa8c4e
  updating to branch default
  3 files updated, 0 files merged, 0 files removed, 0 files unresolved
  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
  $ hg --cwd d log --template "{desc}\n"
  a
  $ hg --cwd d qpush -a
  applying b.patch
  now at: b.patch
  $ hg --cwd d log --template "{desc}\n"
  imported patch b.patch
  a

test --mq works and uses correct repository config

  $ hg --cwd d outgoing --mq
  comparing with http://localhost:$HGPORT2/a/.hg/patches
  searching for changes
  no changes found
  [1]
  $ hg --cwd d log --mq --template '{rev} {desc|firstline}\n'
  0 b.patch

  $ killdaemons.py