view mercurial/pushkey.py @ 35767:5f5fb279fd39

streamclone: also stream caches to the client

When stream clone is used over bundle2, relevant cache files are also streamed.
This is expected to be a massive performance win for clone since no important
cache will have to be recomputed.

Some performance numbers:

(All times are wall-clock times in seconds, 2 attempts per case.)

# Mozilla-Central

## Clone over ssh over lan
V1 streaming: 234.3 239.6
V2 streaming: 248.4 243.7

## Clone over ssh over Internet
V1 streaming: 175.5 110.9
V2 streaming: 109.1 111.0

## Clone over HTTP over lan
V1 streaming: 105.3 105.6
V2 streaming: 112.7 111.4

## Clone over HTTP over internet
V1 streaming: 105.6 114.6
V2 streaming: 226.7 225.9

## Hg tags
V1 streaming (no cache): 1.084 1.071
V2 streaming (cache):    0.312 0.325

## Hg branches
V1 streaming (no cache):   14.047 14.148
V2 streaming (with cache):  0.312  0.333

# Pypy

## Clone over ssh over internet
V1 streaming: 29.4 30.1
V2 streaming: 31.2 30.1

## Clone over http over internet
V1 streaming: 29.7 29.7
V2 streaming: 75.2 72.9

(since ssh and lan are not affected, there seems to be an issue with how we
read/write the http stream on connection with latency, unrelated to the format)

## Hg tags
V1 streaming (no cache):   1.752 1.664
V2 streaming (with cache): 0.274 0.260

## Hg branches
V1 streaming (no cache):   4.469 4.728
V2 streaming (with cache): 0.318 0.321

# Private repository:
    * 500K revision revisions
    * 11K topological heads
    * 28K branch heads

## hg tags
no cache:   1543.332
with cache:    4.900

## hg branches
no cache:   91.828
with cache:  2.955

# pushkey.py - dispatching for pushing and pulling keys
#
# Copyright 2010 Matt Mackall <[e-mail]>
#
# This software may be used and distributed according to the terms of the
# GNU General Public License version 2 or any later version.

from __future__ import absolute_import

from . import (
    bookmarks,
    encoding,
    obsolete,
    phases,
)

def _nslist(repo):
    n = {}
    for k in _namespaces:
        n[k] = ""
    if not obsolete.isenabled(repo, obsolete.exchangeopt):
        n.pop('obsolete')
    return n

_namespaces = {"namespaces": (lambda *x: False, _nslist),
               "bookmarks": (bookmarks.pushbookmark, bookmarks.listbookmarks),
               "phases": (phases.pushphase, phases.listphases),
               "obsolete": (obsolete.pushmarker, obsolete.listmarkers),
              }

def register(namespace, pushkey, listkeys):
    _namespaces[namespace] = (pushkey, listkeys)

def _get(namespace):
    return _namespaces.get(namespace, (lambda *x: False, lambda *x: {}))

def push(repo, namespace, key, old, new):
    '''should succeed iff value was old'''
    pk = _get(namespace)[0]
    return pk(repo, key, old, new)

def list(repo, namespace):
    '''return a dict'''
    lk = _get(namespace)[1]
    return lk(repo)

encode = encoding.fromlocal

decode = encoding.tolocal

def encodekeys(keys):
    """encode the content of a pushkey namespace for exchange over the wire"""
    return '\n'.join(['%s\t%s' % (encode(k), encode(v)) for k, v in keys])

def decodekeys(data):
    """decode the content of a pushkey namespace from exchange over the wire"""
    result = {}
    for l in data.splitlines():
        k, v = l.split('\t')
        result[decode(k)] = decode(v)
    return result