mercurial/pushkey.py

nodemap: also use persistent nodemap for manifest

The manifest as a different usage pattern than the changelog. First, while the
lookup in changelog are not garanteed to match, the lookup in the manifest
nodemap come from changelog and will exist in the manifest. In addition, looking
up a manifest almost always result in unpacking a manifest an operation that
rarely come cheap.

Nevertheless, using a persistent nodemap provide a significant gain for some
operations.


For our measurementw, we use `hg cat --rev REV FILE` on the our reference
mozilla-try. On this repository the persistent nodemap cache is about 29 MB in
side for a total store side of 11,988 MB

File with large history (file: b2g/config/gaia.json, revision: 195a1146daa0)

    no optimisation:                             0.358s
    using mmap for index:                        0.297s (-0.061s)
    persistent nodemap for changelog only:       0.275s (-0.024s)
    persistent nodemap for manifest too:         0.258s (-0.017s)

File with small history (file: .hgignore, revision: 195a1146daa0)

    no optimisation:                             0.377s
    using mmap for index:                        0.296s (-0.061s)
    persistent nodemap for changelog only:       0.274s (-0.022s)
    persistent nodemap for manifest too:         0.257s (-0.017s)

Same file but using a revision (8ba995b74e18) with a smaller manifest (3944829
bytes vs  10 bytes)

    no optimisation:                             0.192s (-0.185s)
    using mmap for index:                        0.131s (-0.061s)
    persistent nodemap for changelog only:       0.106s (-0.025s)
    persistent nodemap for manifest too:         0.087s (-0.019s)

Differential Revision: https://phab.mercurial-scm.org/D8410

# pushkey.py - dispatching for pushing and pulling keys
#
# Copyright 2010 Matt Mackall <[e-mail]>
#
# This software may be used and distributed according to the terms of the
# GNU General Public License version 2 or any later version.

from __future__ import absolute_import

from . import (
    bookmarks,
    encoding,
    obsolete,
    phases,
)


def _nslist(repo):
    n = {}
    for k in _namespaces:
        n[k] = b""
    if not obsolete.isenabled(repo, obsolete.exchangeopt):
        n.pop(b'obsolete')
    return n


_namespaces = {
    b"namespaces": (lambda *x: False, _nslist),
    b"bookmarks": (bookmarks.pushbookmark, bookmarks.listbookmarks),
    b"phases": (phases.pushphase, phases.listphases),
    b"obsolete": (obsolete.pushmarker, obsolete.listmarkers),
}


def register(namespace, pushkey, listkeys):
    _namespaces[namespace] = (pushkey, listkeys)


def _get(namespace):
    return _namespaces.get(namespace, (lambda *x: False, lambda *x: {}))


def push(repo, namespace, key, old, new):
    '''should succeed iff value was old'''
    pk = _get(namespace)[0]
    return pk(repo, key, old, new)


def list(repo, namespace):
    '''return a dict'''
    lk = _get(namespace)[1]
    return lk(repo)


encode = encoding.fromlocal

decode = encoding.tolocal


def encodekeys(keys):
    """encode the content of a pushkey namespace for exchange over the wire"""
    return b'\n'.join([b'%s\t%s' % (encode(k), encode(v)) for k, v in keys])


def decodekeys(data):
    """decode the content of a pushkey namespace from exchange over the wire"""
    result = {}
    for l in data.splitlines():
        k, v = l.split(b'\t')
        result[decode(k)] = decode(v)
    return result