contrib/undumprevlog

nodemap: also use persistent nodemap for manifest

The manifest as a different usage pattern than the changelog. First, while the
lookup in changelog are not garanteed to match, the lookup in the manifest
nodemap come from changelog and will exist in the manifest. In addition, looking
up a manifest almost always result in unpacking a manifest an operation that
rarely come cheap.

Nevertheless, using a persistent nodemap provide a significant gain for some
operations.


For our measurementw, we use `hg cat --rev REV FILE` on the our reference
mozilla-try. On this repository the persistent nodemap cache is about 29 MB in
side for a total store side of 11,988 MB

File with large history (file: b2g/config/gaia.json, revision: 195a1146daa0)

    no optimisation:                             0.358s
    using mmap for index:                        0.297s (-0.061s)
    persistent nodemap for changelog only:       0.275s (-0.024s)
    persistent nodemap for manifest too:         0.258s (-0.017s)

File with small history (file: .hgignore, revision: 195a1146daa0)

    no optimisation:                             0.377s
    using mmap for index:                        0.296s (-0.061s)
    persistent nodemap for changelog only:       0.274s (-0.022s)
    persistent nodemap for manifest too:         0.257s (-0.017s)

Same file but using a revision (8ba995b74e18) with a smaller manifest (3944829
bytes vs  10 bytes)

    no optimisation:                             0.192s (-0.185s)
    using mmap for index:                        0.131s (-0.061s)
    persistent nodemap for changelog only:       0.106s (-0.025s)
    persistent nodemap for manifest too:         0.087s (-0.019s)

Differential Revision: https://phab.mercurial-scm.org/D8410

#!/usr/bin/env python
# Undump a dump from dumprevlog
# $ hg init
# $ undumprevlog < repo.dump

from __future__ import absolute_import, print_function

import sys
from mercurial import (
    encoding,
    node,
    pycompat,
    revlog,
    transaction,
    vfs as vfsmod,
)
from mercurial.utils import procutil

for fp in (sys.stdin, sys.stdout, sys.stderr):
    procutil.setbinary(fp)

opener = vfsmod.vfs(b'.', False)
tr = transaction.transaction(
    sys.stderr.write, opener, {b'store': opener}, b"undump.journal"
)
while True:
    l = sys.stdin.readline()
    if not l:
        break
    if l.startswith("file:"):
        f = encoding.strtolocal(l[6:-1])
        r = revlog.revlog(opener, f)
        pycompat.stdout.write(b'%s\n' % f)
    elif l.startswith("node:"):
        n = node.bin(l[6:-1])
    elif l.startswith("linkrev:"):
        lr = int(l[9:-1])
    elif l.startswith("parents:"):
        p = l[9:-1].split()
        p1 = node.bin(p[0])
        p2 = node.bin(p[1])
    elif l.startswith("length:"):
        length = int(l[8:-1])
        sys.stdin.readline()  # start marker
        d = encoding.strtolocal(sys.stdin.read(length))
        sys.stdin.readline()  # end marker
        r.addrevision(d, tr, lr, p1, p2)

tr.close()