tests/autodiff.py

verify: check directory manifests

In repos with treemanifests, there is no specific verification of
directory manifest revlogs. It simply collects all file nodes by
reading each manifest delta. With treemanifests, that's means calling
the manifest._slowreaddelta(). If there are missing revlog entries in
a subdirectory revlog, 'hg verify' will simply report the exception
that occurred while trying to read the root manifest:


  manifest@0: reading delta 1700e2e92882: meta/b/00manifest.i@67688a370455: no node

This patch changes the verify code to load only the root manifest at
first and verify all revisions of it, then verify all revisions of
each direct subdirectory, and so on, recursively. The above message
becomes

  b/@0: parent-directory manifest refers to unknown revision 67688a370455

Since the new algorithm reads a single revlog at a time and in order,
'hg verify' on a treemanifest version of the hg core repo goes from
~50s to ~14s. As expected, there is no significant difference on a
repo with flat manifests.

# Extension dedicated to test patch.diff() upgrade modes

from __future__ import absolute_import

from mercurial import (
    cmdutil,
    error,
    patch,
    scmutil,
)

cmdtable = {}
command = cmdutil.command(cmdtable)

@command('autodiff',
    [('', 'git', '', 'git upgrade mode (yes/no/auto/warn/abort)')],
    '[OPTION]... [FILE]...')
def autodiff(ui, repo, *pats, **opts):
    diffopts = patch.difffeatureopts(ui, opts)
    git = opts.get('git', 'no')
    brokenfiles = set()
    losedatafn = None
    if git in ('yes', 'no'):
        diffopts.git = git == 'yes'
        diffopts.upgrade = False
    elif git == 'auto':
        diffopts.git = False
        diffopts.upgrade = True
    elif git == 'warn':
        diffopts.git = False
        diffopts.upgrade = True
        def losedatafn(fn=None, **kwargs):
            brokenfiles.add(fn)
            return True
    elif git == 'abort':
        diffopts.git = False
        diffopts.upgrade = True
        def losedatafn(fn=None, **kwargs):
            raise error.Abort('losing data for %s' % fn)
    else:
        raise error.Abort('--git must be yes, no or auto')

    node1, node2 = scmutil.revpair(repo, [])
    m = scmutil.match(repo[node2], pats, opts)
    it = patch.diff(repo, node1, node2, match=m, opts=diffopts,
                    losedatafn=losedatafn)
    for chunk in it:
        ui.write(chunk)
    for fn in sorted(brokenfiles):
        ui.write(('data lost for: %s\n' % fn))