contrib/dirstatenonnormalcheck.py
author Pulkit Goyal <pulkit@yandex-team.ru>
Sun, 26 Aug 2018 20:20:34 +0300
changeset 39356 c8e4eae84808
parent 35878 6e7fae8f1c6c
child 43076 2372284d9457
permissions -rw-r--r--
narrow: add server logic to send cg while widening without ellipsis Before this patch, if you try to widen a narrow clone without ellipsis enabled, it will be broken and the exchange.pull() done by tracked command to widen the clone will be no-op because no custom logic exists for this and server sees that we have all csets and it says `no changes found`. The widening with ellipsis send KILL for existing changegroups and send new changegroups because of the change in ellipsis hash, but we can prevent that in non-ellipsis cases. This patch adds server side logic to send the changegroups for the changesets which are on the client again with filelogs and manifests for the new includes. This is a very starting implementation and we send changegroups and manifests too while we can prevent them. Following things can definitely be improved in the logic this patch adds: 1) Send just the filelogs and treemanifests 2) Send the filelogs only for the additions in the include I tried 1) here but the code is coupled tightly and the way I was able to do that was hacking into the changegroup generation code in a very dirty way, like adding conditionals and preventing the yield. This patch also adds a 'widen' kwarg to prevent other commands except widening to go through that codepath. The test changes demonstrate that the new implementation is correct and fixes things. Differential Revision: https://phab.mercurial-scm.org/D4383
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
27591
127cc7f78475 dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff changeset
     1
# dirstatenonnormalcheck.py - extension to check the consistency of the
127cc7f78475 dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff changeset
     2
# dirstate's non-normal map
127cc7f78475 dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff changeset
     3
#
127cc7f78475 dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff changeset
     4
# For most operations on dirstate, this extensions checks that the nonnormalset
127cc7f78475 dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff changeset
     5
# contains the right entries.
127cc7f78475 dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff changeset
     6
# It compares the nonnormal file to a nonnormalset built from the map of all
127cc7f78475 dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff changeset
     7
# the files in the dirstate to check that they contain the same files.
127cc7f78475 dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff changeset
     8
127cc7f78475 dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff changeset
     9
from __future__ import absolute_import
127cc7f78475 dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff changeset
    10
127cc7f78475 dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff changeset
    11
from mercurial import (
127cc7f78475 dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff changeset
    12
    dirstate,
127cc7f78475 dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff changeset
    13
    extensions,
127cc7f78475 dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff changeset
    14
)
127cc7f78475 dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff changeset
    15
127cc7f78475 dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff changeset
    16
def nonnormalentries(dmap):
127cc7f78475 dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff changeset
    17
    """Compute nonnormal entries from dirstate's dmap"""
127cc7f78475 dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff changeset
    18
    res = set()
127cc7f78475 dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff changeset
    19
    for f, e in dmap.iteritems():
35878
6e7fae8f1c6c contrib: fix dirstatenonnormalcheck to work in Python 3
Augie Fackler <augie@google.com>
parents: 34674
diff changeset
    20
        if e[0] != b'n' or e[3] == -1:
27591
127cc7f78475 dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff changeset
    21
            res.add(f)
127cc7f78475 dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff changeset
    22
    return res
127cc7f78475 dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff changeset
    23
127cc7f78475 dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff changeset
    24
def checkconsistency(ui, orig, dmap, _nonnormalset, label):
127cc7f78475 dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff changeset
    25
    """Compute nonnormalset from dmap, check that it matches _nonnormalset"""
127cc7f78475 dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff changeset
    26
    nonnormalcomputedmap = nonnormalentries(dmap)
127cc7f78475 dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff changeset
    27
    if _nonnormalset != nonnormalcomputedmap:
35878
6e7fae8f1c6c contrib: fix dirstatenonnormalcheck to work in Python 3
Augie Fackler <augie@google.com>
parents: 34674
diff changeset
    28
        ui.develwarn(b"%s call to %s\n" % (label, orig), config=b'dirstate')
6e7fae8f1c6c contrib: fix dirstatenonnormalcheck to work in Python 3
Augie Fackler <augie@google.com>
parents: 34674
diff changeset
    29
        ui.develwarn(b"inconsistency in nonnormalset\n", config=b'dirstate')
6e7fae8f1c6c contrib: fix dirstatenonnormalcheck to work in Python 3
Augie Fackler <augie@google.com>
parents: 34674
diff changeset
    30
        ui.develwarn(b"[nonnormalset] %s\n" % _nonnormalset, config=b'dirstate')
6e7fae8f1c6c contrib: fix dirstatenonnormalcheck to work in Python 3
Augie Fackler <augie@google.com>
parents: 34674
diff changeset
    31
        ui.develwarn(b"[map] %s\n" % nonnormalcomputedmap, config=b'dirstate')
27591
127cc7f78475 dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff changeset
    32
127cc7f78475 dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff changeset
    33
def _checkdirstate(orig, self, arg):
127cc7f78475 dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff changeset
    34
    """Check nonnormal set consistency before and after the call to orig"""
34674
60927b19ed65 dirstate: move nonnormal and otherparent sets to dirstatemap
Durham Goode <durham@fb.com>
parents: 29100
diff changeset
    35
    checkconsistency(self._ui, orig, self._map, self._map.nonnormalset,
35878
6e7fae8f1c6c contrib: fix dirstatenonnormalcheck to work in Python 3
Augie Fackler <augie@google.com>
parents: 34674
diff changeset
    36
                     b"before")
27637
b502138f5faa cleanup: remove superfluous space after space after equals (python)
timeless <timeless@mozdev.org>
parents: 27591
diff changeset
    37
    r = orig(self, arg)
35878
6e7fae8f1c6c contrib: fix dirstatenonnormalcheck to work in Python 3
Augie Fackler <augie@google.com>
parents: 34674
diff changeset
    38
    checkconsistency(self._ui, orig, self._map, self._map.nonnormalset,
6e7fae8f1c6c contrib: fix dirstatenonnormalcheck to work in Python 3
Augie Fackler <augie@google.com>
parents: 34674
diff changeset
    39
                     b"after")
27591
127cc7f78475 dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff changeset
    40
    return r
127cc7f78475 dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff changeset
    41
127cc7f78475 dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff changeset
    42
def extsetup(ui):
127cc7f78475 dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff changeset
    43
    """Wrap functions modifying dirstate to check nonnormalset consistency"""
127cc7f78475 dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff changeset
    44
    dirstatecl = dirstate.dirstate
35878
6e7fae8f1c6c contrib: fix dirstatenonnormalcheck to work in Python 3
Augie Fackler <augie@google.com>
parents: 34674
diff changeset
    45
    devel = ui.configbool(b'devel', b'all-warnings')
6e7fae8f1c6c contrib: fix dirstatenonnormalcheck to work in Python 3
Augie Fackler <augie@google.com>
parents: 34674
diff changeset
    46
    paranoid = ui.configbool(b'experimental', b'nonnormalparanoidcheck')
27591
127cc7f78475 dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff changeset
    47
    if devel:
127cc7f78475 dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff changeset
    48
        extensions.wrapfunction(dirstatecl, '_writedirstate', _checkdirstate)
127cc7f78475 dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff changeset
    49
        if paranoid:
127cc7f78475 dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff changeset
    50
            # We don't do all these checks when paranoid is disable as it would
127cc7f78475 dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff changeset
    51
            # make the extension run very slowly on large repos
127cc7f78475 dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff changeset
    52
            extensions.wrapfunction(dirstatecl, 'normallookup', _checkdirstate)
127cc7f78475 dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff changeset
    53
            extensions.wrapfunction(dirstatecl, 'otherparent', _checkdirstate)
127cc7f78475 dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff changeset
    54
            extensions.wrapfunction(dirstatecl, 'normal', _checkdirstate)
127cc7f78475 dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff changeset
    55
            extensions.wrapfunction(dirstatecl, 'write', _checkdirstate)
127cc7f78475 dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff changeset
    56
            extensions.wrapfunction(dirstatecl, 'add', _checkdirstate)
127cc7f78475 dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff changeset
    57
            extensions.wrapfunction(dirstatecl, 'remove', _checkdirstate)
127cc7f78475 dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff changeset
    58
            extensions.wrapfunction(dirstatecl, 'merge', _checkdirstate)
127cc7f78475 dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff changeset
    59
            extensions.wrapfunction(dirstatecl, 'drop', _checkdirstate)