annotate mercurial/verify.py @ 39583:ee1e74ee037c

formatter: fill missing resources by formatter, not by resource mapper While working on demand loading of ctx/fctx objects, I found it's weird to support lookup in both directions. For instance, fctx can be loaded from (ctx, path) pair, but ctx may also be derived from fctx.changectx() in the original mapping. If the original mapping has had fctx but no ctx, and if the new mapping provides {path}, we can't be sure if fctx should be updated by fctx'.changectx()[path] or not. This patch simply drops the support for the resolution in fctx -> ctx -> repo direction.
author Yuya Nishihara <yuya@tcha.org>
date Sat, 01 Sep 2018 13:21:45 +0900
parents f1186c292d03
children b63dee7bd0d9
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
2778
fdc232d8a193 Move repo.verify
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
1 # verify.py - repository integrity checking for Mercurial
fdc232d8a193 Move repo.verify
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
2 #
4635
63b9d2deed48 Updated copyright notices and add "and others" to "hg version"
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4395
diff changeset
3 # Copyright 2006, 2007 Matt Mackall <mpm@selenic.com>
2778
fdc232d8a193 Move repo.verify
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
4 #
8225
46293a0c7e9f updated license to be explicit about GPL version 2
Martin Geisler <mg@lazybytes.net>
parents: 8209
diff changeset
5 # This software may be used and distributed according to the terms of the
10263
25e572394f5c Update license to GPLv2+
Matt Mackall <mpm@selenic.com>
parents: 9690
diff changeset
6 # GNU General Public License version 2 or any later version.
2778
fdc232d8a193 Move repo.verify
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
7
25991
d21d1774c73b verify: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25846
diff changeset
8 from __future__ import absolute_import
d21d1774c73b verify: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25846
diff changeset
9
17860
a45b33f12627 verify: fix all doubled-slash sites (issue3665)
Bryan O'Sullivan <bryano@fb.com>
parents: 17851
diff changeset
10 import os
25991
d21d1774c73b verify: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25846
diff changeset
11
d21d1774c73b verify: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25846
diff changeset
12 from .i18n import _
d21d1774c73b verify: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25846
diff changeset
13 from .node import (
d21d1774c73b verify: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25846
diff changeset
14 nullid,
d21d1774c73b verify: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25846
diff changeset
15 short,
d21d1774c73b verify: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25846
diff changeset
16 )
d21d1774c73b verify: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25846
diff changeset
17
d21d1774c73b verify: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25846
diff changeset
18 from . import (
d21d1774c73b verify: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25846
diff changeset
19 error,
35585
35fb3367f72d py3: use pycompat.bytestr() instead of str()
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33499
diff changeset
20 pycompat,
25991
d21d1774c73b verify: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25846
diff changeset
21 revlog,
30866
5249b6470de9 verify: replace _validpath() by matcher
Martin von Zweigbergk <martinvonz@google.com>
parents: 30393
diff changeset
22 scmutil,
25991
d21d1774c73b verify: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25846
diff changeset
23 util,
d21d1774c73b verify: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25846
diff changeset
24 )
2778
fdc232d8a193 Move repo.verify
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
25
fdc232d8a193 Move repo.verify
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
26 def verify(repo):
27849
900d36a3e4dd with: use context manager in verify
Bryan O'Sullivan <bryano@fb.com>
parents: 27695
diff changeset
27 with repo.lock():
27444
6647401858ab verify: move widely used variables into class members
Durham Goode <durham@fb.com>
parents: 27443
diff changeset
28 return verifier(repo).verify()
4915
97b734fb9c6f Use try/finally pattern to cleanup locks and transactions
Matt Mackall <mpm@selenic.com>
parents: 4635
diff changeset
29
17860
a45b33f12627 verify: fix all doubled-slash sites (issue3665)
Bryan O'Sullivan <bryano@fb.com>
parents: 17851
diff changeset
30 def _normpath(f):
a45b33f12627 verify: fix all doubled-slash sites (issue3665)
Bryan O'Sullivan <bryano@fb.com>
parents: 17851
diff changeset
31 # under hg < 2.4, convert didn't sanitize paths properly, so a
a45b33f12627 verify: fix all doubled-slash sites (issue3665)
Bryan O'Sullivan <bryano@fb.com>
parents: 17851
diff changeset
32 # converted repo may contain repeated slashes
a45b33f12627 verify: fix all doubled-slash sites (issue3665)
Bryan O'Sullivan <bryano@fb.com>
parents: 17851
diff changeset
33 while '//' in f:
a45b33f12627 verify: fix all doubled-slash sites (issue3665)
Bryan O'Sullivan <bryano@fb.com>
parents: 17851
diff changeset
34 f = f.replace('//', '/')
a45b33f12627 verify: fix all doubled-slash sites (issue3665)
Bryan O'Sullivan <bryano@fb.com>
parents: 17851
diff changeset
35 return f
a45b33f12627 verify: fix all doubled-slash sites (issue3665)
Bryan O'Sullivan <bryano@fb.com>
parents: 17851
diff changeset
36
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
37 class verifier(object):
30866
5249b6470de9 verify: replace _validpath() by matcher
Martin von Zweigbergk <martinvonz@google.com>
parents: 30393
diff changeset
38 # The match argument is always None in hg core, but e.g. the narrowhg
5249b6470de9 verify: replace _validpath() by matcher
Martin von Zweigbergk <martinvonz@google.com>
parents: 30393
diff changeset
39 # extension will pass in a matcher here.
5249b6470de9 verify: replace _validpath() by matcher
Martin von Zweigbergk <martinvonz@google.com>
parents: 30393
diff changeset
40 def __init__(self, repo, match=None):
27444
6647401858ab verify: move widely used variables into class members
Durham Goode <durham@fb.com>
parents: 27443
diff changeset
41 self.repo = repo.unfiltered()
6647401858ab verify: move widely used variables into class members
Durham Goode <durham@fb.com>
parents: 27443
diff changeset
42 self.ui = repo.ui
30866
5249b6470de9 verify: replace _validpath() by matcher
Martin von Zweigbergk <martinvonz@google.com>
parents: 30393
diff changeset
43 self.match = match or scmutil.matchall(repo)
27444
6647401858ab verify: move widely used variables into class members
Durham Goode <durham@fb.com>
parents: 27443
diff changeset
44 self.badrevs = set()
27453
8462d7f2c4fe verify: clean up weird error/warning lists
Matt Mackall <mpm@selenic.com>
parents: 27450
diff changeset
45 self.errors = 0
8462d7f2c4fe verify: clean up weird error/warning lists
Matt Mackall <mpm@selenic.com>
parents: 27450
diff changeset
46 self.warnings = 0
27444
6647401858ab verify: move widely used variables into class members
Durham Goode <durham@fb.com>
parents: 27443
diff changeset
47 self.havecl = len(repo.changelog) > 0
39244
73cf21b2e8a6 manifest: add getstorage() to manifestlog and use it globally
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38397
diff changeset
48 self.havemf = len(repo.manifestlog.getstorage(b'')) > 0
27444
6647401858ab verify: move widely used variables into class members
Durham Goode <durham@fb.com>
parents: 27443
diff changeset
49 self.revlogv1 = repo.changelog.version != revlog.REVLOGV0
37302
00f18dd1d3d6 verify: remove dependence on repo.changectx()
Martin von Zweigbergk <martinvonz@google.com>
parents: 36577
diff changeset
50 self.lrugetctx = util.lrucachefunc(repo.__getitem__)
27444
6647401858ab verify: move widely used variables into class members
Durham Goode <durham@fb.com>
parents: 27443
diff changeset
51 self.refersmf = False
27445
cc178057ab49 verify: move fncachewarned up to a class variable
Durham Goode <durham@fb.com>
parents: 27444
diff changeset
52 self.fncachewarned = False
32288
a2ab9ebcd85b verify: add a config option to skip certain flag processors
Jun Wu <quark@fb.com>
parents: 32250
diff changeset
53 # developer config: verify.skipflags
a2ab9ebcd85b verify: add a config option to skip certain flag processors
Jun Wu <quark@fb.com>
parents: 32250
diff changeset
54 self.skipflags = repo.ui.configint('verify', 'skipflags')
37417
76d2115cb817 verify: allow suppressing warnings about extra files
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37410
diff changeset
55 self.warnorphanstorefiles = True
27444
6647401858ab verify: move widely used variables into class members
Durham Goode <durham@fb.com>
parents: 27443
diff changeset
56
27446
6b2c1a1871a6 verify: move warn() to a class level function
Durham Goode <durham@fb.com>
parents: 27445
diff changeset
57 def warn(self, msg):
6b2c1a1871a6 verify: move warn() to a class level function
Durham Goode <durham@fb.com>
parents: 27445
diff changeset
58 self.ui.warn(msg + "\n")
27453
8462d7f2c4fe verify: clean up weird error/warning lists
Matt Mackall <mpm@selenic.com>
parents: 27450
diff changeset
59 self.warnings += 1
27446
6b2c1a1871a6 verify: move warn() to a class level function
Durham Goode <durham@fb.com>
parents: 27445
diff changeset
60
27447
d1b91c10ce70 verify: move err() to be a class function
Durham Goode <durham@fb.com>
parents: 27446
diff changeset
61 def err(self, linkrev, msg, filename=None):
d1b91c10ce70 verify: move err() to be a class function
Durham Goode <durham@fb.com>
parents: 27446
diff changeset
62 if linkrev is not None:
d1b91c10ce70 verify: move err() to be a class function
Durham Goode <durham@fb.com>
parents: 27446
diff changeset
63 self.badrevs.add(linkrev)
36185
acc8e6e52af6 py3: use "%d" to convert integers to bytes
Pulkit Goyal <7895pulkit@gmail.com>
parents: 35585
diff changeset
64 linkrev = "%d" % linkrev
27447
d1b91c10ce70 verify: move err() to be a class function
Durham Goode <durham@fb.com>
parents: 27446
diff changeset
65 else:
d1b91c10ce70 verify: move err() to be a class function
Durham Goode <durham@fb.com>
parents: 27446
diff changeset
66 linkrev = '?'
d1b91c10ce70 verify: move err() to be a class function
Durham Goode <durham@fb.com>
parents: 27446
diff changeset
67 msg = "%s: %s" % (linkrev, msg)
d1b91c10ce70 verify: move err() to be a class function
Durham Goode <durham@fb.com>
parents: 27446
diff changeset
68 if filename:
d1b91c10ce70 verify: move err() to be a class function
Durham Goode <durham@fb.com>
parents: 27446
diff changeset
69 msg = "%s@%s" % (filename, msg)
d1b91c10ce70 verify: move err() to be a class function
Durham Goode <durham@fb.com>
parents: 27446
diff changeset
70 self.ui.warn(" " + msg + "\n")
27453
8462d7f2c4fe verify: clean up weird error/warning lists
Matt Mackall <mpm@selenic.com>
parents: 27450
diff changeset
71 self.errors += 1
27447
d1b91c10ce70 verify: move err() to be a class function
Durham Goode <durham@fb.com>
parents: 27446
diff changeset
72
27448
f4f2179077cb verify: move exc() function onto class
Durham Goode <durham@fb.com>
parents: 27447
diff changeset
73 def exc(self, linkrev, msg, inst, filename=None):
36577
d85ef895d5f6 verify: fix exception formatting bug in Python 3
Augie Fackler <augie@google.com>
parents: 36339
diff changeset
74 fmsg = pycompat.bytestr(inst)
d85ef895d5f6 verify: fix exception formatting bug in Python 3
Augie Fackler <augie@google.com>
parents: 36339
diff changeset
75 if not fmsg:
d85ef895d5f6 verify: fix exception formatting bug in Python 3
Augie Fackler <augie@google.com>
parents: 36339
diff changeset
76 fmsg = pycompat.byterepr(inst)
d85ef895d5f6 verify: fix exception formatting bug in Python 3
Augie Fackler <augie@google.com>
parents: 36339
diff changeset
77 self.err(linkrev, "%s: %s" % (msg, fmsg), filename)
27448
f4f2179077cb verify: move exc() function onto class
Durham Goode <durham@fb.com>
parents: 27447
diff changeset
78
27642
f6457349985b verify: move checklog() onto class
Durham Goode <durham@fb.com>
parents: 27453
diff changeset
79 def checklog(self, obj, name, linkrev):
f6457349985b verify: move checklog() onto class
Durham Goode <durham@fb.com>
parents: 27453
diff changeset
80 if not len(obj) and (self.havecl or self.havemf):
f6457349985b verify: move checklog() onto class
Durham Goode <durham@fb.com>
parents: 27453
diff changeset
81 self.err(linkrev, _("empty or missing %s") % name)
f6457349985b verify: move checklog() onto class
Durham Goode <durham@fb.com>
parents: 27453
diff changeset
82 return
f6457349985b verify: move checklog() onto class
Durham Goode <durham@fb.com>
parents: 27453
diff changeset
83
f6457349985b verify: move checklog() onto class
Durham Goode <durham@fb.com>
parents: 27453
diff changeset
84 d = obj.checksize()
f6457349985b verify: move checklog() onto class
Durham Goode <durham@fb.com>
parents: 27453
diff changeset
85 if d[0]:
f6457349985b verify: move checklog() onto class
Durham Goode <durham@fb.com>
parents: 27453
diff changeset
86 self.err(None, _("data length off by %d bytes") % d[0], name)
f6457349985b verify: move checklog() onto class
Durham Goode <durham@fb.com>
parents: 27453
diff changeset
87 if d[1]:
f6457349985b verify: move checklog() onto class
Durham Goode <durham@fb.com>
parents: 27453
diff changeset
88 self.err(None, _("index contains %d extra bytes") % d[1], name)
f6457349985b verify: move checklog() onto class
Durham Goode <durham@fb.com>
parents: 27453
diff changeset
89
f6457349985b verify: move checklog() onto class
Durham Goode <durham@fb.com>
parents: 27453
diff changeset
90 if obj.version != revlog.REVLOGV0:
f6457349985b verify: move checklog() onto class
Durham Goode <durham@fb.com>
parents: 27453
diff changeset
91 if not self.revlogv1:
f6457349985b verify: move checklog() onto class
Durham Goode <durham@fb.com>
parents: 27453
diff changeset
92 self.warn(_("warning: `%s' uses revlog format 1") % name)
f6457349985b verify: move checklog() onto class
Durham Goode <durham@fb.com>
parents: 27453
diff changeset
93 elif self.revlogv1:
f6457349985b verify: move checklog() onto class
Durham Goode <durham@fb.com>
parents: 27453
diff changeset
94 self.warn(_("warning: `%s' uses revlog format 0") % name)
f6457349985b verify: move checklog() onto class
Durham Goode <durham@fb.com>
parents: 27453
diff changeset
95
27643
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
96 def checkentry(self, obj, i, node, seen, linkrevs, f):
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
97 lr = obj.linkrev(obj.rev(node))
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
98 if lr < 0 or (self.havecl and lr not in linkrevs):
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
99 if lr < 0 or lr >= len(self.repo.changelog):
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
100 msg = _("rev %d points to nonexistent changeset %d")
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
101 else:
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
102 msg = _("rev %d points to unexpected changeset %d")
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
103 self.err(None, msg % (i, lr), f)
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
104 if linkrevs:
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
105 if f and len(linkrevs) > 1:
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
106 try:
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
107 # attempt to filter down to real linkrevs
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
108 linkrevs = [l for l in linkrevs
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
109 if self.lrugetctx(l)[f].filenode() == node]
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
110 except Exception:
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
111 pass
35585
35fb3367f72d py3: use pycompat.bytestr() instead of str()
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33499
diff changeset
112 self.warn(_(" (expected %s)") % " ".join
35fb3367f72d py3: use pycompat.bytestr() instead of str()
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33499
diff changeset
113 (map(pycompat.bytestr, linkrevs)))
27643
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
114 lr = None # can't be trusted
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
115
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
116 try:
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
117 p1, p2 = obj.parents(node)
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
118 if p1 not in seen and p1 != nullid:
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
119 self.err(lr, _("unknown parent 1 %s of %s") %
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
120 (short(p1), short(node)), f)
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
121 if p2 not in seen and p2 != nullid:
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
122 self.err(lr, _("unknown parent 2 %s of %s") %
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
123 (short(p2), short(node)), f)
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
124 except Exception as inst:
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
125 self.exc(lr, _("checking parents of %s") % short(node), inst, f)
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
126
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
127 if node in seen:
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
128 self.err(lr, _("duplicate revision %d (%d)") % (i, seen[node]), f)
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
129 seen[node] = i
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
130 return lr
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
131
27444
6647401858ab verify: move widely used variables into class members
Durham Goode <durham@fb.com>
parents: 27443
diff changeset
132 def verify(self):
6647401858ab verify: move widely used variables into class members
Durham Goode <durham@fb.com>
parents: 27443
diff changeset
133 repo = self.repo
27648
e72e669dd51f verify: get rid of some unnecessary local variables
Durham Goode <durham@fb.com>
parents: 27647
diff changeset
134
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
135 ui = repo.ui
2778
fdc232d8a193 Move repo.verify
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
136
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
137 if not repo.url().startswith('file:'):
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
138 raise error.Abort(_("cannot verify bundle or remote repos"))
6752
e79a8f36c2a5 verify: lots of refactoring
Matt Mackall <mpm@selenic.com>
parents: 6751
diff changeset
139
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
140 if os.path.exists(repo.sjoin("journal")):
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
141 ui.warn(_("abandoned transaction found - run hg recover\n"))
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
142
27648
e72e669dd51f verify: get rid of some unnecessary local variables
Durham Goode <durham@fb.com>
parents: 27647
diff changeset
143 if ui.verbose or not self.revlogv1:
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
144 ui.status(_("repository uses revlog format %d\n") %
27648
e72e669dd51f verify: get rid of some unnecessary local variables
Durham Goode <durham@fb.com>
parents: 27647
diff changeset
145 (self.revlogv1 and 1 or 0))
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
146
27695
fb0cc863d172 verify: replace "output parameters" by return values
Martin von Zweigbergk <martinvonz@google.com>
parents: 27648
diff changeset
147 mflinkrevs, filelinkrevs = self._verifychangelog()
27647
2c2858f3c1bb verify: move changelog verificaiton to its own function
Durham Goode <durham@fb.com>
parents: 27646
diff changeset
148
27695
fb0cc863d172 verify: replace "output parameters" by return values
Martin von Zweigbergk <martinvonz@google.com>
parents: 27648
diff changeset
149 filenodes = self._verifymanifest(mflinkrevs)
28111
06205989264b verify: move cross-checking of changeset/manifest out of _crosscheckfiles()
Martin von Zweigbergk <martinvonz@google.com>
parents: 28007
diff changeset
150 del mflinkrevs
27647
2c2858f3c1bb verify: move changelog verificaiton to its own function
Durham Goode <durham@fb.com>
parents: 27646
diff changeset
151
28111
06205989264b verify: move cross-checking of changeset/manifest out of _crosscheckfiles()
Martin von Zweigbergk <martinvonz@google.com>
parents: 28007
diff changeset
152 self._crosscheckfiles(filelinkrevs, filenodes)
27647
2c2858f3c1bb verify: move changelog verificaiton to its own function
Durham Goode <durham@fb.com>
parents: 27646
diff changeset
153
2c2858f3c1bb verify: move changelog verificaiton to its own function
Durham Goode <durham@fb.com>
parents: 27646
diff changeset
154 totalfiles, filerevisions = self._verifyfiles(filenodes, filelinkrevs)
2c2858f3c1bb verify: move changelog verificaiton to its own function
Durham Goode <durham@fb.com>
parents: 27646
diff changeset
155
39489
f1186c292d03 verify: make output less confusing (issue5924)
Meirambek Omyrzak <meirambek77@gmail.com>
parents: 39244
diff changeset
156 ui.status(_("checked %d changesets with %d changes to %d files\n") %
f1186c292d03 verify: make output less confusing (issue5924)
Meirambek Omyrzak <meirambek77@gmail.com>
parents: 39244
diff changeset
157 (len(repo.changelog), filerevisions, totalfiles))
27647
2c2858f3c1bb verify: move changelog verificaiton to its own function
Durham Goode <durham@fb.com>
parents: 27646
diff changeset
158 if self.warnings:
2c2858f3c1bb verify: move changelog verificaiton to its own function
Durham Goode <durham@fb.com>
parents: 27646
diff changeset
159 ui.warn(_("%d warnings encountered!\n") % self.warnings)
2c2858f3c1bb verify: move changelog verificaiton to its own function
Durham Goode <durham@fb.com>
parents: 27646
diff changeset
160 if self.fncachewarned:
2c2858f3c1bb verify: move changelog verificaiton to its own function
Durham Goode <durham@fb.com>
parents: 27646
diff changeset
161 ui.warn(_('hint: run "hg debugrebuildfncache" to recover from '
2c2858f3c1bb verify: move changelog verificaiton to its own function
Durham Goode <durham@fb.com>
parents: 27646
diff changeset
162 'corrupt fncache\n'))
2c2858f3c1bb verify: move changelog verificaiton to its own function
Durham Goode <durham@fb.com>
parents: 27646
diff changeset
163 if self.errors:
2c2858f3c1bb verify: move changelog verificaiton to its own function
Durham Goode <durham@fb.com>
parents: 27646
diff changeset
164 ui.warn(_("%d integrity errors encountered!\n") % self.errors)
27648
e72e669dd51f verify: get rid of some unnecessary local variables
Durham Goode <durham@fb.com>
parents: 27647
diff changeset
165 if self.badrevs:
27647
2c2858f3c1bb verify: move changelog verificaiton to its own function
Durham Goode <durham@fb.com>
parents: 27646
diff changeset
166 ui.warn(_("(first damaged changeset appears to be %d)\n")
27648
e72e669dd51f verify: get rid of some unnecessary local variables
Durham Goode <durham@fb.com>
parents: 27647
diff changeset
167 % min(self.badrevs))
27647
2c2858f3c1bb verify: move changelog verificaiton to its own function
Durham Goode <durham@fb.com>
parents: 27646
diff changeset
168 return 1
2c2858f3c1bb verify: move changelog verificaiton to its own function
Durham Goode <durham@fb.com>
parents: 27646
diff changeset
169
27695
fb0cc863d172 verify: replace "output parameters" by return values
Martin von Zweigbergk <martinvonz@google.com>
parents: 27648
diff changeset
170 def _verifychangelog(self):
27647
2c2858f3c1bb verify: move changelog verificaiton to its own function
Durham Goode <durham@fb.com>
parents: 27646
diff changeset
171 ui = self.ui
2c2858f3c1bb verify: move changelog verificaiton to its own function
Durham Goode <durham@fb.com>
parents: 27646
diff changeset
172 repo = self.repo
30866
5249b6470de9 verify: replace _validpath() by matcher
Martin von Zweigbergk <martinvonz@google.com>
parents: 30393
diff changeset
173 match = self.match
27647
2c2858f3c1bb verify: move changelog verificaiton to its own function
Durham Goode <durham@fb.com>
parents: 27646
diff changeset
174 cl = repo.changelog
2c2858f3c1bb verify: move changelog verificaiton to its own function
Durham Goode <durham@fb.com>
parents: 27646
diff changeset
175
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
176 ui.status(_("checking changesets\n"))
27695
fb0cc863d172 verify: replace "output parameters" by return values
Martin von Zweigbergk <martinvonz@google.com>
parents: 27648
diff changeset
177 mflinkrevs = {}
fb0cc863d172 verify: replace "output parameters" by return values
Martin von Zweigbergk <martinvonz@google.com>
parents: 27648
diff changeset
178 filelinkrevs = {}
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
179 seen = {}
27642
f6457349985b verify: move checklog() onto class
Durham Goode <durham@fb.com>
parents: 27453
diff changeset
180 self.checklog(cl, "changelog", 0)
38397
1249475f0bd6 verify: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 38396
diff changeset
181 progress = ui.makeprogress(_('checking'), unit=_('changesets'),
1249475f0bd6 verify: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 38396
diff changeset
182 total=len(repo))
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
183 for i in repo:
38397
1249475f0bd6 verify: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 38396
diff changeset
184 progress.update(i)
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
185 n = cl.node(i)
27643
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
186 self.checkentry(cl, i, n, seen, [i], "changelog")
2778
fdc232d8a193 Move repo.verify
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
187
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
188 try:
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
189 changes = cl.read(n)
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
190 if changes[0] != nullid:
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
191 mflinkrevs.setdefault(changes[0], []).append(i)
27444
6647401858ab verify: move widely used variables into class members
Durham Goode <durham@fb.com>
parents: 27443
diff changeset
192 self.refersmf = True
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
193 for f in changes[3]:
30866
5249b6470de9 verify: replace _validpath() by matcher
Martin von Zweigbergk <martinvonz@google.com>
parents: 30393
diff changeset
194 if match(f):
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
195 filelinkrevs.setdefault(_normpath(f), []).append(i)
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
196 except Exception as inst:
27444
6647401858ab verify: move widely used variables into class members
Durham Goode <durham@fb.com>
parents: 27443
diff changeset
197 self.refersmf = True
27448
f4f2179077cb verify: move exc() function onto class
Durham Goode <durham@fb.com>
parents: 27447
diff changeset
198 self.exc(i, _("unpacking changeset %s") % short(n), inst)
38397
1249475f0bd6 verify: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 38396
diff changeset
199 progress.complete()
27695
fb0cc863d172 verify: replace "output parameters" by return values
Martin von Zweigbergk <martinvonz@google.com>
parents: 27648
diff changeset
200 return mflinkrevs, filelinkrevs
2778
fdc232d8a193 Move repo.verify
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
201
28205
53f42c8d5f71 verify: show progress while verifying dirlogs
Martin von Zweigbergk <martinvonz@google.com>
parents: 28204
diff changeset
202 def _verifymanifest(self, mflinkrevs, dir="", storefiles=None,
38396
0ddbe03c5aaa verify: use progress helper for subdirectory progress
Martin von Zweigbergk <martinvonz@google.com>
parents: 37417
diff changeset
203 subdirprogress=None):
27646
8f43793382c6 verify: move manifest verification to its own function
Durham Goode <durham@fb.com>
parents: 27645
diff changeset
204 repo = self.repo
8f43793382c6 verify: move manifest verification to its own function
Durham Goode <durham@fb.com>
parents: 27645
diff changeset
205 ui = self.ui
30866
5249b6470de9 verify: replace _validpath() by matcher
Martin von Zweigbergk <martinvonz@google.com>
parents: 30393
diff changeset
206 match = self.match
30295
f65faa4422c8 manifest: remove manifest.readshallowdelta
Durham Goode <durham@fb.com>
parents: 28467
diff changeset
207 mfl = self.repo.manifestlog
39244
73cf21b2e8a6 manifest: add getstorage() to manifestlog and use it globally
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38397
diff changeset
208 mf = mfl.getstorage(dir)
27646
8f43793382c6 verify: move manifest verification to its own function
Durham Goode <durham@fb.com>
parents: 27645
diff changeset
209
28203
7297e9e13a8a verify: check directory manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28115
diff changeset
210 if not dir:
7297e9e13a8a verify: check directory manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28115
diff changeset
211 self.ui.status(_("checking manifests\n"))
7297e9e13a8a verify: check directory manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28115
diff changeset
212
27695
fb0cc863d172 verify: replace "output parameters" by return values
Martin von Zweigbergk <martinvonz@google.com>
parents: 27648
diff changeset
213 filenodes = {}
28203
7297e9e13a8a verify: check directory manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28115
diff changeset
214 subdirnodes = {}
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
215 seen = {}
28115
bd279da57e4b verify: extract "manifest" constant into variable
Martin von Zweigbergk <martinvonz@google.com>
parents: 28114
diff changeset
216 label = "manifest"
28203
7297e9e13a8a verify: check directory manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28115
diff changeset
217 if dir:
7297e9e13a8a verify: check directory manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28115
diff changeset
218 label = dir
28204
962921c330b0 verify: check for orphaned dirlogs
Martin von Zweigbergk <martinvonz@google.com>
parents: 28203
diff changeset
219 revlogfiles = mf.files()
962921c330b0 verify: check for orphaned dirlogs
Martin von Zweigbergk <martinvonz@google.com>
parents: 28203
diff changeset
220 storefiles.difference_update(revlogfiles)
38396
0ddbe03c5aaa verify: use progress helper for subdirectory progress
Martin von Zweigbergk <martinvonz@google.com>
parents: 37417
diff changeset
221 if subdirprogress: # should be true since we're in a subdirectory
0ddbe03c5aaa verify: use progress helper for subdirectory progress
Martin von Zweigbergk <martinvonz@google.com>
parents: 37417
diff changeset
222 subdirprogress.increment()
27444
6647401858ab verify: move widely used variables into class members
Durham Goode <durham@fb.com>
parents: 27443
diff changeset
223 if self.refersmf:
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
224 # Do not check manifest if there are only changelog entries with
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
225 # null manifests.
28115
bd279da57e4b verify: extract "manifest" constant into variable
Martin von Zweigbergk <martinvonz@google.com>
parents: 28114
diff changeset
226 self.checklog(mf, label, 0)
38397
1249475f0bd6 verify: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 38396
diff changeset
227 progress = ui.makeprogress(_('checking'), unit=_('manifests'),
1249475f0bd6 verify: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 38396
diff changeset
228 total=len(mf))
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
229 for i in mf:
28203
7297e9e13a8a verify: check directory manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28115
diff changeset
230 if not dir:
38397
1249475f0bd6 verify: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 38396
diff changeset
231 progress.update(i)
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
232 n = mf.node(i)
28115
bd279da57e4b verify: extract "manifest" constant into variable
Martin von Zweigbergk <martinvonz@google.com>
parents: 28114
diff changeset
233 lr = self.checkentry(mf, i, n, seen, mflinkrevs.get(n, []), label)
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
234 if n in mflinkrevs:
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
235 del mflinkrevs[n]
28203
7297e9e13a8a verify: check directory manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28115
diff changeset
236 elif dir:
7297e9e13a8a verify: check directory manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28115
diff changeset
237 self.err(lr, _("%s not in parent-directory manifest") %
7297e9e13a8a verify: check directory manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28115
diff changeset
238 short(n), label)
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
239 else:
28115
bd279da57e4b verify: extract "manifest" constant into variable
Martin von Zweigbergk <martinvonz@google.com>
parents: 28114
diff changeset
240 self.err(lr, _("%s not in changesets") % short(n), label)
2778
fdc232d8a193 Move repo.verify
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
241
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
242 try:
30295
f65faa4422c8 manifest: remove manifest.readshallowdelta
Durham Goode <durham@fb.com>
parents: 28467
diff changeset
243 mfdelta = mfl.get(dir, n).readdelta(shallow=True)
f65faa4422c8 manifest: remove manifest.readshallowdelta
Durham Goode <durham@fb.com>
parents: 28467
diff changeset
244 for f, fn, fl in mfdelta.iterentries():
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
245 if not f:
28203
7297e9e13a8a verify: check directory manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28115
diff changeset
246 self.err(lr, _("entry without name in manifest"))
7297e9e13a8a verify: check directory manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28115
diff changeset
247 elif f == "/dev/null": # ignore this in very old repos
7297e9e13a8a verify: check directory manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28115
diff changeset
248 continue
7297e9e13a8a verify: check directory manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28115
diff changeset
249 fullpath = dir + _normpath(f)
7297e9e13a8a verify: check directory manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28115
diff changeset
250 if fl == 't':
30866
5249b6470de9 verify: replace _validpath() by matcher
Martin von Zweigbergk <martinvonz@google.com>
parents: 30393
diff changeset
251 if not match.visitdir(fullpath):
5249b6470de9 verify: replace _validpath() by matcher
Martin von Zweigbergk <martinvonz@google.com>
parents: 30393
diff changeset
252 continue
28203
7297e9e13a8a verify: check directory manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28115
diff changeset
253 subdirnodes.setdefault(fullpath + '/', {}).setdefault(
7297e9e13a8a verify: check directory manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28115
diff changeset
254 fn, []).append(lr)
7297e9e13a8a verify: check directory manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28115
diff changeset
255 else:
30866
5249b6470de9 verify: replace _validpath() by matcher
Martin von Zweigbergk <martinvonz@google.com>
parents: 30393
diff changeset
256 if not match(fullpath):
5249b6470de9 verify: replace _validpath() by matcher
Martin von Zweigbergk <martinvonz@google.com>
parents: 30393
diff changeset
257 continue
28203
7297e9e13a8a verify: check directory manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28115
diff changeset
258 filenodes.setdefault(fullpath, {}).setdefault(fn, lr)
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
259 except Exception as inst:
28115
bd279da57e4b verify: extract "manifest" constant into variable
Martin von Zweigbergk <martinvonz@google.com>
parents: 28114
diff changeset
260 self.exc(lr, _("reading delta %s") % short(n), inst, label)
28203
7297e9e13a8a verify: check directory manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28115
diff changeset
261 if not dir:
38397
1249475f0bd6 verify: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 38396
diff changeset
262 progress.complete()
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
263
28111
06205989264b verify: move cross-checking of changeset/manifest out of _crosscheckfiles()
Martin von Zweigbergk <martinvonz@google.com>
parents: 28007
diff changeset
264 if self.havemf:
06205989264b verify: move cross-checking of changeset/manifest out of _crosscheckfiles()
Martin von Zweigbergk <martinvonz@google.com>
parents: 28007
diff changeset
265 for c, m in sorted([(c, m) for m in mflinkrevs
06205989264b verify: move cross-checking of changeset/manifest out of _crosscheckfiles()
Martin von Zweigbergk <martinvonz@google.com>
parents: 28007
diff changeset
266 for c in mflinkrevs[m]]):
28203
7297e9e13a8a verify: check directory manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28115
diff changeset
267 if dir:
7297e9e13a8a verify: check directory manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28115
diff changeset
268 self.err(c, _("parent-directory manifest refers to unknown "
7297e9e13a8a verify: check directory manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28115
diff changeset
269 "revision %s") % short(m), label)
7297e9e13a8a verify: check directory manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28115
diff changeset
270 else:
7297e9e13a8a verify: check directory manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28115
diff changeset
271 self.err(c, _("changeset refers to unknown revision %s") %
7297e9e13a8a verify: check directory manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28115
diff changeset
272 short(m), label)
7297e9e13a8a verify: check directory manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28115
diff changeset
273
7297e9e13a8a verify: check directory manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28115
diff changeset
274 if not dir and subdirnodes:
7297e9e13a8a verify: check directory manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28115
diff changeset
275 self.ui.status(_("checking directory manifests\n"))
28204
962921c330b0 verify: check for orphaned dirlogs
Martin von Zweigbergk <martinvonz@google.com>
parents: 28203
diff changeset
276 storefiles = set()
28205
53f42c8d5f71 verify: show progress while verifying dirlogs
Martin von Zweigbergk <martinvonz@google.com>
parents: 28204
diff changeset
277 subdirs = set()
28204
962921c330b0 verify: check for orphaned dirlogs
Martin von Zweigbergk <martinvonz@google.com>
parents: 28203
diff changeset
278 revlogv1 = self.revlogv1
962921c330b0 verify: check for orphaned dirlogs
Martin von Zweigbergk <martinvonz@google.com>
parents: 28203
diff changeset
279 for f, f2, size in repo.store.datafiles():
962921c330b0 verify: check for orphaned dirlogs
Martin von Zweigbergk <martinvonz@google.com>
parents: 28203
diff changeset
280 if not f:
962921c330b0 verify: check for orphaned dirlogs
Martin von Zweigbergk <martinvonz@google.com>
parents: 28203
diff changeset
281 self.err(None, _("cannot decode filename '%s'") % f2)
962921c330b0 verify: check for orphaned dirlogs
Martin von Zweigbergk <martinvonz@google.com>
parents: 28203
diff changeset
282 elif (size > 0 or not revlogv1) and f.startswith('meta/'):
962921c330b0 verify: check for orphaned dirlogs
Martin von Zweigbergk <martinvonz@google.com>
parents: 28203
diff changeset
283 storefiles.add(_normpath(f))
28205
53f42c8d5f71 verify: show progress while verifying dirlogs
Martin von Zweigbergk <martinvonz@google.com>
parents: 28204
diff changeset
284 subdirs.add(os.path.dirname(f))
38396
0ddbe03c5aaa verify: use progress helper for subdirectory progress
Martin von Zweigbergk <martinvonz@google.com>
parents: 37417
diff changeset
285 subdirprogress = ui.makeprogress(_('checking'), unit=_('manifests'),
0ddbe03c5aaa verify: use progress helper for subdirectory progress
Martin von Zweigbergk <martinvonz@google.com>
parents: 37417
diff changeset
286 total=len(subdirs))
28204
962921c330b0 verify: check for orphaned dirlogs
Martin von Zweigbergk <martinvonz@google.com>
parents: 28203
diff changeset
287
28203
7297e9e13a8a verify: check directory manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28115
diff changeset
288 for subdir, linkrevs in subdirnodes.iteritems():
28205
53f42c8d5f71 verify: show progress while verifying dirlogs
Martin von Zweigbergk <martinvonz@google.com>
parents: 28204
diff changeset
289 subdirfilenodes = self._verifymanifest(linkrevs, subdir, storefiles,
38396
0ddbe03c5aaa verify: use progress helper for subdirectory progress
Martin von Zweigbergk <martinvonz@google.com>
parents: 37417
diff changeset
290 subdirprogress)
28203
7297e9e13a8a verify: check directory manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28115
diff changeset
291 for f, onefilenodes in subdirfilenodes.iteritems():
7297e9e13a8a verify: check directory manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28115
diff changeset
292 filenodes.setdefault(f, {}).update(onefilenodes)
28111
06205989264b verify: move cross-checking of changeset/manifest out of _crosscheckfiles()
Martin von Zweigbergk <martinvonz@google.com>
parents: 28007
diff changeset
293
28204
962921c330b0 verify: check for orphaned dirlogs
Martin von Zweigbergk <martinvonz@google.com>
parents: 28203
diff changeset
294 if not dir and subdirnodes:
38396
0ddbe03c5aaa verify: use progress helper for subdirectory progress
Martin von Zweigbergk <martinvonz@google.com>
parents: 37417
diff changeset
295 subdirprogress.complete()
37417
76d2115cb817 verify: allow suppressing warnings about extra files
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37410
diff changeset
296 if self.warnorphanstorefiles:
76d2115cb817 verify: allow suppressing warnings about extra files
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37410
diff changeset
297 for f in sorted(storefiles):
76d2115cb817 verify: allow suppressing warnings about extra files
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37410
diff changeset
298 self.warn(_("warning: orphan data file '%s'") % f)
28204
962921c330b0 verify: check for orphaned dirlogs
Martin von Zweigbergk <martinvonz@google.com>
parents: 28203
diff changeset
299
27695
fb0cc863d172 verify: replace "output parameters" by return values
Martin von Zweigbergk <martinvonz@google.com>
parents: 27648
diff changeset
300 return filenodes
27645
df8973e1fb45 verify: move file cross checking to its own function
Durham Goode <durham@fb.com>
parents: 27644
diff changeset
301
28111
06205989264b verify: move cross-checking of changeset/manifest out of _crosscheckfiles()
Martin von Zweigbergk <martinvonz@google.com>
parents: 28007
diff changeset
302 def _crosscheckfiles(self, filelinkrevs, filenodes):
27645
df8973e1fb45 verify: move file cross checking to its own function
Durham Goode <durham@fb.com>
parents: 27644
diff changeset
303 repo = self.repo
df8973e1fb45 verify: move file cross checking to its own function
Durham Goode <durham@fb.com>
parents: 27644
diff changeset
304 ui = self.ui
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
305 ui.status(_("crosschecking files in changesets and manifests\n"))
2778
fdc232d8a193 Move repo.verify
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
306
28111
06205989264b verify: move cross-checking of changeset/manifest out of _crosscheckfiles()
Martin von Zweigbergk <martinvonz@google.com>
parents: 28007
diff changeset
307 total = len(filelinkrevs) + len(filenodes)
38397
1249475f0bd6 verify: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 38396
diff changeset
308 progress = ui.makeprogress(_('crosschecking'), total=total)
27645
df8973e1fb45 verify: move file cross checking to its own function
Durham Goode <durham@fb.com>
parents: 27644
diff changeset
309 if self.havemf:
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
310 for f in sorted(filelinkrevs):
38397
1249475f0bd6 verify: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 38396
diff changeset
311 progress.increment()
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
312 if f not in filenodes:
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
313 lr = filelinkrevs[f][0]
27447
d1b91c10ce70 verify: move err() to be a class function
Durham Goode <durham@fb.com>
parents: 27446
diff changeset
314 self.err(lr, _("in changeset but not in manifest"), f)
6892
dab95717058d verify: check repo.store
Adrian Buehlmann <adrian@cadifra.com>
parents: 6889
diff changeset
315
27645
df8973e1fb45 verify: move file cross checking to its own function
Durham Goode <durham@fb.com>
parents: 27644
diff changeset
316 if self.havecl:
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
317 for f in sorted(filenodes):
38397
1249475f0bd6 verify: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 38396
diff changeset
318 progress.increment()
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
319 if f not in filelinkrevs:
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
320 try:
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
321 fl = repo.file(f)
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
322 lr = min([fl.linkrev(fl.rev(n)) for n in filenodes[f]])
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
323 except Exception:
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
324 lr = None
27447
d1b91c10ce70 verify: move err() to be a class function
Durham Goode <durham@fb.com>
parents: 27446
diff changeset
325 self.err(lr, _("in manifest but not in changeset"), f)
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
326
38397
1249475f0bd6 verify: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 38396
diff changeset
327 progress.complete()
8291
f5c1a9094e41 verify: avoid exception on missing file revlog
Henrik Stuart <hg@hstuart.dk>
parents: 8225
diff changeset
328
27644
331e5c28f5f0 verify: move filelog verification to its own function
Durham Goode <durham@fb.com>
parents: 27643
diff changeset
329 def _verifyfiles(self, filenodes, filelinkrevs):
331e5c28f5f0 verify: move filelog verification to its own function
Durham Goode <durham@fb.com>
parents: 27643
diff changeset
330 repo = self.repo
331e5c28f5f0 verify: move filelog verification to its own function
Durham Goode <durham@fb.com>
parents: 27643
diff changeset
331 ui = self.ui
331e5c28f5f0 verify: move filelog verification to its own function
Durham Goode <durham@fb.com>
parents: 27643
diff changeset
332 lrugetctx = self.lrugetctx
331e5c28f5f0 verify: move filelog verification to its own function
Durham Goode <durham@fb.com>
parents: 27643
diff changeset
333 revlogv1 = self.revlogv1
331e5c28f5f0 verify: move filelog verification to its own function
Durham Goode <durham@fb.com>
parents: 27643
diff changeset
334 havemf = self.havemf
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
335 ui.status(_("checking files\n"))
8291
f5c1a9094e41 verify: avoid exception on missing file revlog
Henrik Stuart <hg@hstuart.dk>
parents: 8225
diff changeset
336
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
337 storefiles = set()
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
338 for f, f2, size in repo.store.datafiles():
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
339 if not f:
27447
d1b91c10ce70 verify: move err() to be a class function
Durham Goode <durham@fb.com>
parents: 27446
diff changeset
340 self.err(None, _("cannot decode filename '%s'") % f2)
28007
fb92927f9775 treemanifests: fix streaming clone
Martin von Zweigbergk <martinvonz@google.com>
parents: 27964
diff changeset
341 elif (size > 0 or not revlogv1) and f.startswith('data/'):
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
342 storefiles.add(_normpath(f))
6892
dab95717058d verify: check repo.store
Adrian Buehlmann <adrian@cadifra.com>
parents: 6889
diff changeset
343
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
344 files = sorted(set(filenodes) | set(filelinkrevs))
27644
331e5c28f5f0 verify: move filelog verification to its own function
Durham Goode <durham@fb.com>
parents: 27643
diff changeset
345 revisions = 0
38397
1249475f0bd6 verify: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 38396
diff changeset
346 progress = ui.makeprogress(_('checking'), unit=_('files'),
1249475f0bd6 verify: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 38396
diff changeset
347 total=len(files))
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
348 for i, f in enumerate(files):
38397
1249475f0bd6 verify: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 38396
diff changeset
349 progress.update(i, item=f)
6892
dab95717058d verify: check repo.store
Adrian Buehlmann <adrian@cadifra.com>
parents: 6889
diff changeset
350 try:
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
351 linkrevs = filelinkrevs[f]
6892
dab95717058d verify: check repo.store
Adrian Buehlmann <adrian@cadifra.com>
parents: 6889
diff changeset
352 except KeyError:
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
353 # in manifest but not in changelog
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
354 linkrevs = []
2778
fdc232d8a193 Move repo.verify
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
355
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
356 if linkrevs:
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
357 lr = linkrevs[0]
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
358 else:
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
359 lr = None
2778
fdc232d8a193 Move repo.verify
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
360
3744
d626fc9e3985 verify: add rename link checking
Matt Mackall <mpm@selenic.com>
parents: 3473
diff changeset
361 try:
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
362 fl = repo.file(f)
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
363 except error.RevlogError as e:
27447
d1b91c10ce70 verify: move err() to be a class function
Durham Goode <durham@fb.com>
parents: 27446
diff changeset
364 self.err(lr, _("broken revlog! (%s)") % e, f)
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
365 continue
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
366
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
367 for ff in fl.files():
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
368 try:
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
369 storefiles.remove(ff)
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
370 except KeyError:
37417
76d2115cb817 verify: allow suppressing warnings about extra files
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37410
diff changeset
371 if self.warnorphanstorefiles:
76d2115cb817 verify: allow suppressing warnings about extra files
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37410
diff changeset
372 self.warn(_(" warning: revlog '%s' not in fncache!") %
76d2115cb817 verify: allow suppressing warnings about extra files
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37410
diff changeset
373 ff)
76d2115cb817 verify: allow suppressing warnings about extra files
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37410
diff changeset
374 self.fncachewarned = True
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
375
27642
f6457349985b verify: move checklog() onto class
Durham Goode <durham@fb.com>
parents: 27453
diff changeset
376 self.checklog(fl, f, lr)
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
377 seen = {}
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
378 rp = None
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
379 for i in fl:
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
380 revisions += 1
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
381 n = fl.node(i)
27643
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
382 lr = self.checkentry(fl, i, n, seen, linkrevs, f)
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
383 if f in filenodes:
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
384 if havemf and n not in filenodes[f]:
27447
d1b91c10ce70 verify: move err() to be a class function
Durham Goode <durham@fb.com>
parents: 27446
diff changeset
385 self.err(lr, _("%s not in manifests") % (short(n)), f)
6534
9b35a9f34675 verify: check copy source revlog and nodeid
Patrick Mezard <pmezard@gmail.com>
parents: 6211
diff changeset
386 else:
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
387 del filenodes[f][n]
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
388
31761
b044c339c06d verify: document corner cases
Jun Wu <quark@fb.com>
parents: 30866
diff changeset
389 # Verify contents. 4 cases to care about:
b044c339c06d verify: document corner cases
Jun Wu <quark@fb.com>
parents: 30866
diff changeset
390 #
b044c339c06d verify: document corner cases
Jun Wu <quark@fb.com>
parents: 30866
diff changeset
391 # common: the most common case
b044c339c06d verify: document corner cases
Jun Wu <quark@fb.com>
parents: 30866
diff changeset
392 # rename: with a rename
b044c339c06d verify: document corner cases
Jun Wu <quark@fb.com>
parents: 30866
diff changeset
393 # meta: file content starts with b'\1\n', the metadata
b044c339c06d verify: document corner cases
Jun Wu <quark@fb.com>
parents: 30866
diff changeset
394 # header defined in filelog.py, but without a rename
b044c339c06d verify: document corner cases
Jun Wu <quark@fb.com>
parents: 30866
diff changeset
395 # ext: content stored externally
b044c339c06d verify: document corner cases
Jun Wu <quark@fb.com>
parents: 30866
diff changeset
396 #
b044c339c06d verify: document corner cases
Jun Wu <quark@fb.com>
parents: 30866
diff changeset
397 # More formally, their differences are shown below:
b044c339c06d verify: document corner cases
Jun Wu <quark@fb.com>
parents: 30866
diff changeset
398 #
b044c339c06d verify: document corner cases
Jun Wu <quark@fb.com>
parents: 30866
diff changeset
399 # | common | rename | meta | ext
b044c339c06d verify: document corner cases
Jun Wu <quark@fb.com>
parents: 30866
diff changeset
400 # -------------------------------------------------------
b044c339c06d verify: document corner cases
Jun Wu <quark@fb.com>
parents: 30866
diff changeset
401 # flags() | 0 | 0 | 0 | not 0
b044c339c06d verify: document corner cases
Jun Wu <quark@fb.com>
parents: 30866
diff changeset
402 # renamed() | False | True | False | ?
b044c339c06d verify: document corner cases
Jun Wu <quark@fb.com>
parents: 30866
diff changeset
403 # rawtext[0:2]=='\1\n'| False | True | True | ?
b044c339c06d verify: document corner cases
Jun Wu <quark@fb.com>
parents: 30866
diff changeset
404 #
b044c339c06d verify: document corner cases
Jun Wu <quark@fb.com>
parents: 30866
diff changeset
405 # "rawtext" means the raw text stored in revlog data, which
b044c339c06d verify: document corner cases
Jun Wu <quark@fb.com>
parents: 30866
diff changeset
406 # could be retrieved by "revision(rev, raw=True)". "text"
b044c339c06d verify: document corner cases
Jun Wu <quark@fb.com>
parents: 30866
diff changeset
407 # mentioned below is "revision(rev, raw=False)".
b044c339c06d verify: document corner cases
Jun Wu <quark@fb.com>
parents: 30866
diff changeset
408 #
b044c339c06d verify: document corner cases
Jun Wu <quark@fb.com>
parents: 30866
diff changeset
409 # There are 3 different lengths stored physically:
b044c339c06d verify: document corner cases
Jun Wu <quark@fb.com>
parents: 30866
diff changeset
410 # 1. L1: rawsize, stored in revlog index
b044c339c06d verify: document corner cases
Jun Wu <quark@fb.com>
parents: 30866
diff changeset
411 # 2. L2: len(rawtext), stored in revlog data
b044c339c06d verify: document corner cases
Jun Wu <quark@fb.com>
parents: 30866
diff changeset
412 # 3. L3: len(text), stored in revlog data if flags==0, or
b044c339c06d verify: document corner cases
Jun Wu <quark@fb.com>
parents: 30866
diff changeset
413 # possibly somewhere else if flags!=0
b044c339c06d verify: document corner cases
Jun Wu <quark@fb.com>
parents: 30866
diff changeset
414 #
b044c339c06d verify: document corner cases
Jun Wu <quark@fb.com>
parents: 30866
diff changeset
415 # L1 should be equal to L2. L3 could be different from them.
b044c339c06d verify: document corner cases
Jun Wu <quark@fb.com>
parents: 30866
diff changeset
416 # "text" may or may not affect commit hash depending on flag
b044c339c06d verify: document corner cases
Jun Wu <quark@fb.com>
parents: 30866
diff changeset
417 # processors (see revlog.addflagprocessor).
b044c339c06d verify: document corner cases
Jun Wu <quark@fb.com>
parents: 30866
diff changeset
418 #
b044c339c06d verify: document corner cases
Jun Wu <quark@fb.com>
parents: 30866
diff changeset
419 # | common | rename | meta | ext
b044c339c06d verify: document corner cases
Jun Wu <quark@fb.com>
parents: 30866
diff changeset
420 # -------------------------------------------------
b044c339c06d verify: document corner cases
Jun Wu <quark@fb.com>
parents: 30866
diff changeset
421 # rawsize() | L1 | L1 | L1 | L1
b044c339c06d verify: document corner cases
Jun Wu <quark@fb.com>
parents: 30866
diff changeset
422 # size() | L1 | L2-LM | L1(*) | L1 (?)
b044c339c06d verify: document corner cases
Jun Wu <quark@fb.com>
parents: 30866
diff changeset
423 # len(rawtext) | L2 | L2 | L2 | L2
b044c339c06d verify: document corner cases
Jun Wu <quark@fb.com>
parents: 30866
diff changeset
424 # len(text) | L2 | L2 | L2 | L3
b044c339c06d verify: document corner cases
Jun Wu <quark@fb.com>
parents: 30866
diff changeset
425 # len(read()) | L2 | L2-LM | L2-LM | L3 (?)
b044c339c06d verify: document corner cases
Jun Wu <quark@fb.com>
parents: 30866
diff changeset
426 #
b044c339c06d verify: document corner cases
Jun Wu <quark@fb.com>
parents: 30866
diff changeset
427 # LM: length of metadata, depending on rawtext
b044c339c06d verify: document corner cases
Jun Wu <quark@fb.com>
parents: 30866
diff changeset
428 # (*): not ideal, see comment in filelog.size
b044c339c06d verify: document corner cases
Jun Wu <quark@fb.com>
parents: 30866
diff changeset
429 # (?): could be "- len(meta)" if the resolved content has
b044c339c06d verify: document corner cases
Jun Wu <quark@fb.com>
parents: 30866
diff changeset
430 # rename metadata
b044c339c06d verify: document corner cases
Jun Wu <quark@fb.com>
parents: 30866
diff changeset
431 #
b044c339c06d verify: document corner cases
Jun Wu <quark@fb.com>
parents: 30866
diff changeset
432 # Checks needed to be done:
b044c339c06d verify: document corner cases
Jun Wu <quark@fb.com>
parents: 30866
diff changeset
433 # 1. length check: L1 == L2, in all cases.
b044c339c06d verify: document corner cases
Jun Wu <quark@fb.com>
parents: 30866
diff changeset
434 # 2. hash check: depending on flag processor, we may need to
b044c339c06d verify: document corner cases
Jun Wu <quark@fb.com>
parents: 30866
diff changeset
435 # use either "text" (external), or "rawtext" (in revlog).
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
436 try:
32288
a2ab9ebcd85b verify: add a config option to skip certain flag processors
Jun Wu <quark@fb.com>
parents: 32250
diff changeset
437 skipflags = self.skipflags
a2ab9ebcd85b verify: add a config option to skip certain flag processors
Jun Wu <quark@fb.com>
parents: 32250
diff changeset
438 if skipflags:
a2ab9ebcd85b verify: add a config option to skip certain flag processors
Jun Wu <quark@fb.com>
parents: 32250
diff changeset
439 skipflags &= fl.flags(i)
a2ab9ebcd85b verify: add a config option to skip certain flag processors
Jun Wu <quark@fb.com>
parents: 32250
diff changeset
440 if not skipflags:
a2ab9ebcd85b verify: add a config option to skip certain flag processors
Jun Wu <quark@fb.com>
parents: 32250
diff changeset
441 fl.read(n) # side effect: read content and do checkhash
a2ab9ebcd85b verify: add a config option to skip certain flag processors
Jun Wu <quark@fb.com>
parents: 32250
diff changeset
442 rp = fl.renamed(n)
32250
8a137ef6e5da verify: always check rawsize
Jun Wu <quark@fb.com>
parents: 31762
diff changeset
443 # the "L1 == L2" check
8a137ef6e5da verify: always check rawsize
Jun Wu <quark@fb.com>
parents: 31762
diff changeset
444 l1 = fl.rawsize(i)
8a137ef6e5da verify: always check rawsize
Jun Wu <quark@fb.com>
parents: 31762
diff changeset
445 l2 = len(fl.revision(n, raw=True))
8a137ef6e5da verify: always check rawsize
Jun Wu <quark@fb.com>
parents: 31762
diff changeset
446 if l1 != l2:
8a137ef6e5da verify: always check rawsize
Jun Wu <quark@fb.com>
parents: 31762
diff changeset
447 self.err(lr, _("unpacked size is %s, %s expected") %
8a137ef6e5da verify: always check rawsize
Jun Wu <quark@fb.com>
parents: 31762
diff changeset
448 (l2, l1), f)
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
449 except error.CensoredNodeError:
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
450 # experimental config: censor.policy
33499
0407a51b9d8c codemod: register core configitems using a script
Jun Wu <quark@fb.com>
parents: 32288
diff changeset
451 if ui.config("censor", "policy") == "abort":
27447
d1b91c10ce70 verify: move err() to be a class function
Durham Goode <durham@fb.com>
parents: 27446
diff changeset
452 self.err(lr, _("censored file data"), f)
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
453 except Exception as inst:
27448
f4f2179077cb verify: move exc() function onto class
Durham Goode <durham@fb.com>
parents: 27447
diff changeset
454 self.exc(lr, _("unpacking %s") % short(n), inst, f)
3744
d626fc9e3985 verify: add rename link checking
Matt Mackall <mpm@selenic.com>
parents: 3473
diff changeset
455
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
456 # check renames
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
457 try:
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
458 if rp:
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
459 if lr is not None and ui.verbose:
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
460 ctx = lrugetctx(lr)
36339
a4d41ba4ad23 verify: don't reimplement any()
Martin von Zweigbergk <martinvonz@google.com>
parents: 36185
diff changeset
461 if not any(rp[0] in pctx for pctx in ctx.parents()):
27446
6b2c1a1871a6 verify: move warn() to a class level function
Durham Goode <durham@fb.com>
parents: 27445
diff changeset
462 self.warn(_("warning: copy source of '%s' not"
6b2c1a1871a6 verify: move warn() to a class level function
Durham Goode <durham@fb.com>
parents: 27445
diff changeset
463 " in parents of %s") % (f, ctx))
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
464 fl2 = repo.file(rp[0])
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
465 if not len(fl2):
27447
d1b91c10ce70 verify: move err() to be a class function
Durham Goode <durham@fb.com>
parents: 27446
diff changeset
466 self.err(lr, _("empty or missing copy source "
d1b91c10ce70 verify: move err() to be a class function
Durham Goode <durham@fb.com>
parents: 27446
diff changeset
467 "revlog %s:%s") % (rp[0], short(rp[1])), f)
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
468 elif rp[1] == nullid:
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
469 ui.note(_("warning: %s@%s: copy source"
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
470 " revision is nullid %s:%s\n")
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
471 % (f, lr, rp[0], short(rp[1])))
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
472 else:
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
473 fl2.rev(rp[1])
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
474 except Exception as inst:
27448
f4f2179077cb verify: move exc() function onto class
Durham Goode <durham@fb.com>
parents: 27447
diff changeset
475 self.exc(lr, _("checking rename of %s") % short(n), inst, f)
6892
dab95717058d verify: check repo.store
Adrian Buehlmann <adrian@cadifra.com>
parents: 6889
diff changeset
476
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
477 # cross-check
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
478 if f in filenodes:
30393
b667b78099eb verify: avoid shadowing two variables with a list comprehension
Augie Fackler <augie@google.com>
parents: 30375
diff changeset
479 fns = [(v, k) for k, v in filenodes[f].iteritems()]
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
480 for lr, node in sorted(fns):
28114
2a03a365f645 verify: use similar language for missing manifest and file revisions
Martin von Zweigbergk <martinvonz@google.com>
parents: 28113
diff changeset
481 self.err(lr, _("manifest refers to unknown revision %s") %
2a03a365f645 verify: use similar language for missing manifest and file revisions
Martin von Zweigbergk <martinvonz@google.com>
parents: 28113
diff changeset
482 short(node), f)
38397
1249475f0bd6 verify: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 38396
diff changeset
483 progress.complete()
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
484
37417
76d2115cb817 verify: allow suppressing warnings about extra files
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37410
diff changeset
485 if self.warnorphanstorefiles:
76d2115cb817 verify: allow suppressing warnings about extra files
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37410
diff changeset
486 for f in sorted(storefiles):
76d2115cb817 verify: allow suppressing warnings about extra files
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37410
diff changeset
487 self.warn(_("warning: orphan data file '%s'") % f)
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
488
27644
331e5c28f5f0 verify: move filelog verification to its own function
Durham Goode <durham@fb.com>
parents: 27643
diff changeset
489 return len(files), revisions