annotate mercurial/verify.py @ 43297:8a2925265402

sidedatacopies: fast path data fetching if revision has no sidedata When using the side data mode, we know their won't be any copy information sidedata. Skipping revision restoration give an important speed boost. In the future, there will be other user of sidedata, reducing the efficiency of this. We should consider adding a dedicated flag in revlog V2 to preserve this optimisation. The current situation is good enough for now. revision: large amount; added files: large amount; rename small amount; c3b14617fbd7 9ba6ab77fd29 before: ! wall 2.401569 comb 2.400000 user 2.390000 sys 0.010000 (median of 10) after: ! wall 1.429294 comb 1.430000 user 1.410000 sys 0.020000 (median of 10) revision: large amount; added files: small amount; rename small amount; c3b14617fbd7 f650a9b140d2 before: ! wall 3.519140 comb 3.520000 user 3.470000 sys 0.050000 (median of 10) after: ! wall 1.963332 comb 1.960000 user 1.960000 sys 0.000000 (median of 10) revision: large amount; added files: large amount; rename large amount; 08ea3258278e d9fa043f30c0 before: ! wall 0.593880 comb 0.600000 user 0.590000 sys 0.010000 (median of 15) after: ! wall 0.251679 comb 0.250000 user 0.250000 sys 0.000000 (median of 38) revision: small amount; added files: large amount; rename large amount; df6f7a526b60 a83dc6a2d56f before: ! wall 0.013414 comb 0.020000 user 0.020000 sys 0.000000 (median of 220) after: ! wall 0.013222 comb 0.020000 user 0.020000 sys 0.000000 (median of 223) revision: small amount; added files: large amount; rename small amount; 4aa4e1f8e19a 169138063d63 before: ! wall 0.002711 comb 0.000000 user 0.000000 sys 0.000000 (median of 1000) after: ! wall 0.001631 comb 0.000000 user 0.000000 sys 0.000000 (median of 1000) revision: small amount; added files: small amount; rename small amount; 4bc173b045a6 964879152e2e before: ! wall 0.000077 comb 0.000000 user 0.000000 sys 0.000000 (median of 12208) after: ! wall 0.000078 comb 0.000000 user 0.000000 sys 0.000000 (median of 12012) revision: medium amount; added files: large amount; rename medium amount; c95f1ced15f2 2c68e87c3efe before: ! wall 0.410067 comb 0.410000 user 0.410000 sys 0.000000 (median of 23) after: ! wall 0.207786 comb 0.200000 user 0.200000 sys 0.000000 (median of 46) revision: medium amount; added files: medium amount; rename small amount; d343da0c55a8 d7746d32bf9d before: ! wall 0.097004 comb 0.090000 user 0.090000 sys 0.000000 (median of 100) after: ! wall 0.038495 comb 0.030000 user 0.030000 sys 0.000000 (median of 100) Differential Revision: https://phab.mercurial-scm.org/D7074
author Pierre-Yves David <pierre-yves.david@octobus.net>
date Wed, 02 Oct 2019 18:16:02 -0400
parents d783f945a701
children a447efd991b9
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
2778
fdc232d8a193 Move repo.verify
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
1 # verify.py - repository integrity checking for Mercurial
fdc232d8a193 Move repo.verify
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
2 #
4635
63b9d2deed48 Updated copyright notices and add "and others" to "hg version"
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4395
diff changeset
3 # Copyright 2006, 2007 Matt Mackall <mpm@selenic.com>
2778
fdc232d8a193 Move repo.verify
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
4 #
8225
46293a0c7e9f updated license to be explicit about GPL version 2
Martin Geisler <mg@lazybytes.net>
parents: 8209
diff changeset
5 # This software may be used and distributed according to the terms of the
10263
25e572394f5c Update license to GPLv2+
Matt Mackall <mpm@selenic.com>
parents: 9690
diff changeset
6 # GNU General Public License version 2 or any later version.
2778
fdc232d8a193 Move repo.verify
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
7
25991
d21d1774c73b verify: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25846
diff changeset
8 from __future__ import absolute_import
d21d1774c73b verify: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25846
diff changeset
9
17860
a45b33f12627 verify: fix all doubled-slash sites (issue3665)
Bryan O'Sullivan <bryano@fb.com>
parents: 17851
diff changeset
10 import os
25991
d21d1774c73b verify: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25846
diff changeset
11
d21d1774c73b verify: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25846
diff changeset
12 from .i18n import _
d21d1774c73b verify: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25846
diff changeset
13 from .node import (
d21d1774c73b verify: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25846
diff changeset
14 nullid,
d21d1774c73b verify: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25846
diff changeset
15 short,
d21d1774c73b verify: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25846
diff changeset
16 )
d21d1774c73b verify: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25846
diff changeset
17
d21d1774c73b verify: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25846
diff changeset
18 from . import (
d21d1774c73b verify: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25846
diff changeset
19 error,
35585
35fb3367f72d py3: use pycompat.bytestr() instead of str()
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33499
diff changeset
20 pycompat,
25991
d21d1774c73b verify: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25846
diff changeset
21 revlog,
d21d1774c73b verify: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25846
diff changeset
22 util,
d21d1774c73b verify: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25846
diff changeset
23 )
2778
fdc232d8a193 Move repo.verify
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
24
42155
57539e5ea2e0 verify: introduce a notion of "level"
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41877
diff changeset
25 VERIFY_DEFAULT = 0
42156
496ac8a02380 verify: introduce an experimental --full flag
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42155
diff changeset
26 VERIFY_FULL = 1
42155
57539e5ea2e0 verify: introduce a notion of "level"
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41877
diff changeset
27
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
28
42155
57539e5ea2e0 verify: introduce a notion of "level"
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41877
diff changeset
29 def verify(repo, level=None):
27849
900d36a3e4dd with: use context manager in verify
Bryan O'Sullivan <bryano@fb.com>
parents: 27695
diff changeset
30 with repo.lock():
42155
57539e5ea2e0 verify: introduce a notion of "level"
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41877
diff changeset
31 v = verifier(repo, level)
57539e5ea2e0 verify: introduce a notion of "level"
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41877
diff changeset
32 return v.verify()
4915
97b734fb9c6f Use try/finally pattern to cleanup locks and transactions
Matt Mackall <mpm@selenic.com>
parents: 4635
diff changeset
33
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
34
17860
a45b33f12627 verify: fix all doubled-slash sites (issue3665)
Bryan O'Sullivan <bryano@fb.com>
parents: 17851
diff changeset
35 def _normpath(f):
a45b33f12627 verify: fix all doubled-slash sites (issue3665)
Bryan O'Sullivan <bryano@fb.com>
parents: 17851
diff changeset
36 # under hg < 2.4, convert didn't sanitize paths properly, so a
a45b33f12627 verify: fix all doubled-slash sites (issue3665)
Bryan O'Sullivan <bryano@fb.com>
parents: 17851
diff changeset
37 # converted repo may contain repeated slashes
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
38 while b'//' in f:
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
39 f = f.replace(b'//', b'/')
17860
a45b33f12627 verify: fix all doubled-slash sites (issue3665)
Bryan O'Sullivan <bryano@fb.com>
parents: 17851
diff changeset
40 return f
a45b33f12627 verify: fix all doubled-slash sites (issue3665)
Bryan O'Sullivan <bryano@fb.com>
parents: 17851
diff changeset
41
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
42
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
43 class verifier(object):
42155
57539e5ea2e0 verify: introduce a notion of "level"
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41877
diff changeset
44 def __init__(self, repo, level=None):
27444
6647401858ab verify: move widely used variables into class members
Durham Goode <durham@fb.com>
parents: 27443
diff changeset
45 self.repo = repo.unfiltered()
6647401858ab verify: move widely used variables into class members
Durham Goode <durham@fb.com>
parents: 27443
diff changeset
46 self.ui = repo.ui
39938
fec944719324 narrow: move support for `hg verify` into core
Martin von Zweigbergk <martinvonz@google.com>
parents: 39872
diff changeset
47 self.match = repo.narrowmatch()
42155
57539e5ea2e0 verify: introduce a notion of "level"
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41877
diff changeset
48 if level is None:
57539e5ea2e0 verify: introduce a notion of "level"
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41877
diff changeset
49 level = VERIFY_DEFAULT
57539e5ea2e0 verify: introduce a notion of "level"
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41877
diff changeset
50 self._level = level
27444
6647401858ab verify: move widely used variables into class members
Durham Goode <durham@fb.com>
parents: 27443
diff changeset
51 self.badrevs = set()
27453
8462d7f2c4fe verify: clean up weird error/warning lists
Matt Mackall <mpm@selenic.com>
parents: 27450
diff changeset
52 self.errors = 0
8462d7f2c4fe verify: clean up weird error/warning lists
Matt Mackall <mpm@selenic.com>
parents: 27450
diff changeset
53 self.warnings = 0
27444
6647401858ab verify: move widely used variables into class members
Durham Goode <durham@fb.com>
parents: 27443
diff changeset
54 self.havecl = len(repo.changelog) > 0
39244
73cf21b2e8a6 manifest: add getstorage() to manifestlog and use it globally
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38397
diff changeset
55 self.havemf = len(repo.manifestlog.getstorage(b'')) > 0
27444
6647401858ab verify: move widely used variables into class members
Durham Goode <durham@fb.com>
parents: 27443
diff changeset
56 self.revlogv1 = repo.changelog.version != revlog.REVLOGV0
37302
00f18dd1d3d6 verify: remove dependence on repo.changectx()
Martin von Zweigbergk <martinvonz@google.com>
parents: 36577
diff changeset
57 self.lrugetctx = util.lrucachefunc(repo.__getitem__)
27444
6647401858ab verify: move widely used variables into class members
Durham Goode <durham@fb.com>
parents: 27443
diff changeset
58 self.refersmf = False
27445
cc178057ab49 verify: move fncachewarned up to a class variable
Durham Goode <durham@fb.com>
parents: 27444
diff changeset
59 self.fncachewarned = False
32288
a2ab9ebcd85b verify: add a config option to skip certain flag processors
Jun Wu <quark@fb.com>
parents: 32250
diff changeset
60 # developer config: verify.skipflags
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
61 self.skipflags = repo.ui.configint(b'verify', b'skipflags')
37417
76d2115cb817 verify: allow suppressing warnings about extra files
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37410
diff changeset
62 self.warnorphanstorefiles = True
27444
6647401858ab verify: move widely used variables into class members
Durham Goode <durham@fb.com>
parents: 27443
diff changeset
63
41862
c66037fb1bc5 verify: make the `warn` method private
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41861
diff changeset
64 def _warn(self, msg):
41861
a58748300e61 verify: document the `warn` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 40616
diff changeset
65 """record a "warning" level issue"""
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
66 self.ui.warn(msg + b"\n")
27453
8462d7f2c4fe verify: clean up weird error/warning lists
Matt Mackall <mpm@selenic.com>
parents: 27450
diff changeset
67 self.warnings += 1
27446
6b2c1a1871a6 verify: move warn() to a class level function
Durham Goode <durham@fb.com>
parents: 27445
diff changeset
68
41864
7eaf4b1ac2a3 verify: make `err` a private method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41863
diff changeset
69 def _err(self, linkrev, msg, filename=None):
41863
9534f3cb6b6c verify: document the `err` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41862
diff changeset
70 """record a "error" level issue"""
27447
d1b91c10ce70 verify: move err() to be a class function
Durham Goode <durham@fb.com>
parents: 27446
diff changeset
71 if linkrev is not None:
d1b91c10ce70 verify: move err() to be a class function
Durham Goode <durham@fb.com>
parents: 27446
diff changeset
72 self.badrevs.add(linkrev)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
73 linkrev = b"%d" % linkrev
27447
d1b91c10ce70 verify: move err() to be a class function
Durham Goode <durham@fb.com>
parents: 27446
diff changeset
74 else:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
75 linkrev = b'?'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
76 msg = b"%s: %s" % (linkrev, msg)
27447
d1b91c10ce70 verify: move err() to be a class function
Durham Goode <durham@fb.com>
parents: 27446
diff changeset
77 if filename:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
78 msg = b"%s@%s" % (filename, msg)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
79 self.ui.warn(b" " + msg + b"\n")
27453
8462d7f2c4fe verify: clean up weird error/warning lists
Matt Mackall <mpm@selenic.com>
parents: 27450
diff changeset
80 self.errors += 1
27447
d1b91c10ce70 verify: move err() to be a class function
Durham Goode <durham@fb.com>
parents: 27446
diff changeset
81
41866
cfe08588d711 verify: make the `exc` method private
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41865
diff changeset
82 def _exc(self, linkrev, msg, inst, filename=None):
41865
5df8475c5343 verify: document the `exc` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41864
diff changeset
83 """record exception raised during the verify process"""
36577
d85ef895d5f6 verify: fix exception formatting bug in Python 3
Augie Fackler <augie@google.com>
parents: 36339
diff changeset
84 fmsg = pycompat.bytestr(inst)
d85ef895d5f6 verify: fix exception formatting bug in Python 3
Augie Fackler <augie@google.com>
parents: 36339
diff changeset
85 if not fmsg:
d85ef895d5f6 verify: fix exception formatting bug in Python 3
Augie Fackler <augie@google.com>
parents: 36339
diff changeset
86 fmsg = pycompat.byterepr(inst)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
87 self._err(linkrev, b"%s: %s" % (msg, fmsg), filename)
27448
f4f2179077cb verify: move exc() function onto class
Durham Goode <durham@fb.com>
parents: 27447
diff changeset
88
41874
1f412223f5bb verify: rename the `checklog` to `_checkrevlog`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41873
diff changeset
89 def _checkrevlog(self, obj, name, linkrev):
41873
08d977451f26 verify: document the `checklog` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41871
diff changeset
90 """verify high level property of a revlog
08d977451f26 verify: document the `checklog` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41871
diff changeset
91
08d977451f26 verify: document the `checklog` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41871
diff changeset
92 - revlog is present,
08d977451f26 verify: document the `checklog` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41871
diff changeset
93 - revlog is non-empty,
08d977451f26 verify: document the `checklog` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41871
diff changeset
94 - sizes (index and data) are correct,
08d977451f26 verify: document the `checklog` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41871
diff changeset
95 - revlog's format version is correct.
08d977451f26 verify: document the `checklog` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41871
diff changeset
96 """
27642
f6457349985b verify: move checklog() onto class
Durham Goode <durham@fb.com>
parents: 27453
diff changeset
97 if not len(obj) and (self.havecl or self.havemf):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
98 self._err(linkrev, _(b"empty or missing %s") % name)
27642
f6457349985b verify: move checklog() onto class
Durham Goode <durham@fb.com>
parents: 27453
diff changeset
99 return
f6457349985b verify: move checklog() onto class
Durham Goode <durham@fb.com>
parents: 27453
diff changeset
100
f6457349985b verify: move checklog() onto class
Durham Goode <durham@fb.com>
parents: 27453
diff changeset
101 d = obj.checksize()
f6457349985b verify: move checklog() onto class
Durham Goode <durham@fb.com>
parents: 27453
diff changeset
102 if d[0]:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
103 self._err(None, _(b"data length off by %d bytes") % d[0], name)
27642
f6457349985b verify: move checklog() onto class
Durham Goode <durham@fb.com>
parents: 27453
diff changeset
104 if d[1]:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
105 self._err(None, _(b"index contains %d extra bytes") % d[1], name)
27642
f6457349985b verify: move checklog() onto class
Durham Goode <durham@fb.com>
parents: 27453
diff changeset
106
f6457349985b verify: move checklog() onto class
Durham Goode <durham@fb.com>
parents: 27453
diff changeset
107 if obj.version != revlog.REVLOGV0:
f6457349985b verify: move checklog() onto class
Durham Goode <durham@fb.com>
parents: 27453
diff changeset
108 if not self.revlogv1:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
109 self._warn(_(b"warning: `%s' uses revlog format 1") % name)
27642
f6457349985b verify: move checklog() onto class
Durham Goode <durham@fb.com>
parents: 27453
diff changeset
110 elif self.revlogv1:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
111 self._warn(_(b"warning: `%s' uses revlog format 0") % name)
27642
f6457349985b verify: move checklog() onto class
Durham Goode <durham@fb.com>
parents: 27453
diff changeset
112
41871
9e737ca539f6 verify: make `checkentry` a private method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41870
diff changeset
113 def _checkentry(self, obj, i, node, seen, linkrevs, f):
41870
00c9fde75c1a verify: document the `checkentry` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41869
diff changeset
114 """verify a single revlog entry
00c9fde75c1a verify: document the `checkentry` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41869
diff changeset
115
00c9fde75c1a verify: document the `checkentry` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41869
diff changeset
116 arguments are:
00c9fde75c1a verify: document the `checkentry` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41869
diff changeset
117 - obj: the source revlog
00c9fde75c1a verify: document the `checkentry` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41869
diff changeset
118 - i: the revision number
00c9fde75c1a verify: document the `checkentry` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41869
diff changeset
119 - node: the revision node id
00c9fde75c1a verify: document the `checkentry` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41869
diff changeset
120 - seen: nodes previously seen for this revlog
00c9fde75c1a verify: document the `checkentry` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41869
diff changeset
121 - linkrevs: [changelog-revisions] introducing "node"
00c9fde75c1a verify: document the `checkentry` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41869
diff changeset
122 - f: string label ("changelog", "manifest", or filename)
00c9fde75c1a verify: document the `checkentry` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41869
diff changeset
123
00c9fde75c1a verify: document the `checkentry` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41869
diff changeset
124 Performs the following checks:
00c9fde75c1a verify: document the `checkentry` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41869
diff changeset
125 - linkrev points to an existing changelog revision,
00c9fde75c1a verify: document the `checkentry` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41869
diff changeset
126 - linkrev points to a changelog revision that introduces this revision,
00c9fde75c1a verify: document the `checkentry` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41869
diff changeset
127 - linkrev points to the lowest of these changesets,
00c9fde75c1a verify: document the `checkentry` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41869
diff changeset
128 - both parents exist in the revlog,
00c9fde75c1a verify: document the `checkentry` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41869
diff changeset
129 - the revision is not duplicated.
00c9fde75c1a verify: document the `checkentry` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41869
diff changeset
130
00c9fde75c1a verify: document the `checkentry` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41869
diff changeset
131 Return the linkrev of the revision (or None for changelog's revisions).
00c9fde75c1a verify: document the `checkentry` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41869
diff changeset
132 """
27643
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
133 lr = obj.linkrev(obj.rev(node))
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
134 if lr < 0 or (self.havecl and lr not in linkrevs):
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
135 if lr < 0 or lr >= len(self.repo.changelog):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
136 msg = _(b"rev %d points to nonexistent changeset %d")
27643
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
137 else:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
138 msg = _(b"rev %d points to unexpected changeset %d")
41864
7eaf4b1ac2a3 verify: make `err` a private method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41863
diff changeset
139 self._err(None, msg % (i, lr), f)
27643
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
140 if linkrevs:
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
141 if f and len(linkrevs) > 1:
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
142 try:
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
143 # attempt to filter down to real linkrevs
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
144 linkrevs = [
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
145 l
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
146 for l in linkrevs
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
147 if self.lrugetctx(l)[f].filenode() == node
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
148 ]
27643
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
149 except Exception:
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
150 pass
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
151 self._warn(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
152 _(b" (expected %s)")
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
153 % b" ".join(map(pycompat.bytestr, linkrevs))
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
154 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
155 lr = None # can't be trusted
27643
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
156
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
157 try:
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
158 p1, p2 = obj.parents(node)
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
159 if p1 not in seen and p1 != nullid:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
160 self._err(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
161 lr,
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
162 _(b"unknown parent 1 %s of %s") % (short(p1), short(node)),
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
163 f,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
164 )
27643
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
165 if p2 not in seen and p2 != nullid:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
166 self._err(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
167 lr,
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
168 _(b"unknown parent 2 %s of %s") % (short(p2), short(node)),
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
169 f,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
170 )
27643
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
171 except Exception as inst:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
172 self._exc(lr, _(b"checking parents of %s") % short(node), inst, f)
27643
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
173
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
174 if node in seen:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
175 self._err(lr, _(b"duplicate revision %d (%d)") % (i, seen[node]), f)
27643
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
176 seen[node] = i
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
177 return lr
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
178
27444
6647401858ab verify: move widely used variables into class members
Durham Goode <durham@fb.com>
parents: 27443
diff changeset
179 def verify(self):
41867
e8c4a9f5b986 verify: minimal documentation for `verifier.verify`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41866
diff changeset
180 """verify the content of the Mercurial repository
e8c4a9f5b986 verify: minimal documentation for `verifier.verify`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41866
diff changeset
181
e8c4a9f5b986 verify: minimal documentation for `verifier.verify`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41866
diff changeset
182 This method run all verifications, displaying issues as they are found.
e8c4a9f5b986 verify: minimal documentation for `verifier.verify`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41866
diff changeset
183
41868
567892b4306c verify: explicitly return 0 if no error are encountered
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41867
diff changeset
184 return 1 if any error have been encountered, 0 otherwise."""
41869
4da2261e949b verify: add some inline documentation to the top level `verify` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41868
diff changeset
185 # initial validation and generic report
27444
6647401858ab verify: move widely used variables into class members
Durham Goode <durham@fb.com>
parents: 27443
diff changeset
186 repo = self.repo
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
187 ui = repo.ui
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
188 if not repo.url().startswith(b'file:'):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
189 raise error.Abort(_(b"cannot verify bundle or remote repos"))
6752
e79a8f36c2a5 verify: lots of refactoring
Matt Mackall <mpm@selenic.com>
parents: 6751
diff changeset
190
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
191 if os.path.exists(repo.sjoin(b"journal")):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
192 ui.warn(_(b"abandoned transaction found - run hg recover\n"))
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
193
27648
e72e669dd51f verify: get rid of some unnecessary local variables
Durham Goode <durham@fb.com>
parents: 27647
diff changeset
194 if ui.verbose or not self.revlogv1:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
195 ui.status(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
196 _(b"repository uses revlog format %d\n")
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
197 % (self.revlogv1 and 1 or 0)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
198 )
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
199
41869
4da2261e949b verify: add some inline documentation to the top level `verify` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41868
diff changeset
200 # data verification
27695
fb0cc863d172 verify: replace "output parameters" by return values
Martin von Zweigbergk <martinvonz@google.com>
parents: 27648
diff changeset
201 mflinkrevs, filelinkrevs = self._verifychangelog()
fb0cc863d172 verify: replace "output parameters" by return values
Martin von Zweigbergk <martinvonz@google.com>
parents: 27648
diff changeset
202 filenodes = self._verifymanifest(mflinkrevs)
28111
06205989264b verify: move cross-checking of changeset/manifest out of _crosscheckfiles()
Martin von Zweigbergk <martinvonz@google.com>
parents: 28007
diff changeset
203 del mflinkrevs
06205989264b verify: move cross-checking of changeset/manifest out of _crosscheckfiles()
Martin von Zweigbergk <martinvonz@google.com>
parents: 28007
diff changeset
204 self._crosscheckfiles(filelinkrevs, filenodes)
27647
2c2858f3c1bb verify: move changelog verificaiton to its own function
Durham Goode <durham@fb.com>
parents: 27646
diff changeset
205 totalfiles, filerevisions = self._verifyfiles(filenodes, filelinkrevs)
2c2858f3c1bb verify: move changelog verificaiton to its own function
Durham Goode <durham@fb.com>
parents: 27646
diff changeset
206
41869
4da2261e949b verify: add some inline documentation to the top level `verify` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41868
diff changeset
207 # final report
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
208 ui.status(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
209 _(b"checked %d changesets with %d changes to %d files\n")
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
210 % (len(repo.changelog), filerevisions, totalfiles)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
211 )
27647
2c2858f3c1bb verify: move changelog verificaiton to its own function
Durham Goode <durham@fb.com>
parents: 27646
diff changeset
212 if self.warnings:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
213 ui.warn(_(b"%d warnings encountered!\n") % self.warnings)
27647
2c2858f3c1bb verify: move changelog verificaiton to its own function
Durham Goode <durham@fb.com>
parents: 27646
diff changeset
214 if self.fncachewarned:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
215 ui.warn(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
216 _(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
217 b'hint: run "hg debugrebuildfncache" to recover from '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
218 b'corrupt fncache\n'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
219 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
220 )
27647
2c2858f3c1bb verify: move changelog verificaiton to its own function
Durham Goode <durham@fb.com>
parents: 27646
diff changeset
221 if self.errors:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
222 ui.warn(_(b"%d integrity errors encountered!\n") % self.errors)
27648
e72e669dd51f verify: get rid of some unnecessary local variables
Durham Goode <durham@fb.com>
parents: 27647
diff changeset
223 if self.badrevs:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
224 ui.warn(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
225 _(b"(first damaged changeset appears to be %d)\n")
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
226 % min(self.badrevs)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
227 )
27647
2c2858f3c1bb verify: move changelog verificaiton to its own function
Durham Goode <durham@fb.com>
parents: 27646
diff changeset
228 return 1
41868
567892b4306c verify: explicitly return 0 if no error are encountered
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41867
diff changeset
229 return 0
27647
2c2858f3c1bb verify: move changelog verificaiton to its own function
Durham Goode <durham@fb.com>
parents: 27646
diff changeset
230
27695
fb0cc863d172 verify: replace "output parameters" by return values
Martin von Zweigbergk <martinvonz@google.com>
parents: 27648
diff changeset
231 def _verifychangelog(self):
41875
2f1f475e9646 verify: document `_verifychangelog`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41874
diff changeset
232 """verify the changelog of a repository
2f1f475e9646 verify: document `_verifychangelog`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41874
diff changeset
233
2f1f475e9646 verify: document `_verifychangelog`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41874
diff changeset
234 The following checks are performed:
2f1f475e9646 verify: document `_verifychangelog`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41874
diff changeset
235 - all of `_checkrevlog` checks,
2f1f475e9646 verify: document `_verifychangelog`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41874
diff changeset
236 - all of `_checkentry` checks (for each revisions),
2f1f475e9646 verify: document `_verifychangelog`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41874
diff changeset
237 - each revision can be read.
2f1f475e9646 verify: document `_verifychangelog`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41874
diff changeset
238
2f1f475e9646 verify: document `_verifychangelog`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41874
diff changeset
239 The function returns some of the data observed in the changesets as a
2f1f475e9646 verify: document `_verifychangelog`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41874
diff changeset
240 (mflinkrevs, filelinkrevs) tuples:
2f1f475e9646 verify: document `_verifychangelog`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41874
diff changeset
241 - mflinkrevs: is a { manifest-node -> [changelog-rev] } mapping
2f1f475e9646 verify: document `_verifychangelog`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41874
diff changeset
242 - filelinkrevs: is a { file-path -> [changelog-rev] } mapping
2f1f475e9646 verify: document `_verifychangelog`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41874
diff changeset
243
2f1f475e9646 verify: document `_verifychangelog`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41874
diff changeset
244 If a matcher was specified, filelinkrevs will only contains matched
2f1f475e9646 verify: document `_verifychangelog`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41874
diff changeset
245 files.
2f1f475e9646 verify: document `_verifychangelog`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41874
diff changeset
246 """
27647
2c2858f3c1bb verify: move changelog verificaiton to its own function
Durham Goode <durham@fb.com>
parents: 27646
diff changeset
247 ui = self.ui
2c2858f3c1bb verify: move changelog verificaiton to its own function
Durham Goode <durham@fb.com>
parents: 27646
diff changeset
248 repo = self.repo
30866
5249b6470de9 verify: replace _validpath() by matcher
Martin von Zweigbergk <martinvonz@google.com>
parents: 30393
diff changeset
249 match = self.match
27647
2c2858f3c1bb verify: move changelog verificaiton to its own function
Durham Goode <durham@fb.com>
parents: 27646
diff changeset
250 cl = repo.changelog
2c2858f3c1bb verify: move changelog verificaiton to its own function
Durham Goode <durham@fb.com>
parents: 27646
diff changeset
251
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
252 ui.status(_(b"checking changesets\n"))
27695
fb0cc863d172 verify: replace "output parameters" by return values
Martin von Zweigbergk <martinvonz@google.com>
parents: 27648
diff changeset
253 mflinkrevs = {}
fb0cc863d172 verify: replace "output parameters" by return values
Martin von Zweigbergk <martinvonz@google.com>
parents: 27648
diff changeset
254 filelinkrevs = {}
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
255 seen = {}
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
256 self._checkrevlog(cl, b"changelog", 0)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
257 progress = ui.makeprogress(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
258 _(b'checking'), unit=_(b'changesets'), total=len(repo)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
259 )
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
260 for i in repo:
38397
1249475f0bd6 verify: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 38396
diff changeset
261 progress.update(i)
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
262 n = cl.node(i)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
263 self._checkentry(cl, i, n, seen, [i], b"changelog")
2778
fdc232d8a193 Move repo.verify
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
264
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
265 try:
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
266 changes = cl.read(n)
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
267 if changes[0] != nullid:
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
268 mflinkrevs.setdefault(changes[0], []).append(i)
27444
6647401858ab verify: move widely used variables into class members
Durham Goode <durham@fb.com>
parents: 27443
diff changeset
269 self.refersmf = True
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
270 for f in changes[3]:
30866
5249b6470de9 verify: replace _validpath() by matcher
Martin von Zweigbergk <martinvonz@google.com>
parents: 30393
diff changeset
271 if match(f):
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
272 filelinkrevs.setdefault(_normpath(f), []).append(i)
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
273 except Exception as inst:
27444
6647401858ab verify: move widely used variables into class members
Durham Goode <durham@fb.com>
parents: 27443
diff changeset
274 self.refersmf = True
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
275 self._exc(i, _(b"unpacking changeset %s") % short(n), inst)
38397
1249475f0bd6 verify: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 38396
diff changeset
276 progress.complete()
27695
fb0cc863d172 verify: replace "output parameters" by return values
Martin von Zweigbergk <martinvonz@google.com>
parents: 27648
diff changeset
277 return mflinkrevs, filelinkrevs
2778
fdc232d8a193 Move repo.verify
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
278
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
279 def _verifymanifest(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
280 self, mflinkrevs, dir=b"", storefiles=None, subdirprogress=None
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
281 ):
41876
5ad5a70df2f7 verify: document the `_verifymanifest` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41875
diff changeset
282 """verify the manifestlog content
5ad5a70df2f7 verify: document the `_verifymanifest` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41875
diff changeset
283
5ad5a70df2f7 verify: document the `_verifymanifest` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41875
diff changeset
284 Inputs:
5ad5a70df2f7 verify: document the `_verifymanifest` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41875
diff changeset
285 - mflinkrevs: a {manifest-node -> [changelog-revisions]} mapping
5ad5a70df2f7 verify: document the `_verifymanifest` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41875
diff changeset
286 - dir: a subdirectory to check (for tree manifest repo)
5ad5a70df2f7 verify: document the `_verifymanifest` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41875
diff changeset
287 - storefiles: set of currently "orphan" files.
5ad5a70df2f7 verify: document the `_verifymanifest` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41875
diff changeset
288 - subdirprogress: a progress object
5ad5a70df2f7 verify: document the `_verifymanifest` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41875
diff changeset
289
5ad5a70df2f7 verify: document the `_verifymanifest` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41875
diff changeset
290 This function checks:
5ad5a70df2f7 verify: document the `_verifymanifest` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41875
diff changeset
291 * all of `_checkrevlog` checks (for all manifest related revlogs)
5ad5a70df2f7 verify: document the `_verifymanifest` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41875
diff changeset
292 * all of `_checkentry` checks (for all manifest related revisions)
5ad5a70df2f7 verify: document the `_verifymanifest` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41875
diff changeset
293 * nodes for subdirectory exists in the sub-directory manifest
5ad5a70df2f7 verify: document the `_verifymanifest` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41875
diff changeset
294 * each manifest entries have a file path
5ad5a70df2f7 verify: document the `_verifymanifest` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41875
diff changeset
295 * each manifest node refered in mflinkrevs exist in the manifest log
5ad5a70df2f7 verify: document the `_verifymanifest` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41875
diff changeset
296
5ad5a70df2f7 verify: document the `_verifymanifest` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41875
diff changeset
297 If tree manifest is in use and a matchers is specified, only the
5ad5a70df2f7 verify: document the `_verifymanifest` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41875
diff changeset
298 sub-directories matching it will be verified.
5ad5a70df2f7 verify: document the `_verifymanifest` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41875
diff changeset
299
5ad5a70df2f7 verify: document the `_verifymanifest` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41875
diff changeset
300 return a two level mapping:
5ad5a70df2f7 verify: document the `_verifymanifest` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41875
diff changeset
301 {"path" -> { filenode -> changelog-revision}}
5ad5a70df2f7 verify: document the `_verifymanifest` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41875
diff changeset
302
5ad5a70df2f7 verify: document the `_verifymanifest` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41875
diff changeset
303 This mapping primarily contains entries for every files in the
5ad5a70df2f7 verify: document the `_verifymanifest` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41875
diff changeset
304 repository. In addition, when tree-manifest is used, it also contains
5ad5a70df2f7 verify: document the `_verifymanifest` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41875
diff changeset
305 sub-directory entries.
5ad5a70df2f7 verify: document the `_verifymanifest` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41875
diff changeset
306
5ad5a70df2f7 verify: document the `_verifymanifest` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41875
diff changeset
307 If a matcher is provided, only matching paths will be included.
5ad5a70df2f7 verify: document the `_verifymanifest` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41875
diff changeset
308 """
27646
8f43793382c6 verify: move manifest verification to its own function
Durham Goode <durham@fb.com>
parents: 27645
diff changeset
309 repo = self.repo
8f43793382c6 verify: move manifest verification to its own function
Durham Goode <durham@fb.com>
parents: 27645
diff changeset
310 ui = self.ui
30866
5249b6470de9 verify: replace _validpath() by matcher
Martin von Zweigbergk <martinvonz@google.com>
parents: 30393
diff changeset
311 match = self.match
30295
f65faa4422c8 manifest: remove manifest.readshallowdelta
Durham Goode <durham@fb.com>
parents: 28467
diff changeset
312 mfl = self.repo.manifestlog
39244
73cf21b2e8a6 manifest: add getstorage() to manifestlog and use it globally
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38397
diff changeset
313 mf = mfl.getstorage(dir)
27646
8f43793382c6 verify: move manifest verification to its own function
Durham Goode <durham@fb.com>
parents: 27645
diff changeset
314
28203
7297e9e13a8a verify: check directory manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28115
diff changeset
315 if not dir:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
316 self.ui.status(_(b"checking manifests\n"))
28203
7297e9e13a8a verify: check directory manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28115
diff changeset
317
27695
fb0cc863d172 verify: replace "output parameters" by return values
Martin von Zweigbergk <martinvonz@google.com>
parents: 27648
diff changeset
318 filenodes = {}
28203
7297e9e13a8a verify: check directory manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28115
diff changeset
319 subdirnodes = {}
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
320 seen = {}
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
321 label = b"manifest"
28203
7297e9e13a8a verify: check directory manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28115
diff changeset
322 if dir:
7297e9e13a8a verify: check directory manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28115
diff changeset
323 label = dir
28204
962921c330b0 verify: check for orphaned dirlogs
Martin von Zweigbergk <martinvonz@google.com>
parents: 28203
diff changeset
324 revlogfiles = mf.files()
962921c330b0 verify: check for orphaned dirlogs
Martin von Zweigbergk <martinvonz@google.com>
parents: 28203
diff changeset
325 storefiles.difference_update(revlogfiles)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
326 if subdirprogress: # should be true since we're in a subdirectory
38396
0ddbe03c5aaa verify: use progress helper for subdirectory progress
Martin von Zweigbergk <martinvonz@google.com>
parents: 37417
diff changeset
327 subdirprogress.increment()
27444
6647401858ab verify: move widely used variables into class members
Durham Goode <durham@fb.com>
parents: 27443
diff changeset
328 if self.refersmf:
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
329 # Do not check manifest if there are only changelog entries with
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
330 # null manifests.
41874
1f412223f5bb verify: rename the `checklog` to `_checkrevlog`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41873
diff changeset
331 self._checkrevlog(mf, label, 0)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
332 progress = ui.makeprogress(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
333 _(b'checking'), unit=_(b'manifests'), total=len(mf)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
334 )
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
335 for i in mf:
28203
7297e9e13a8a verify: check directory manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28115
diff changeset
336 if not dir:
38397
1249475f0bd6 verify: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 38396
diff changeset
337 progress.update(i)
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
338 n = mf.node(i)
41871
9e737ca539f6 verify: make `checkentry` a private method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41870
diff changeset
339 lr = self._checkentry(mf, i, n, seen, mflinkrevs.get(n, []), label)
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
340 if n in mflinkrevs:
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
341 del mflinkrevs[n]
28203
7297e9e13a8a verify: check directory manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28115
diff changeset
342 elif dir:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
343 self._err(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
344 lr,
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
345 _(b"%s not in parent-directory manifest") % short(n),
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
346 label,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
347 )
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
348 else:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
349 self._err(lr, _(b"%s not in changesets") % short(n), label)
2778
fdc232d8a193 Move repo.verify
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
350
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
351 try:
30295
f65faa4422c8 manifest: remove manifest.readshallowdelta
Durham Goode <durham@fb.com>
parents: 28467
diff changeset
352 mfdelta = mfl.get(dir, n).readdelta(shallow=True)
f65faa4422c8 manifest: remove manifest.readshallowdelta
Durham Goode <durham@fb.com>
parents: 28467
diff changeset
353 for f, fn, fl in mfdelta.iterentries():
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
354 if not f:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
355 self._err(lr, _(b"entry without name in manifest"))
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
356 elif f == b"/dev/null": # ignore this in very old repos
28203
7297e9e13a8a verify: check directory manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28115
diff changeset
357 continue
7297e9e13a8a verify: check directory manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28115
diff changeset
358 fullpath = dir + _normpath(f)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
359 if fl == b't':
30866
5249b6470de9 verify: replace _validpath() by matcher
Martin von Zweigbergk <martinvonz@google.com>
parents: 30393
diff changeset
360 if not match.visitdir(fullpath):
5249b6470de9 verify: replace _validpath() by matcher
Martin von Zweigbergk <martinvonz@google.com>
parents: 30393
diff changeset
361 continue
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
362 subdirnodes.setdefault(fullpath + b'/', {}).setdefault(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
363 fn, []
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
364 ).append(lr)
28203
7297e9e13a8a verify: check directory manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28115
diff changeset
365 else:
30866
5249b6470de9 verify: replace _validpath() by matcher
Martin von Zweigbergk <martinvonz@google.com>
parents: 30393
diff changeset
366 if not match(fullpath):
5249b6470de9 verify: replace _validpath() by matcher
Martin von Zweigbergk <martinvonz@google.com>
parents: 30393
diff changeset
367 continue
28203
7297e9e13a8a verify: check directory manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28115
diff changeset
368 filenodes.setdefault(fullpath, {}).setdefault(fn, lr)
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
369 except Exception as inst:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
370 self._exc(lr, _(b"reading delta %s") % short(n), inst, label)
42157
7755b89cadaf verify: also check full manifest validity during verify runs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42156
diff changeset
371 if self._level >= VERIFY_FULL:
7755b89cadaf verify: also check full manifest validity during verify runs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42156
diff changeset
372 try:
7755b89cadaf verify: also check full manifest validity during verify runs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42156
diff changeset
373 # Various issues can affect manifest. So we read each full
7755b89cadaf verify: also check full manifest validity during verify runs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42156
diff changeset
374 # text from storage. This triggers the checks from the core
7755b89cadaf verify: also check full manifest validity during verify runs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42156
diff changeset
375 # code (eg: hash verification, filename are ordered, etc.)
7755b89cadaf verify: also check full manifest validity during verify runs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42156
diff changeset
376 mfdelta = mfl.get(dir, n).read()
7755b89cadaf verify: also check full manifest validity during verify runs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42156
diff changeset
377 except Exception as inst:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
378 self._exc(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
379 lr,
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
380 _(b"reading full manifest %s") % short(n),
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
381 inst,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
382 label,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
383 )
42157
7755b89cadaf verify: also check full manifest validity during verify runs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42156
diff changeset
384
28203
7297e9e13a8a verify: check directory manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28115
diff changeset
385 if not dir:
38397
1249475f0bd6 verify: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 38396
diff changeset
386 progress.complete()
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
387
28111
06205989264b verify: move cross-checking of changeset/manifest out of _crosscheckfiles()
Martin von Zweigbergk <martinvonz@google.com>
parents: 28007
diff changeset
388 if self.havemf:
41877
9c5a6af74afa verify: small refactoring and documentation in `_verifymanifest`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41876
diff changeset
389 # since we delete entry in `mflinkrevs` during iteration, any
9c5a6af74afa verify: small refactoring and documentation in `_verifymanifest`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41876
diff changeset
390 # remaining entries are "missing". We need to issue errors for them.
9c5a6af74afa verify: small refactoring and documentation in `_verifymanifest`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41876
diff changeset
391 changesetpairs = [(c, m) for m in mflinkrevs for c in mflinkrevs[m]]
9c5a6af74afa verify: small refactoring and documentation in `_verifymanifest`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41876
diff changeset
392 for c, m in sorted(changesetpairs):
28203
7297e9e13a8a verify: check directory manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28115
diff changeset
393 if dir:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
394 self._err(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
395 c,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
396 _(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
397 b"parent-directory manifest refers to unknown"
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
398 b" revision %s"
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
399 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
400 % short(m),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
401 label,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
402 )
28203
7297e9e13a8a verify: check directory manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28115
diff changeset
403 else:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
404 self._err(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
405 c,
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
406 _(b"changeset refers to unknown revision %s")
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
407 % short(m),
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
408 label,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
409 )
28203
7297e9e13a8a verify: check directory manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28115
diff changeset
410
7297e9e13a8a verify: check directory manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28115
diff changeset
411 if not dir and subdirnodes:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
412 self.ui.status(_(b"checking directory manifests\n"))
28204
962921c330b0 verify: check for orphaned dirlogs
Martin von Zweigbergk <martinvonz@google.com>
parents: 28203
diff changeset
413 storefiles = set()
28205
53f42c8d5f71 verify: show progress while verifying dirlogs
Martin von Zweigbergk <martinvonz@google.com>
parents: 28204
diff changeset
414 subdirs = set()
28204
962921c330b0 verify: check for orphaned dirlogs
Martin von Zweigbergk <martinvonz@google.com>
parents: 28203
diff changeset
415 revlogv1 = self.revlogv1
962921c330b0 verify: check for orphaned dirlogs
Martin von Zweigbergk <martinvonz@google.com>
parents: 28203
diff changeset
416 for f, f2, size in repo.store.datafiles():
962921c330b0 verify: check for orphaned dirlogs
Martin von Zweigbergk <martinvonz@google.com>
parents: 28203
diff changeset
417 if not f:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
418 self._err(None, _(b"cannot decode filename '%s'") % f2)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
419 elif (size > 0 or not revlogv1) and f.startswith(b'meta/'):
28204
962921c330b0 verify: check for orphaned dirlogs
Martin von Zweigbergk <martinvonz@google.com>
parents: 28203
diff changeset
420 storefiles.add(_normpath(f))
28205
53f42c8d5f71 verify: show progress while verifying dirlogs
Martin von Zweigbergk <martinvonz@google.com>
parents: 28204
diff changeset
421 subdirs.add(os.path.dirname(f))
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
422 subdirprogress = ui.makeprogress(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
423 _(b'checking'), unit=_(b'manifests'), total=len(subdirs)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
424 )
28204
962921c330b0 verify: check for orphaned dirlogs
Martin von Zweigbergk <martinvonz@google.com>
parents: 28203
diff changeset
425
43106
d783f945a701 py3: finish porting iteritems() to pycompat and remove source transformer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43077
diff changeset
426 for subdir, linkrevs in pycompat.iteritems(subdirnodes):
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
427 subdirfilenodes = self._verifymanifest(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
428 linkrevs, subdir, storefiles, subdirprogress
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
429 )
43106
d783f945a701 py3: finish porting iteritems() to pycompat and remove source transformer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43077
diff changeset
430 for f, onefilenodes in pycompat.iteritems(subdirfilenodes):
28203
7297e9e13a8a verify: check directory manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28115
diff changeset
431 filenodes.setdefault(f, {}).update(onefilenodes)
28111
06205989264b verify: move cross-checking of changeset/manifest out of _crosscheckfiles()
Martin von Zweigbergk <martinvonz@google.com>
parents: 28007
diff changeset
432
28204
962921c330b0 verify: check for orphaned dirlogs
Martin von Zweigbergk <martinvonz@google.com>
parents: 28203
diff changeset
433 if not dir and subdirnodes:
38396
0ddbe03c5aaa verify: use progress helper for subdirectory progress
Martin von Zweigbergk <martinvonz@google.com>
parents: 37417
diff changeset
434 subdirprogress.complete()
37417
76d2115cb817 verify: allow suppressing warnings about extra files
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37410
diff changeset
435 if self.warnorphanstorefiles:
76d2115cb817 verify: allow suppressing warnings about extra files
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37410
diff changeset
436 for f in sorted(storefiles):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
437 self._warn(_(b"warning: orphan data file '%s'") % f)
28204
962921c330b0 verify: check for orphaned dirlogs
Martin von Zweigbergk <martinvonz@google.com>
parents: 28203
diff changeset
438
27695
fb0cc863d172 verify: replace "output parameters" by return values
Martin von Zweigbergk <martinvonz@google.com>
parents: 27648
diff changeset
439 return filenodes
27645
df8973e1fb45 verify: move file cross checking to its own function
Durham Goode <durham@fb.com>
parents: 27644
diff changeset
440
28111
06205989264b verify: move cross-checking of changeset/manifest out of _crosscheckfiles()
Martin von Zweigbergk <martinvonz@google.com>
parents: 28007
diff changeset
441 def _crosscheckfiles(self, filelinkrevs, filenodes):
27645
df8973e1fb45 verify: move file cross checking to its own function
Durham Goode <durham@fb.com>
parents: 27644
diff changeset
442 repo = self.repo
df8973e1fb45 verify: move file cross checking to its own function
Durham Goode <durham@fb.com>
parents: 27644
diff changeset
443 ui = self.ui
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
444 ui.status(_(b"crosschecking files in changesets and manifests\n"))
2778
fdc232d8a193 Move repo.verify
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
445
28111
06205989264b verify: move cross-checking of changeset/manifest out of _crosscheckfiles()
Martin von Zweigbergk <martinvonz@google.com>
parents: 28007
diff changeset
446 total = len(filelinkrevs) + len(filenodes)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
447 progress = ui.makeprogress(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
448 _(b'crosschecking'), unit=_(b'files'), total=total
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
449 )
27645
df8973e1fb45 verify: move file cross checking to its own function
Durham Goode <durham@fb.com>
parents: 27644
diff changeset
450 if self.havemf:
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
451 for f in sorted(filelinkrevs):
38397
1249475f0bd6 verify: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 38396
diff changeset
452 progress.increment()
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
453 if f not in filenodes:
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
454 lr = filelinkrevs[f][0]
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
455 self._err(lr, _(b"in changeset but not in manifest"), f)
6892
dab95717058d verify: check repo.store
Adrian Buehlmann <adrian@cadifra.com>
parents: 6889
diff changeset
456
27645
df8973e1fb45 verify: move file cross checking to its own function
Durham Goode <durham@fb.com>
parents: 27644
diff changeset
457 if self.havecl:
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
458 for f in sorted(filenodes):
38397
1249475f0bd6 verify: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 38396
diff changeset
459 progress.increment()
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
460 if f not in filelinkrevs:
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
461 try:
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
462 fl = repo.file(f)
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
463 lr = min([fl.linkrev(fl.rev(n)) for n in filenodes[f]])
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
464 except Exception:
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
465 lr = None
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
466 self._err(lr, _(b"in manifest but not in changeset"), f)
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
467
38397
1249475f0bd6 verify: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 38396
diff changeset
468 progress.complete()
8291
f5c1a9094e41 verify: avoid exception on missing file revlog
Henrik Stuart <hg@hstuart.dk>
parents: 8225
diff changeset
469
27644
331e5c28f5f0 verify: move filelog verification to its own function
Durham Goode <durham@fb.com>
parents: 27643
diff changeset
470 def _verifyfiles(self, filenodes, filelinkrevs):
331e5c28f5f0 verify: move filelog verification to its own function
Durham Goode <durham@fb.com>
parents: 27643
diff changeset
471 repo = self.repo
331e5c28f5f0 verify: move filelog verification to its own function
Durham Goode <durham@fb.com>
parents: 27643
diff changeset
472 ui = self.ui
331e5c28f5f0 verify: move filelog verification to its own function
Durham Goode <durham@fb.com>
parents: 27643
diff changeset
473 lrugetctx = self.lrugetctx
331e5c28f5f0 verify: move filelog verification to its own function
Durham Goode <durham@fb.com>
parents: 27643
diff changeset
474 revlogv1 = self.revlogv1
331e5c28f5f0 verify: move filelog verification to its own function
Durham Goode <durham@fb.com>
parents: 27643
diff changeset
475 havemf = self.havemf
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
476 ui.status(_(b"checking files\n"))
8291
f5c1a9094e41 verify: avoid exception on missing file revlog
Henrik Stuart <hg@hstuart.dk>
parents: 8225
diff changeset
477
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
478 storefiles = set()
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
479 for f, f2, size in repo.store.datafiles():
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
480 if not f:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
481 self._err(None, _(b"cannot decode filename '%s'") % f2)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
482 elif (size > 0 or not revlogv1) and f.startswith(b'data/'):
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
483 storefiles.add(_normpath(f))
6892
dab95717058d verify: check repo.store
Adrian Buehlmann <adrian@cadifra.com>
parents: 6889
diff changeset
484
39842
97986c9c69d3 verify: start to abstract file verification
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39777
diff changeset
485 state = {
39845
e6d3d39cc1c7 revlog: use proper version comparison during verify
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39842
diff changeset
486 # TODO this assumes revlog storage for changelog.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
487 b'expectedversion': self.repo.changelog.version & 0xFFFF,
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
488 b'skipflags': self.skipflags,
39872
733db72f0f54 revlog: move revision verification out of verify
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39845
diff changeset
489 # experimental config: censor.policy
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
490 b'erroroncensored': ui.config(b'censor', b'policy') == b'abort',
39842
97986c9c69d3 verify: start to abstract file verification
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39777
diff changeset
491 }
97986c9c69d3 verify: start to abstract file verification
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39777
diff changeset
492
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
493 files = sorted(set(filenodes) | set(filelinkrevs))
27644
331e5c28f5f0 verify: move filelog verification to its own function
Durham Goode <durham@fb.com>
parents: 27643
diff changeset
494 revisions = 0
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
495 progress = ui.makeprogress(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
496 _(b'checking'), unit=_(b'files'), total=len(files)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
497 )
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
498 for i, f in enumerate(files):
38397
1249475f0bd6 verify: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 38396
diff changeset
499 progress.update(i, item=f)
6892
dab95717058d verify: check repo.store
Adrian Buehlmann <adrian@cadifra.com>
parents: 6889
diff changeset
500 try:
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
501 linkrevs = filelinkrevs[f]
6892
dab95717058d verify: check repo.store
Adrian Buehlmann <adrian@cadifra.com>
parents: 6889
diff changeset
502 except KeyError:
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
503 # in manifest but not in changelog
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
504 linkrevs = []
2778
fdc232d8a193 Move repo.verify
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
505
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
506 if linkrevs:
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
507 lr = linkrevs[0]
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
508 else:
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
509 lr = None
2778
fdc232d8a193 Move repo.verify
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
510
3744
d626fc9e3985 verify: add rename link checking
Matt Mackall <mpm@selenic.com>
parents: 3473
diff changeset
511 try:
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
512 fl = repo.file(f)
39777
b63dee7bd0d9 global: replace most uses of RevlogError with StorageError (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39489
diff changeset
513 except error.StorageError as e:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
514 self._err(lr, _(b"broken revlog! (%s)") % e, f)
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
515 continue
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
516
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
517 for ff in fl.files():
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
518 try:
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
519 storefiles.remove(ff)
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
520 except KeyError:
37417
76d2115cb817 verify: allow suppressing warnings about extra files
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37410
diff changeset
521 if self.warnorphanstorefiles:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
522 self._warn(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
523 _(b" warning: revlog '%s' not in fncache!") % ff
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
524 )
37417
76d2115cb817 verify: allow suppressing warnings about extra files
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37410
diff changeset
525 self.fncachewarned = True
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
526
39842
97986c9c69d3 verify: start to abstract file verification
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39777
diff changeset
527 if not len(fl) and (self.havecl or self.havemf):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
528 self._err(lr, _(b"empty or missing %s") % f)
39842
97986c9c69d3 verify: start to abstract file verification
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39777
diff changeset
529 else:
39872
733db72f0f54 revlog: move revision verification out of verify
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39845
diff changeset
530 # Guard against implementations not setting this.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
531 state[b'skipread'] = set()
39842
97986c9c69d3 verify: start to abstract file verification
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39777
diff changeset
532 for problem in fl.verifyintegrity(state):
39872
733db72f0f54 revlog: move revision verification out of verify
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39845
diff changeset
533 if problem.node is not None:
733db72f0f54 revlog: move revision verification out of verify
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39845
diff changeset
534 linkrev = fl.linkrev(fl.rev(problem.node))
733db72f0f54 revlog: move revision verification out of verify
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39845
diff changeset
535 else:
733db72f0f54 revlog: move revision verification out of verify
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39845
diff changeset
536 linkrev = None
733db72f0f54 revlog: move revision verification out of verify
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39845
diff changeset
537
39842
97986c9c69d3 verify: start to abstract file verification
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39777
diff changeset
538 if problem.warning:
41862
c66037fb1bc5 verify: make the `warn` method private
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41861
diff changeset
539 self._warn(problem.warning)
39842
97986c9c69d3 verify: start to abstract file verification
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39777
diff changeset
540 elif problem.error:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
541 self._err(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
542 linkrev if linkrev is not None else lr,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
543 problem.error,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
544 f,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
545 )
39842
97986c9c69d3 verify: start to abstract file verification
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39777
diff changeset
546 else:
97986c9c69d3 verify: start to abstract file verification
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39777
diff changeset
547 raise error.ProgrammingError(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
548 b'problem instance does not set warning or error '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
549 b'attribute: %s' % problem.msg
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
550 )
39842
97986c9c69d3 verify: start to abstract file verification
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39777
diff changeset
551
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
552 seen = {}
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
553 for i in fl:
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
554 revisions += 1
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
555 n = fl.node(i)
41871
9e737ca539f6 verify: make `checkentry` a private method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41870
diff changeset
556 lr = self._checkentry(fl, i, n, seen, linkrevs, f)
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
557 if f in filenodes:
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
558 if havemf and n not in filenodes[f]:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
559 self._err(lr, _(b"%s not in manifests") % (short(n)), f)
6534
9b35a9f34675 verify: check copy source revlog and nodeid
Patrick Mezard <pmezard@gmail.com>
parents: 6211
diff changeset
560 else:
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
561 del filenodes[f][n]
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
562
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
563 if n in state[b'skipread']:
39872
733db72f0f54 revlog: move revision verification out of verify
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39845
diff changeset
564 continue
3744
d626fc9e3985 verify: add rename link checking
Matt Mackall <mpm@selenic.com>
parents: 3473
diff changeset
565
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
566 # check renames
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
567 try:
39872
733db72f0f54 revlog: move revision verification out of verify
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39845
diff changeset
568 # This requires resolving fulltext (at least on revlogs). We
733db72f0f54 revlog: move revision verification out of verify
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39845
diff changeset
569 # may want ``verifyintegrity()`` to pass a set of nodes with
733db72f0f54 revlog: move revision verification out of verify
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39845
diff changeset
570 # rename metadata as an optimization.
733db72f0f54 revlog: move revision verification out of verify
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39845
diff changeset
571 rp = fl.renamed(n)
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
572 if rp:
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
573 if lr is not None and ui.verbose:
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
574 ctx = lrugetctx(lr)
36339
a4d41ba4ad23 verify: don't reimplement any()
Martin von Zweigbergk <martinvonz@google.com>
parents: 36185
diff changeset
575 if not any(rp[0] in pctx for pctx in ctx.parents()):
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
576 self._warn(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
577 _(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
578 b"warning: copy source of '%s' not"
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
579 b" in parents of %s"
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
580 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
581 % (f, ctx)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
582 )
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
583 fl2 = repo.file(rp[0])
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
584 if not len(fl2):
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
585 self._err(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
586 lr,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
587 _(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
588 b"empty or missing copy source revlog "
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
589 b"%s:%s"
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
590 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
591 % (rp[0], short(rp[1])),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
592 f,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
593 )
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
594 elif rp[1] == nullid:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
595 ui.note(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
596 _(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
597 b"warning: %s@%s: copy source"
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
598 b" revision is nullid %s:%s\n"
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
599 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
600 % (f, lr, rp[0], short(rp[1]))
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
601 )
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
602 else:
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
603 fl2.rev(rp[1])
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
604 except Exception as inst:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
605 self._exc(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
606 lr, _(b"checking rename of %s") % short(n), inst, f
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
607 )
6892
dab95717058d verify: check repo.store
Adrian Buehlmann <adrian@cadifra.com>
parents: 6889
diff changeset
608
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
609 # cross-check
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
610 if f in filenodes:
43106
d783f945a701 py3: finish porting iteritems() to pycompat and remove source transformer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43077
diff changeset
611 fns = [(v, k) for k, v in pycompat.iteritems(filenodes[f])]
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
612 for lr, node in sorted(fns):
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
613 self._err(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
614 lr,
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
615 _(b"manifest refers to unknown revision %s")
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
616 % short(node),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
617 f,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42387
diff changeset
618 )
38397
1249475f0bd6 verify: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 38396
diff changeset
619 progress.complete()
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
620
37417
76d2115cb817 verify: allow suppressing warnings about extra files
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37410
diff changeset
621 if self.warnorphanstorefiles:
76d2115cb817 verify: allow suppressing warnings about extra files
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37410
diff changeset
622 for f in sorted(storefiles):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
623 self._warn(_(b"warning: orphan data file '%s'") % f)
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
624
27644
331e5c28f5f0 verify: move filelog verification to its own function
Durham Goode <durham@fb.com>
parents: 27643
diff changeset
625 return len(files), revisions