annotate mercurial/revlogutils/rewrite.py @ 47819:c02ce6def30c stable

issue6528: implement _is_revision_affected_fast using callback The delta comming from a bundle/stream does not exists in the revlog yet, so we will need other way to retrieve the same information. To prepare for this we split the function to use callbacks in the core logic. Differential Revision: https://phab.mercurial-scm.org/D11268
author Pierre-Yves David <pierre-yves.david@octobus.net>
date Sat, 07 Aug 2021 12:39:01 +0200
parents 5b046c2e3000
children c30ca163b45e
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
47391
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
1 # censor code related to censoring revision
47457
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
2 # coding: utf8
8226
8b2cd04a6e97 put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents: 8225
diff changeset
3 #
47391
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
4 # Copyright 2021 Pierre-Yves David <pierre-yves.david@octobus.net>
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
5 # Copyright 2015 Google, Inc <martinvonz@google.com>
8226
8b2cd04a6e97 put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents: 8225
diff changeset
6 #
8b2cd04a6e97 put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents: 8225
diff changeset
7 # This software may be used and distributed according to the terms of the
10263
25e572394f5c Update license to GPLv2+
Matt Mackall <mpm@selenic.com>
parents: 10047
diff changeset
8 # GNU General Public License version 2 or any later version.
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
9
47815
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
10 import binascii
47457
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
11 import contextlib
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
12 import os
47816
32e21ac3adb1 repair: improve performance of detection of revisions affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47815
diff changeset
13 import struct
47457
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
14
47391
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
15 from ..node import (
27361
29f50344fa83 revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27251
diff changeset
16 nullrev,
39767
db088e133e91 revlog: define ellipsis flag processors in core
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39333
diff changeset
17 )
47457
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
18 from .constants import (
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
19 COMP_MODE_PLAIN,
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
20 ENTRY_DATA_COMPRESSED_LENGTH,
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
21 ENTRY_DATA_COMPRESSION_MODE,
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
22 ENTRY_DATA_OFFSET,
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
23 ENTRY_DATA_UNCOMPRESSED_LENGTH,
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
24 ENTRY_DELTA_BASE,
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
25 ENTRY_LINK_REV,
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
26 ENTRY_NODE_ID,
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
27 ENTRY_PARENT_1,
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
28 ENTRY_PARENT_2,
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
29 ENTRY_SIDEDATA_COMPRESSED_LENGTH,
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
30 ENTRY_SIDEDATA_COMPRESSION_MODE,
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
31 ENTRY_SIDEDATA_OFFSET,
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
32 REVLOGV0,
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
33 REVLOGV1,
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
34 )
47391
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
35 from ..i18n import _
47457
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
36
47391
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
37 from .. import (
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
38 error,
47457
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
39 pycompat,
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
40 revlogutils,
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
41 util,
46310
fc2d5c0aed7f persistent-nodemap: add a "warn" option to the slow-path config
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45942
diff changeset
42 )
47391
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
43 from ..utils import (
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
44 storageutil,
47386
0d0fb091c49f revlog: simplify "partial read" error message
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47385
diff changeset
45 )
47457
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
46 from . import (
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
47 constants,
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
48 deltas,
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
49 )
47246
02a4463565ea revlog: improve documentation of the entry tuple
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47245
diff changeset
50
02a4463565ea revlog: improve documentation of the entry tuple
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47245
diff changeset
51
47391
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
52 def v1_censor(rl, tr, censornode, tombstone=b''):
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
53 """censors a revision in a "version 1" revlog"""
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
54 assert rl._format_version == constants.REVLOGV1, rl._format_version
47151
24be247a13b4 revlog: stop usage of `_indexfile` to computing nodemap path
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47150
diff changeset
55
47391
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
56 # avoid cycle
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
57 from .. import revlog
47144
b6e1fe7ac24b revlog: split the option initialisation in its own method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47143
diff changeset
58
47391
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
59 censorrev = rl.rev(censornode)
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
60 tombstone = storageutil.packmeta({b'censored': tombstone}, b'')
30154
5e72129d75ed revlog: add instance variable controlling delta chain use
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30012
diff changeset
61
47391
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
62 # Rewriting the revlog in place is hard. Our strategy for censoring is
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
63 # to create a new revlog, copy all revisions to it, then replace the
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
64 # revlogs on transaction close.
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
65 #
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
66 # This is a bit dangerous. We could easily have a mismatch of state.
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
67 newrl = revlog.revlog(
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
68 rl.opener,
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
69 target=rl.target,
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
70 radix=rl.radix,
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
71 postfix=b'tmpcensored',
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
72 censorable=True,
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
73 )
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
74 newrl._format_version = rl._format_version
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
75 newrl._format_flags = rl._format_flags
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
76 newrl._generaldelta = rl._generaldelta
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
77 newrl._parse_index = rl._parse_index
23337
3a8a763f4197 revlog: add a method to get missing revs incrementally
Siddharth Agarwal <sid0@fb.com>
parents: 23328
diff changeset
78
47391
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
79 for rev in rl.revs():
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
80 node = rl.node(rev)
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
81 p1, p2 = rl.parents(node)
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
82
47391
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
83 if rev == censorrev:
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
84 newrl.addrawrevision(
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
85 tombstone,
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
86 tr,
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
87 rl.linkrev(censorrev),
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
88 p1,
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
89 p2,
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
90 censornode,
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
91 constants.REVIDX_ISCENSORED,
47294
93a0abe098e7 revlog: avoid raising no-arg RevlogError for internal flow control
Martin von Zweigbergk <martinvonz@google.com>
parents: 47285
diff changeset
92 )
16665
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16533
diff changeset
93
47391
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
94 if newrl.deltaparent(rev) != nullrev:
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
95 m = _(b'censored revision stored as delta; cannot censor')
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
96 h = _(
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
97 b'censoring of revlogs is not fully implemented;'
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
98 b' please report this bug'
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
99 )
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
100 raise error.Abort(m, hint=h)
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
101 continue
42980
0d1272783f24 revlog: introduce a `sidedata` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42879
diff changeset
102
47391
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
103 if rl.iscensored(rev):
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
104 if rl.deltaparent(rev) != nullrev:
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
105 m = _(
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
106 b'cannot censor due to censored '
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
107 b'revision having delta stored'
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
108 )
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
109 raise error.Abort(m)
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
110 rawtext = rl._chunk(rev)
42879
4a3efe0febb5 revlog: stop using `_processflags` directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42876
diff changeset
111 else:
47391
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
112 rawtext = rl.rawdata(rev)
8315
c8493310ad9b revlog: use index to find index size
Matt Mackall <mpm@selenic.com>
parents: 8314
diff changeset
113
47391
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
114 newrl.addrawrevision(
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
115 rawtext, tr, rl.linkrev(rev), p1, p2, node, rl.flags(rev)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43039
diff changeset
116 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43039
diff changeset
117
47391
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
118 tr.addbackup(rl._indexfile, location=b'store')
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
119 if not rl._inline:
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
120 tr.addbackup(rl._datafile, location=b'store')
1667
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
121
47391
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
122 rl.opener.rename(newrl._indexfile, rl._indexfile)
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
123 if not rl._inline:
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
124 rl.opener.rename(newrl._datafile, rl._datafile)
39778
a6b3c4c1019f revlog: move censor logic out of censor extension
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39775
diff changeset
125
47391
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
126 rl.clearcaches()
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
127 rl._loadindex()
47457
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
128
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
129
47471
aab064416f0c censor: rename `rl` to `revlog` in the main function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47470
diff changeset
130 def v2_censor(revlog, tr, censornode, tombstone=b''):
47457
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
131 """censors a revision in a "version 2" revlog"""
47473
5045ba2a3afd censor: split the core of the logic into its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
132 assert revlog._format_version != REVLOGV0, revlog._format_version
5045ba2a3afd censor: split the core of the logic into its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
133 assert revlog._format_version != REVLOGV1, revlog._format_version
5045ba2a3afd censor: split the core of the logic into its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
134
5045ba2a3afd censor: split the core of the logic into its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
135 censor_revs = {revlog.rev(censornode)}
5045ba2a3afd censor: split the core of the logic into its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
136 _rewrite_v2(revlog, tr, censor_revs, tombstone)
5045ba2a3afd censor: split the core of the logic into its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
137
5045ba2a3afd censor: split the core of the logic into its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
138
5045ba2a3afd censor: split the core of the logic into its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
139 def _rewrite_v2(revlog, tr, censor_revs, tombstone=b''):
5045ba2a3afd censor: split the core of the logic into its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
140 """rewrite a revlog to censor some of its content
5045ba2a3afd censor: split the core of the logic into its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
141
5045ba2a3afd censor: split the core of the logic into its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
142 General principle
47457
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
143
47473
5045ba2a3afd censor: split the core of the logic into its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
144 We create new revlog files (index/data/sidedata) to copy the content of
5045ba2a3afd censor: split the core of the logic into its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
145 the existing data without the censored data.
5045ba2a3afd censor: split the core of the logic into its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
146
5045ba2a3afd censor: split the core of the logic into its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
147 We need to recompute new delta for any revision that used the censored
5045ba2a3afd censor: split the core of the logic into its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
148 revision as delta base. As the cumulative size of the new delta may be
5045ba2a3afd censor: split the core of the logic into its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
149 large, we store them in a temporary file until they are stored in their
5045ba2a3afd censor: split the core of the logic into its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
150 final destination.
5045ba2a3afd censor: split the core of the logic into its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
151
5045ba2a3afd censor: split the core of the logic into its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
152 All data before the censored data can be blindly copied. The rest needs
5045ba2a3afd censor: split the core of the logic into its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
153 to be copied as we go and the associated index entry needs adjustement.
5045ba2a3afd censor: split the core of the logic into its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
154 """
47471
aab064416f0c censor: rename `rl` to `revlog` in the main function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47470
diff changeset
155 assert revlog._format_version != REVLOGV0, revlog._format_version
aab064416f0c censor: rename `rl` to `revlog` in the main function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47470
diff changeset
156 assert revlog._format_version != REVLOGV1, revlog._format_version
47457
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
157
47471
aab064416f0c censor: rename `rl` to `revlog` in the main function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47470
diff changeset
158 old_index = revlog.index
aab064416f0c censor: rename `rl` to `revlog` in the main function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47470
diff changeset
159 docket = revlog._docket
47457
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
160
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
161 tombstone = storageutil.packmeta({b'censored': tombstone}, b'')
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
162
47472
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
163 first_excl_rev = min(censor_revs)
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
164
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
165 first_excl_entry = revlog.index[first_excl_rev]
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
166 index_cutoff = revlog.index.entry_size * first_excl_rev
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
167 data_cutoff = first_excl_entry[ENTRY_DATA_OFFSET] >> 16
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
168 sidedata_cutoff = revlog.sidedata_cut_off(first_excl_rev)
47457
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
169
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
170 with pycompat.unnamedtempfile(mode=b"w+b") as tmp_storage:
47469
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
171 # rev → (new_base, data_start, data_end, compression_mode)
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
172 rewritten_entries = _precompute_rewritten_delta(
47471
aab064416f0c censor: rename `rl` to `revlog` in the main function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47470
diff changeset
173 revlog,
47469
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
174 old_index,
47472
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
175 censor_revs,
47469
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
176 tmp_storage,
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
177 )
47457
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
178
47470
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
179 all_files = _setup_new_files(
47471
aab064416f0c censor: rename `rl` to `revlog` in the main function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47470
diff changeset
180 revlog,
47470
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
181 index_cutoff,
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
182 data_cutoff,
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
183 sidedata_cutoff,
47457
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
184 )
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
185
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
186 # we dont need to open the old index file since its content already
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
187 # exist in a usable form in `old_index`.
47466
f7a94e2d4470 censor: put the tuple of open files in an explicit variable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47465
diff changeset
188 with all_files() as open_files:
f7a94e2d4470 censor: put the tuple of open files in an explicit variable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47465
diff changeset
189 (
f7a94e2d4470 censor: put the tuple of open files in an explicit variable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47465
diff changeset
190 old_data_file,
f7a94e2d4470 censor: put the tuple of open files in an explicit variable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47465
diff changeset
191 old_sidedata_file,
f7a94e2d4470 censor: put the tuple of open files in an explicit variable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47465
diff changeset
192 new_index_file,
f7a94e2d4470 censor: put the tuple of open files in an explicit variable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47465
diff changeset
193 new_data_file,
f7a94e2d4470 censor: put the tuple of open files in an explicit variable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47465
diff changeset
194 new_sidedata_file,
f7a94e2d4470 censor: put the tuple of open files in an explicit variable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47465
diff changeset
195 ) = open_files
47457
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
196
47467
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
197 # writing the censored revision
47457
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
198
47468
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
199 # Writing all subsequent revisions
47472
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
200 for rev in range(first_excl_rev, len(old_index)):
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
201 if rev in censor_revs:
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
202 _rewrite_censor(
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
203 revlog,
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
204 old_index,
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
205 open_files,
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
206 rev,
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
207 tombstone,
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
208 )
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
209 else:
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
210 _rewrite_simple(
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
211 revlog,
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
212 old_index,
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
213 open_files,
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
214 rev,
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
215 rewritten_entries,
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
216 tmp_storage,
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
217 )
47468
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
218 docket.write(transaction=None, stripping=True)
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
219
47457
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
220
47469
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
221 def _precompute_rewritten_delta(
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
222 revlog,
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
223 old_index,
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
224 excluded_revs,
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
225 tmp_storage,
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
226 ):
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
227 """Compute new delta for revisions whose delta is based on revision that
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
228 will not survive as is.
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
229
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
230 Return a mapping: {rev → (new_base, data_start, data_end, compression_mode)}
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
231 """
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
232 dc = deltas.deltacomputer(revlog)
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
233 rewritten_entries = {}
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
234 first_excl_rev = min(excluded_revs)
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
235 with revlog._segmentfile._open_read() as dfh:
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
236 for rev in range(first_excl_rev, len(old_index)):
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
237 if rev in excluded_revs:
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
238 # this revision will be preserved as is, so we don't need to
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
239 # consider recomputing a delta.
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
240 continue
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
241 entry = old_index[rev]
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
242 if entry[ENTRY_DELTA_BASE] not in excluded_revs:
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
243 continue
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
244 # This is a revision that use the censored revision as the base
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
245 # for its delta. We need a need new deltas
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
246 if entry[ENTRY_DATA_UNCOMPRESSED_LENGTH] == 0:
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
247 # this revision is empty, we can delta against nullrev
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
248 rewritten_entries[rev] = (nullrev, 0, 0, COMP_MODE_PLAIN)
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
249 else:
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
250
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
251 text = revlog.rawdata(rev, _df=dfh)
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
252 info = revlogutils.revisioninfo(
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
253 node=entry[ENTRY_NODE_ID],
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
254 p1=revlog.node(entry[ENTRY_PARENT_1]),
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
255 p2=revlog.node(entry[ENTRY_PARENT_2]),
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
256 btext=[text],
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
257 textlen=len(text),
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
258 cachedelta=None,
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
259 flags=entry[ENTRY_DATA_OFFSET] & 0xFFFF,
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
260 )
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
261 d = dc.finddeltainfo(
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
262 info, dfh, excluded_bases=excluded_revs, target_rev=rev
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
263 )
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
264 default_comp = revlog._docket.default_compression_header
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
265 comp_mode, d = deltas.delta_compression(default_comp, d)
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
266 # using `tell` is a bit lazy, but we are not here for speed
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
267 start = tmp_storage.tell()
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
268 tmp_storage.write(d.data[1])
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
269 end = tmp_storage.tell()
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
270 rewritten_entries[rev] = (d.base, start, end, comp_mode)
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
271 return rewritten_entries
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
272
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
273
47470
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
274 def _setup_new_files(
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
275 revlog,
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
276 index_cutoff,
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
277 data_cutoff,
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
278 sidedata_cutoff,
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
279 ):
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
280 """
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
281
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
282 return a context manager to open all the relevant files:
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
283 - old_data_file,
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
284 - old_sidedata_file,
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
285 - new_index_file,
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
286 - new_data_file,
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
287 - new_sidedata_file,
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
288
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
289 The old_index_file is not here because it is accessed through the
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
290 `old_index` object if the caller function.
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
291 """
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
292 docket = revlog._docket
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
293 old_index_filepath = revlog.opener.join(docket.index_filepath())
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
294 old_data_filepath = revlog.opener.join(docket.data_filepath())
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
295 old_sidedata_filepath = revlog.opener.join(docket.sidedata_filepath())
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
296
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
297 new_index_filepath = revlog.opener.join(docket.new_index_file())
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
298 new_data_filepath = revlog.opener.join(docket.new_data_file())
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
299 new_sidedata_filepath = revlog.opener.join(docket.new_sidedata_file())
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
300
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
301 util.copyfile(old_index_filepath, new_index_filepath, nb_bytes=index_cutoff)
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
302 util.copyfile(old_data_filepath, new_data_filepath, nb_bytes=data_cutoff)
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
303 util.copyfile(
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
304 old_sidedata_filepath,
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
305 new_sidedata_filepath,
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
306 nb_bytes=sidedata_cutoff,
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
307 )
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
308 revlog.opener.register_file(docket.index_filepath())
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
309 revlog.opener.register_file(docket.data_filepath())
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
310 revlog.opener.register_file(docket.sidedata_filepath())
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
311
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
312 docket.index_end = index_cutoff
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
313 docket.data_end = data_cutoff
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
314 docket.sidedata_end = sidedata_cutoff
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
315
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
316 # reload the revlog internal information
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
317 revlog.clearcaches()
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
318 revlog._loadindex(docket=docket)
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
319
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
320 @contextlib.contextmanager
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
321 def all_files_opener():
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
322 # hide opening in an helper function to please check-code, black
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
323 # and various python version at the same time
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
324 with open(old_data_filepath, 'rb') as old_data_file:
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
325 with open(old_sidedata_filepath, 'rb') as old_sidedata_file:
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
326 with open(new_index_filepath, 'r+b') as new_index_file:
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
327 with open(new_data_filepath, 'r+b') as new_data_file:
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
328 with open(
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
329 new_sidedata_filepath, 'r+b'
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
330 ) as new_sidedata_file:
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
331 new_index_file.seek(0, os.SEEK_END)
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
332 assert new_index_file.tell() == index_cutoff
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
333 new_data_file.seek(0, os.SEEK_END)
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
334 assert new_data_file.tell() == data_cutoff
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
335 new_sidedata_file.seek(0, os.SEEK_END)
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
336 assert new_sidedata_file.tell() == sidedata_cutoff
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
337 yield (
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
338 old_data_file,
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
339 old_sidedata_file,
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
340 new_index_file,
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
341 new_data_file,
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
342 new_sidedata_file,
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
343 )
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
344
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
345 return all_files_opener
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
346
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
347
47468
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
348 def _rewrite_simple(
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
349 revlog,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
350 old_index,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
351 all_files,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
352 rev,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
353 rewritten_entries,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
354 tmp_storage,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
355 ):
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
356 """append a normal revision to the index after the rewritten one(s)"""
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
357 (
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
358 old_data_file,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
359 old_sidedata_file,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
360 new_index_file,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
361 new_data_file,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
362 new_sidedata_file,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
363 ) = all_files
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
364 entry = old_index[rev]
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
365 flags = entry[ENTRY_DATA_OFFSET] & 0xFFFF
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
366 old_data_offset = entry[ENTRY_DATA_OFFSET] >> 16
47457
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
367
47468
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
368 if rev not in rewritten_entries:
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
369 old_data_file.seek(old_data_offset)
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
370 new_data_size = entry[ENTRY_DATA_COMPRESSED_LENGTH]
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
371 new_data = old_data_file.read(new_data_size)
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
372 data_delta_base = entry[ENTRY_DELTA_BASE]
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
373 d_comp_mode = entry[ENTRY_DATA_COMPRESSION_MODE]
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
374 else:
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
375 (
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
376 data_delta_base,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
377 start,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
378 end,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
379 d_comp_mode,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
380 ) = rewritten_entries[rev]
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
381 new_data_size = end - start
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
382 tmp_storage.seek(start)
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
383 new_data = tmp_storage.read(new_data_size)
47457
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
384
47468
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
385 # It might be faster to group continuous read/write operation,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
386 # however, this is censor, an operation that is not focussed
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
387 # around stellar performance. So I have not written this
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
388 # optimisation yet.
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
389 new_data_offset = new_data_file.tell()
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
390 new_data_file.write(new_data)
47457
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
391
47468
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
392 sidedata_size = entry[ENTRY_SIDEDATA_COMPRESSED_LENGTH]
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
393 new_sidedata_offset = new_sidedata_file.tell()
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
394 if 0 < sidedata_size:
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
395 old_sidedata_offset = entry[ENTRY_SIDEDATA_OFFSET]
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
396 old_sidedata_file.seek(old_sidedata_offset)
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
397 new_sidedata = old_sidedata_file.read(sidedata_size)
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
398 new_sidedata_file.write(new_sidedata)
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
399
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
400 data_uncompressed_length = entry[ENTRY_DATA_UNCOMPRESSED_LENGTH]
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
401 sd_com_mode = entry[ENTRY_SIDEDATA_COMPRESSION_MODE]
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
402 assert data_delta_base <= rev, (data_delta_base, rev)
47457
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
403
47468
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
404 new_entry = revlogutils.entry(
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
405 flags=flags,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
406 data_offset=new_data_offset,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
407 data_compressed_length=new_data_size,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
408 data_uncompressed_length=data_uncompressed_length,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
409 data_delta_base=data_delta_base,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
410 link_rev=entry[ENTRY_LINK_REV],
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
411 parent_rev_1=entry[ENTRY_PARENT_1],
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
412 parent_rev_2=entry[ENTRY_PARENT_2],
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
413 node_id=entry[ENTRY_NODE_ID],
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
414 sidedata_offset=new_sidedata_offset,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
415 sidedata_compressed_length=sidedata_size,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
416 data_compression_mode=d_comp_mode,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
417 sidedata_compression_mode=sd_com_mode,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
418 )
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
419 revlog.index.append(new_entry)
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
420 entry_bin = revlog.index.entry_binary(rev)
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
421 new_index_file.write(entry_bin)
47457
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
422
47468
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
423 revlog._docket.index_end = new_index_file.tell()
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
424 revlog._docket.data_end = new_data_file.tell()
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
425 revlog._docket.sidedata_end = new_sidedata_file.tell()
47467
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
426
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
427
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
428 def _rewrite_censor(
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
429 revlog,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
430 old_index,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
431 all_files,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
432 rev,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
433 tombstone,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
434 ):
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
435 """rewrite and append a censored revision"""
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
436 (
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
437 old_data_file,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
438 old_sidedata_file,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
439 new_index_file,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
440 new_data_file,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
441 new_sidedata_file,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
442 ) = all_files
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
443 entry = old_index[rev]
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
444
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
445 # XXX consider trying the default compression too
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
446 new_data_size = len(tombstone)
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
447 new_data_offset = new_data_file.tell()
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
448 new_data_file.write(tombstone)
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
449
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
450 # we are not adding any sidedata as they might leak info about the censored version
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
451
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
452 link_rev = entry[ENTRY_LINK_REV]
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
453
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
454 p1 = entry[ENTRY_PARENT_1]
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
455 p2 = entry[ENTRY_PARENT_2]
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
456
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
457 new_entry = revlogutils.entry(
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
458 flags=constants.REVIDX_ISCENSORED,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
459 data_offset=new_data_offset,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
460 data_compressed_length=new_data_size,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
461 data_uncompressed_length=new_data_size,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
462 data_delta_base=rev,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
463 link_rev=link_rev,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
464 parent_rev_1=p1,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
465 parent_rev_2=p2,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
466 node_id=entry[ENTRY_NODE_ID],
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
467 sidedata_offset=0,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
468 sidedata_compressed_length=0,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
469 data_compression_mode=COMP_MODE_PLAIN,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
470 sidedata_compression_mode=COMP_MODE_PLAIN,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
471 )
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
472 revlog.index.append(new_entry)
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
473 entry_bin = revlog.index.entry_binary(rev)
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
474 new_index_file.write(entry_bin)
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
475 revlog._docket.index_end = new_index_file.tell()
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
476 revlog._docket.data_end = new_data_file.tell()
47815
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
477
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
478
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
479 def _get_filename_from_filelog_index(path):
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
480 # Drop the extension and the `data/` prefix
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
481 path_part = path.rsplit(b'.', 1)[0].split(b'/', 1)
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
482 if len(path_part) < 2:
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
483 msg = _(b"cannot recognize filelog from filename: '%s'")
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
484 msg %= path
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
485 raise error.Abort(msg)
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
486
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
487 return path_part[1]
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
488
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
489
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
490 def _filelog_from_filename(repo, path):
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
491 """Returns the filelog for the given `path`. Stolen from `engine.py`"""
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
492
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
493 from .. import filelog # avoid cycle
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
494
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
495 fl = filelog.filelog(repo.svfs, path)
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
496 return fl
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
497
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
498
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
499 def _write_swapped_parents(repo, rl, rev, offset, fp):
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
500 """Swaps p1 and p2 and overwrites the revlog entry for `rev` in `fp`"""
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
501 from ..pure import parsers # avoid cycle
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
502
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
503 if repo._currentlock(repo._lockref) is None:
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
504 # Let's be paranoid about it
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
505 msg = "repo needs to be locked to rewrite parents"
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
506 raise error.ProgrammingError(msg)
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
507
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
508 index_format = parsers.IndexObject.index_format
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
509 entry = rl.index[rev]
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
510 new_entry = list(entry)
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
511 new_entry[5], new_entry[6] = entry[6], entry[5]
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
512 packed = index_format.pack(*new_entry[:8])
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
513 fp.seek(offset)
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
514 fp.write(packed)
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
515
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
516
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
517 def _reorder_filelog_parents(repo, fl, to_fix):
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
518 """
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
519 Swaps p1 and p2 for all `to_fix` revisions of filelog `fl` and writes the
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
520 new version to disk, overwriting the old one with a rename.
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
521 """
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
522 from ..pure import parsers # avoid cycle
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
523
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
524 ui = repo.ui
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
525 assert len(to_fix) > 0
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
526 rl = fl._revlog
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
527 if rl._format_version != constants.REVLOGV1:
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
528 msg = "expected version 1 revlog, got version '%d'" % rl._format_version
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
529 raise error.ProgrammingError(msg)
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
530
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
531 index_file = rl._indexfile
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
532 new_file_path = index_file + b'.tmp-parents-fix'
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
533 repaired_msg = _(b"repaired revision %d of 'filelog %s'\n")
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
534
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
535 with ui.uninterruptible():
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
536 try:
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
537 util.copyfile(
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
538 rl.opener.join(index_file),
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
539 rl.opener.join(new_file_path),
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
540 checkambig=rl._checkambig,
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
541 )
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
542
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
543 with rl.opener(new_file_path, mode=b"r+") as fp:
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
544 if rl._inline:
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
545 index = parsers.InlinedIndexObject(fp.read())
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
546 for rev in fl.revs():
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
547 if rev in to_fix:
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
548 offset = index._calculate_index(rev)
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
549 _write_swapped_parents(repo, rl, rev, offset, fp)
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
550 ui.write(repaired_msg % (rev, index_file))
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
551 else:
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
552 index_format = parsers.IndexObject.index_format
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
553 for rev in to_fix:
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
554 offset = rev * index_format.size
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
555 _write_swapped_parents(repo, rl, rev, offset, fp)
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
556 ui.write(repaired_msg % (rev, index_file))
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
557
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
558 rl.opener.rename(new_file_path, index_file)
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
559 rl.clearcaches()
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
560 rl._loadindex()
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
561 finally:
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
562 util.tryunlink(new_file_path)
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
563
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
564
47816
32e21ac3adb1 repair: improve performance of detection of revisions affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47815
diff changeset
565 def _is_revision_affected(fl, filerev, metadata_cache=None):
47818
5b046c2e3000 issue6528: implement _is_revision_affected using callback
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47817
diff changeset
566 full_text = lambda: fl._revlog.rawdata(filerev)
5b046c2e3000 issue6528: implement _is_revision_affected using callback
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47817
diff changeset
567 parent_revs = lambda: fl._revlog.parentrevs(filerev)
5b046c2e3000 issue6528: implement _is_revision_affected using callback
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47817
diff changeset
568 return _is_revision_affected_inner(
5b046c2e3000 issue6528: implement _is_revision_affected using callback
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47817
diff changeset
569 full_text, parent_revs, filerev, metadata_cache
5b046c2e3000 issue6528: implement _is_revision_affected using callback
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47817
diff changeset
570 )
5b046c2e3000 issue6528: implement _is_revision_affected using callback
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47817
diff changeset
571
5b046c2e3000 issue6528: implement _is_revision_affected using callback
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47817
diff changeset
572
5b046c2e3000 issue6528: implement _is_revision_affected using callback
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47817
diff changeset
573 def _is_revision_affected_inner(
5b046c2e3000 issue6528: implement _is_revision_affected using callback
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47817
diff changeset
574 full_text,
5b046c2e3000 issue6528: implement _is_revision_affected using callback
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47817
diff changeset
575 parents_revs,
5b046c2e3000 issue6528: implement _is_revision_affected using callback
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47817
diff changeset
576 filerev,
5b046c2e3000 issue6528: implement _is_revision_affected using callback
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47817
diff changeset
577 metadata_cache=None,
5b046c2e3000 issue6528: implement _is_revision_affected using callback
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47817
diff changeset
578 ):
47815
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
579 """Mercurial currently (5.9rc0) uses `p1 == nullrev and p2 != nullrev` as a
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
580 special meaning compared to the reverse in the context of filelog-based
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
581 copytracing. issue6528 exists because new code assumed that parent ordering
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
582 didn't matter, so this detects if the revision contains metadata (since
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
583 it's only used for filelog-based copytracing) and its parents are in the
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
584 "wrong" order."""
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
585 try:
47818
5b046c2e3000 issue6528: implement _is_revision_affected using callback
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47817
diff changeset
586 raw_text = full_text()
47815
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
587 except error.CensoredNodeError:
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
588 # We don't care about censored nodes as they never carry metadata
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
589 return False
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
590 has_meta = raw_text.startswith(b'\x01\n')
47816
32e21ac3adb1 repair: improve performance of detection of revisions affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47815
diff changeset
591 if metadata_cache is not None:
32e21ac3adb1 repair: improve performance of detection of revisions affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47815
diff changeset
592 metadata_cache[filerev] = has_meta
47815
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
593 if has_meta:
47818
5b046c2e3000 issue6528: implement _is_revision_affected using callback
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47817
diff changeset
594 (p1, p2) = parents_revs()
47815
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
595 if p1 != nullrev and p2 == nullrev:
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
596 return True
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
597 return False
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
598
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
599
47816
32e21ac3adb1 repair: improve performance of detection of revisions affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47815
diff changeset
600 def _is_revision_affected_fast(repo, fl, filerev, metadata_cache):
47819
c02ce6def30c issue6528: implement _is_revision_affected_fast using callback
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47818
diff changeset
601 rl = fl._revlog
c02ce6def30c issue6528: implement _is_revision_affected_fast using callback
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47818
diff changeset
602 is_censored = lambda: rl.iscensored(filerev)
c02ce6def30c issue6528: implement _is_revision_affected_fast using callback
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47818
diff changeset
603 delta_base = lambda: rl.deltaparent(filerev)
c02ce6def30c issue6528: implement _is_revision_affected_fast using callback
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47818
diff changeset
604 delta = lambda: rl._chunk(filerev)
c02ce6def30c issue6528: implement _is_revision_affected_fast using callback
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47818
diff changeset
605 full_text = lambda: rl.rawdata(filerev)
c02ce6def30c issue6528: implement _is_revision_affected_fast using callback
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47818
diff changeset
606 parent_revs = lambda: rl.parentrevs(filerev)
c02ce6def30c issue6528: implement _is_revision_affected_fast using callback
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47818
diff changeset
607 return _is_revision_affected_fast_inner(
c02ce6def30c issue6528: implement _is_revision_affected_fast using callback
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47818
diff changeset
608 is_censored,
c02ce6def30c issue6528: implement _is_revision_affected_fast using callback
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47818
diff changeset
609 delta_base,
c02ce6def30c issue6528: implement _is_revision_affected_fast using callback
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47818
diff changeset
610 delta,
c02ce6def30c issue6528: implement _is_revision_affected_fast using callback
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47818
diff changeset
611 full_text,
c02ce6def30c issue6528: implement _is_revision_affected_fast using callback
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47818
diff changeset
612 parent_revs,
c02ce6def30c issue6528: implement _is_revision_affected_fast using callback
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47818
diff changeset
613 filerev,
c02ce6def30c issue6528: implement _is_revision_affected_fast using callback
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47818
diff changeset
614 metadata_cache,
c02ce6def30c issue6528: implement _is_revision_affected_fast using callback
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47818
diff changeset
615 )
c02ce6def30c issue6528: implement _is_revision_affected_fast using callback
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47818
diff changeset
616
c02ce6def30c issue6528: implement _is_revision_affected_fast using callback
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47818
diff changeset
617
c02ce6def30c issue6528: implement _is_revision_affected_fast using callback
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47818
diff changeset
618 def _is_revision_affected_fast_inner(
c02ce6def30c issue6528: implement _is_revision_affected_fast using callback
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47818
diff changeset
619 is_censored,
c02ce6def30c issue6528: implement _is_revision_affected_fast using callback
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47818
diff changeset
620 delta_base,
c02ce6def30c issue6528: implement _is_revision_affected_fast using callback
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47818
diff changeset
621 delta,
c02ce6def30c issue6528: implement _is_revision_affected_fast using callback
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47818
diff changeset
622 full_text,
c02ce6def30c issue6528: implement _is_revision_affected_fast using callback
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47818
diff changeset
623 parent_revs,
c02ce6def30c issue6528: implement _is_revision_affected_fast using callback
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47818
diff changeset
624 filerev,
c02ce6def30c issue6528: implement _is_revision_affected_fast using callback
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47818
diff changeset
625 metadata_cache,
c02ce6def30c issue6528: implement _is_revision_affected_fast using callback
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47818
diff changeset
626 ):
47816
32e21ac3adb1 repair: improve performance of detection of revisions affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47815
diff changeset
627 """Optimization fast-path for `_is_revision_affected`.
32e21ac3adb1 repair: improve performance of detection of revisions affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47815
diff changeset
628
32e21ac3adb1 repair: improve performance of detection of revisions affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47815
diff changeset
629 `metadata_cache` is a dict of `{rev: has_metadata}` which allows any
32e21ac3adb1 repair: improve performance of detection of revisions affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47815
diff changeset
630 revision to check if its base has metadata, saving computation of the full
32e21ac3adb1 repair: improve performance of detection of revisions affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47815
diff changeset
631 text, instead looking at the current delta.
32e21ac3adb1 repair: improve performance of detection of revisions affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47815
diff changeset
632
32e21ac3adb1 repair: improve performance of detection of revisions affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47815
diff changeset
633 This optimization only works if the revisions are looked at in order."""
32e21ac3adb1 repair: improve performance of detection of revisions affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47815
diff changeset
634
47819
c02ce6def30c issue6528: implement _is_revision_affected_fast using callback
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47818
diff changeset
635 if is_censored():
47816
32e21ac3adb1 repair: improve performance of detection of revisions affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47815
diff changeset
636 # Censored revisions don't contain metadata, so they cannot be affected
32e21ac3adb1 repair: improve performance of detection of revisions affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47815
diff changeset
637 metadata_cache[filerev] = False
32e21ac3adb1 repair: improve performance of detection of revisions affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47815
diff changeset
638 return False
32e21ac3adb1 repair: improve performance of detection of revisions affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47815
diff changeset
639
47819
c02ce6def30c issue6528: implement _is_revision_affected_fast using callback
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47818
diff changeset
640 p1, p2 = parent_revs()
47816
32e21ac3adb1 repair: improve performance of detection of revisions affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47815
diff changeset
641 if p1 == nullrev or p2 != nullrev:
32e21ac3adb1 repair: improve performance of detection of revisions affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47815
diff changeset
642 return False
32e21ac3adb1 repair: improve performance of detection of revisions affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47815
diff changeset
643
47819
c02ce6def30c issue6528: implement _is_revision_affected_fast using callback
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47818
diff changeset
644 delta_parent = delta_base()
47816
32e21ac3adb1 repair: improve performance of detection of revisions affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47815
diff changeset
645 parent_has_metadata = metadata_cache.get(delta_parent)
32e21ac3adb1 repair: improve performance of detection of revisions affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47815
diff changeset
646 if parent_has_metadata is None:
47819
c02ce6def30c issue6528: implement _is_revision_affected_fast using callback
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47818
diff changeset
647 return _is_revision_affected_inner(
c02ce6def30c issue6528: implement _is_revision_affected_fast using callback
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47818
diff changeset
648 full_text,
c02ce6def30c issue6528: implement _is_revision_affected_fast using callback
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47818
diff changeset
649 parent_revs,
c02ce6def30c issue6528: implement _is_revision_affected_fast using callback
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47818
diff changeset
650 filerev,
c02ce6def30c issue6528: implement _is_revision_affected_fast using callback
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47818
diff changeset
651 metadata_cache,
c02ce6def30c issue6528: implement _is_revision_affected_fast using callback
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47818
diff changeset
652 )
47816
32e21ac3adb1 repair: improve performance of detection of revisions affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47815
diff changeset
653
47819
c02ce6def30c issue6528: implement _is_revision_affected_fast using callback
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47818
diff changeset
654 chunk = delta()
47816
32e21ac3adb1 repair: improve performance of detection of revisions affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47815
diff changeset
655 if not len(chunk):
32e21ac3adb1 repair: improve performance of detection of revisions affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47815
diff changeset
656 # No diff for this revision
32e21ac3adb1 repair: improve performance of detection of revisions affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47815
diff changeset
657 return parent_has_metadata
32e21ac3adb1 repair: improve performance of detection of revisions affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47815
diff changeset
658
32e21ac3adb1 repair: improve performance of detection of revisions affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47815
diff changeset
659 header_length = 12
32e21ac3adb1 repair: improve performance of detection of revisions affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47815
diff changeset
660 if len(chunk) < header_length:
32e21ac3adb1 repair: improve performance of detection of revisions affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47815
diff changeset
661 raise error.Abort(_(b"patch cannot be decoded"))
32e21ac3adb1 repair: improve performance of detection of revisions affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47815
diff changeset
662
32e21ac3adb1 repair: improve performance of detection of revisions affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47815
diff changeset
663 start, _end, _length = struct.unpack(b">lll", chunk[:header_length])
32e21ac3adb1 repair: improve performance of detection of revisions affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47815
diff changeset
664
32e21ac3adb1 repair: improve performance of detection of revisions affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47815
diff changeset
665 if start < 2: # len(b'\x01\n') == 2
32e21ac3adb1 repair: improve performance of detection of revisions affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47815
diff changeset
666 # This delta does *something* to the metadata marker (if any).
32e21ac3adb1 repair: improve performance of detection of revisions affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47815
diff changeset
667 # Check it the slow way
47819
c02ce6def30c issue6528: implement _is_revision_affected_fast using callback
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47818
diff changeset
668 is_affected = _is_revision_affected_inner(
c02ce6def30c issue6528: implement _is_revision_affected_fast using callback
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47818
diff changeset
669 full_text,
c02ce6def30c issue6528: implement _is_revision_affected_fast using callback
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47818
diff changeset
670 parent_revs,
c02ce6def30c issue6528: implement _is_revision_affected_fast using callback
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47818
diff changeset
671 filerev,
c02ce6def30c issue6528: implement _is_revision_affected_fast using callback
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47818
diff changeset
672 metadata_cache,
c02ce6def30c issue6528: implement _is_revision_affected_fast using callback
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47818
diff changeset
673 )
47816
32e21ac3adb1 repair: improve performance of detection of revisions affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47815
diff changeset
674 return is_affected
32e21ac3adb1 repair: improve performance of detection of revisions affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47815
diff changeset
675
32e21ac3adb1 repair: improve performance of detection of revisions affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47815
diff changeset
676 # The diff did not remove or add the metadata header, it's then in the same
32e21ac3adb1 repair: improve performance of detection of revisions affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47815
diff changeset
677 # situation as its parent
32e21ac3adb1 repair: improve performance of detection of revisions affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47815
diff changeset
678 metadata_cache[filerev] = parent_has_metadata
32e21ac3adb1 repair: improve performance of detection of revisions affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47815
diff changeset
679 return parent_has_metadata
32e21ac3adb1 repair: improve performance of detection of revisions affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47815
diff changeset
680
32e21ac3adb1 repair: improve performance of detection of revisions affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47815
diff changeset
681
47815
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
682 def _from_report(ui, repo, context, from_report, dry_run):
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
683 """
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
684 Fix the revisions given in the `from_report` file, but still checks if the
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
685 revisions are indeed affected to prevent an unfortunate cyclic situation
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
686 where we'd swap well-ordered parents again.
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
687
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
688 See the doc for `debug_fix_issue6528` for the format documentation.
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
689 """
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
690 ui.write(_(b"loading report file '%s'\n") % from_report)
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
691
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
692 with context(), open(from_report, mode='rb') as f:
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
693 for line in f.read().split(b'\n'):
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
694 if not line:
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
695 continue
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
696 filenodes, filename = line.split(b' ', 1)
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
697 fl = _filelog_from_filename(repo, filename)
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
698 to_fix = set(
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
699 fl.rev(binascii.unhexlify(n)) for n in filenodes.split(b',')
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
700 )
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
701 excluded = set()
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
702
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
703 for filerev in to_fix:
47816
32e21ac3adb1 repair: improve performance of detection of revisions affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47815
diff changeset
704 if _is_revision_affected(fl, filerev):
47815
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
705 msg = b"found affected revision %d for filelog '%s'\n"
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
706 ui.warn(msg % (filerev, filename))
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
707 else:
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
708 msg = _(b"revision %s of file '%s' is not affected\n")
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
709 msg %= (binascii.hexlify(fl.node(filerev)), filename)
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
710 ui.warn(msg)
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
711 excluded.add(filerev)
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
712
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
713 to_fix = to_fix - excluded
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
714 if not to_fix:
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
715 msg = _(b"no affected revisions were found for '%s'\n")
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
716 ui.write(msg % filename)
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
717 continue
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
718 if not dry_run:
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
719 _reorder_filelog_parents(repo, fl, sorted(to_fix))
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
720
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
721
47817
855463b5fe49 debugcommands: add a `--paranoid` option to `debug-repair-issue-6528`
Raphaël Gomès <rgomes@octobus.net>
parents: 47816
diff changeset
722 def repair_issue6528(
855463b5fe49 debugcommands: add a `--paranoid` option to `debug-repair-issue-6528`
Raphaël Gomès <rgomes@octobus.net>
parents: 47816
diff changeset
723 ui, repo, dry_run=False, to_report=None, from_report=None, paranoid=False
855463b5fe49 debugcommands: add a `--paranoid` option to `debug-repair-issue-6528`
Raphaël Gomès <rgomes@octobus.net>
parents: 47816
diff changeset
724 ):
47815
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
725 from .. import store # avoid cycle
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
726
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
727 @contextlib.contextmanager
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
728 def context():
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
729 if dry_run or to_report: # No need for locking
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
730 yield
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
731 else:
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
732 with repo.wlock(), repo.lock():
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
733 yield
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
734
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
735 if from_report:
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
736 return _from_report(ui, repo, context, from_report, dry_run)
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
737
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
738 report_entries = []
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
739
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
740 with context():
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
741 files = list(
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
742 (file_type, path)
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
743 for (file_type, path, _e, _s) in repo.store.datafiles()
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
744 if path.endswith(b'.i') and file_type & store.FILEFLAGS_FILELOG
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
745 )
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
746
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
747 progress = ui.makeprogress(
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
748 _(b"looking for affected revisions"),
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
749 unit=_(b"filelogs"),
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
750 total=len(files),
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
751 )
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
752 found_nothing = True
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
753
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
754 for file_type, path in files:
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
755 if (
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
756 not path.endswith(b'.i')
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
757 or not file_type & store.FILEFLAGS_FILELOG
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
758 ):
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
759 continue
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
760 progress.increment()
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
761 filename = _get_filename_from_filelog_index(path)
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
762 fl = _filelog_from_filename(repo, filename)
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
763
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
764 # Set of filerevs (or hex filenodes if `to_report`) that need fixing
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
765 to_fix = set()
47816
32e21ac3adb1 repair: improve performance of detection of revisions affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47815
diff changeset
766 metadata_cache = {}
47815
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
767 for filerev in fl.revs():
47816
32e21ac3adb1 repair: improve performance of detection of revisions affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47815
diff changeset
768 affected = _is_revision_affected_fast(
32e21ac3adb1 repair: improve performance of detection of revisions affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47815
diff changeset
769 repo, fl, filerev, metadata_cache
32e21ac3adb1 repair: improve performance of detection of revisions affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47815
diff changeset
770 )
47817
855463b5fe49 debugcommands: add a `--paranoid` option to `debug-repair-issue-6528`
Raphaël Gomès <rgomes@octobus.net>
parents: 47816
diff changeset
771 if paranoid:
855463b5fe49 debugcommands: add a `--paranoid` option to `debug-repair-issue-6528`
Raphaël Gomès <rgomes@octobus.net>
parents: 47816
diff changeset
772 slow = _is_revision_affected(fl, filerev)
855463b5fe49 debugcommands: add a `--paranoid` option to `debug-repair-issue-6528`
Raphaël Gomès <rgomes@octobus.net>
parents: 47816
diff changeset
773 if slow != affected:
855463b5fe49 debugcommands: add a `--paranoid` option to `debug-repair-issue-6528`
Raphaël Gomès <rgomes@octobus.net>
parents: 47816
diff changeset
774 msg = _(b"paranoid check failed for '%s' at node %s")
855463b5fe49 debugcommands: add a `--paranoid` option to `debug-repair-issue-6528`
Raphaël Gomès <rgomes@octobus.net>
parents: 47816
diff changeset
775 node = binascii.hexlify(fl.node(filerev))
855463b5fe49 debugcommands: add a `--paranoid` option to `debug-repair-issue-6528`
Raphaël Gomès <rgomes@octobus.net>
parents: 47816
diff changeset
776 raise error.Abort(msg % (filename, node))
47815
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
777 if affected:
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
778 msg = b"found affected revision %d for filelog '%s'\n"
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
779 ui.warn(msg % (filerev, path))
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
780 found_nothing = False
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
781 if not dry_run:
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
782 if to_report:
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
783 to_fix.add(binascii.hexlify(fl.node(filerev)))
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
784 else:
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
785 to_fix.add(filerev)
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
786
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
787 if to_fix:
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
788 to_fix = sorted(to_fix)
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
789 if to_report:
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
790 report_entries.append((filename, to_fix))
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
791 else:
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
792 _reorder_filelog_parents(repo, fl, to_fix)
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
793
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
794 if found_nothing:
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
795 ui.write(_(b"no affected revisions were found\n"))
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
796
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
797 if to_report and report_entries:
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
798 with open(to_report, mode="wb") as f:
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
799 for path, to_fix in report_entries:
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
800 f.write(b"%s %s\n" % (b",".join(to_fix), path))
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
801
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528
Raphaël Gomès <rgomes@octobus.net>
parents: 47473
diff changeset
802 progress.complete()