annotate mercurial/revlogutils/deltas.py @ 51323:87fce139e927

delta-find: move snapshot_cache in the _DeltaSearch.__init__ Now that we have an object we can initialize that attribute at initialization time.
author Pierre-Yves David <pierre-yves.david@octobus.net>
date Mon, 20 Nov 2023 05:03:21 +0100
parents a227e061bb4b
children 047358417d01
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
39330
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
1 # revlogdeltas.py - Logic around delta computation for revlog
8226
8b2cd04a6e97 put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents: 8225
diff changeset
2 #
46819
d4ba4d51f85f contributor: change mentions of mpm to olivia
Raphaël Gomès <rgomes@octobus.net>
parents: 43089
diff changeset
3 # Copyright 2005-2007 Olivia Mackall <olivia@selenic.com>
39330
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
4 # Copyright 2018 Octobus <contact@octobus.net>
8226
8b2cd04a6e97 put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents: 8225
diff changeset
5 #
8b2cd04a6e97 put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents: 8225
diff changeset
6 # This software may be used and distributed according to the terms of the
10263
25e572394f5c Update license to GPLv2+
Matt Mackall <mpm@selenic.com>
parents: 10047
diff changeset
7 # GNU General Public License version 2 or any later version.
39330
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
8 """Helper class to compute deltas stored inside revlogs"""
8227
0a9542703300 turn some comments back into module docstrings
Martin Geisler <mg@lazybytes.net>
parents: 8226
diff changeset
9
27361
29f50344fa83 revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27251
diff changeset
10
39493
3ca144f1c8dd snapshot: search for unrelated but reusable full-snapshot
Boris Feld <boris.feld@octobus.net>
parents: 39492
diff changeset
11 import collections
27361
29f50344fa83 revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27251
diff changeset
12 import struct
29f50344fa83 revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27251
diff changeset
13
29f50344fa83 revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27251
diff changeset
14 # import stuff from node for others to import from revlog
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
15 from ..node import nullrev
39330
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
16 from ..i18n import _
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
17
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
18 from .constants import (
47452
c6844912c327 revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47401
diff changeset
19 COMP_MODE_DEFAULT,
c6844912c327 revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47401
diff changeset
20 COMP_MODE_INLINE,
c6844912c327 revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47401
diff changeset
21 COMP_MODE_PLAIN,
49768
bcae90c53def delta-find: add a delta-reuse policy that blindly accepts incoming deltas
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49681
diff changeset
22 DELTA_BASE_REUSE_FORCE,
49677
05db41701ece find-delta: pass the cache-delta usage policy alongside the cache-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49675
diff changeset
23 DELTA_BASE_REUSE_NO,
49226
e6b7c6fbeb48 deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48946
diff changeset
24 KIND_CHANGELOG,
e6b7c6fbeb48 deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48946
diff changeset
25 KIND_FILELOG,
e6b7c6fbeb48 deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48946
diff changeset
26 KIND_MANIFESTLOG,
39329
729082bb9938 revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents: 39232
diff changeset
27 REVIDX_ISCENSORED,
729082bb9938 revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents: 39232
diff changeset
28 REVIDX_RAWTEXT_CHANGING_FLAGS,
729082bb9938 revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents: 39232
diff changeset
29 )
39330
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
30
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
31 from ..thirdparty import attr
39330
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
32
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
33 from .. import (
27361
29f50344fa83 revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27251
diff changeset
34 error,
29f50344fa83 revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27251
diff changeset
35 mdiff,
41108
38e88450138c delta: have a native implementation of _findsnapshot
Boris Feld <boris.feld@octobus.net>
parents: 41079
diff changeset
36 util,
27361
29f50344fa83 revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27251
diff changeset
37 )
10913
f2ecc5733c89 revlog: factor out _maxinline global.
Greg Ward <greg-hg@gerg.ca>
parents: 10404
diff changeset
38
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
39 from . import flagutil
42992
dff95420480f flagprocessors: make `processflagsraw` a module level function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42878
diff changeset
40
39330
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
41 # maximum <delta-chain-data>/<revision-text-length> ratio
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
42 LIMIT_DELTA2TEXT = 2
1091
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
43
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
44
48946
642e31cb55f0 py3: use class X: instead of class X(object):
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48875
diff changeset
45 class _testrevlog:
38637
e33f784f2a44 revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents: 38636
diff changeset
46 """minimalist fake revlog to use in doctests"""
e33f784f2a44 revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents: 38636
diff changeset
47
40641
85b14f0dc334 doctest: add a `issnapshot` method to _testrevlog
Boris Feld <boris.feld@octobus.net>
parents: 40608
diff changeset
48 def __init__(self, data, density=0.5, mingap=0, snapshot=()):
38637
e33f784f2a44 revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents: 38636
diff changeset
49 """data is an list of revision payload boundaries"""
51045
127656e0b97b revlog: use the new Config classes in _testrevlog
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51020
diff changeset
50 from .. import revlog
127656e0b97b revlog: use the new Config classes in _testrevlog
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51020
diff changeset
51
38637
e33f784f2a44 revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents: 38636
diff changeset
52 self._data = data
51045
127656e0b97b revlog: use the new Config classes in _testrevlog
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51020
diff changeset
53 self.data_config = revlog.DataConfig()
127656e0b97b revlog: use the new Config classes in _testrevlog
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51020
diff changeset
54 self.data_config.sr_density_threshold = density
127656e0b97b revlog: use the new Config classes in _testrevlog
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51020
diff changeset
55 self.data_config.sr_min_gap_size = mingap
127656e0b97b revlog: use the new Config classes in _testrevlog
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51020
diff changeset
56 self.delta_config = revlog.DeltaConfig()
127656e0b97b revlog: use the new Config classes in _testrevlog
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51020
diff changeset
57 self.feature_config = revlog.FeatureConfig()
40641
85b14f0dc334 doctest: add a `issnapshot` method to _testrevlog
Boris Feld <boris.feld@octobus.net>
parents: 40608
diff changeset
58 self._snapshot = set(snapshot)
40709
39d29542fe40 sparse-revlog: put the native implementation of slicechunktodensity to use
Boris Feld <boris.feld@octobus.net>
parents: 40657
diff changeset
59 self.index = None
38637
e33f784f2a44 revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents: 38636
diff changeset
60
e33f784f2a44 revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents: 38636
diff changeset
61 def start(self, rev):
41079
88d813cd9acd revlog: fix pure python slicing test when chain contains nullrev
Boris Feld <boris.feld@octobus.net>
parents: 41033
diff changeset
62 if rev == nullrev:
88d813cd9acd revlog: fix pure python slicing test when chain contains nullrev
Boris Feld <boris.feld@octobus.net>
parents: 41033
diff changeset
63 return 0
38637
e33f784f2a44 revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents: 38636
diff changeset
64 if rev == 0:
e33f784f2a44 revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents: 38636
diff changeset
65 return 0
e33f784f2a44 revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents: 38636
diff changeset
66 return self._data[rev - 1]
e33f784f2a44 revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents: 38636
diff changeset
67
e33f784f2a44 revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents: 38636
diff changeset
68 def end(self, rev):
41079
88d813cd9acd revlog: fix pure python slicing test when chain contains nullrev
Boris Feld <boris.feld@octobus.net>
parents: 41033
diff changeset
69 if rev == nullrev:
88d813cd9acd revlog: fix pure python slicing test when chain contains nullrev
Boris Feld <boris.feld@octobus.net>
parents: 41033
diff changeset
70 return 0
38637
e33f784f2a44 revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents: 38636
diff changeset
71 return self._data[rev]
e33f784f2a44 revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents: 38636
diff changeset
72
e33f784f2a44 revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents: 38636
diff changeset
73 def length(self, rev):
e33f784f2a44 revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents: 38636
diff changeset
74 return self.end(rev) - self.start(rev)
e33f784f2a44 revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents: 38636
diff changeset
75
38718
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents: 38717
diff changeset
76 def __len__(self):
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents: 38717
diff changeset
77 return len(self._data)
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents: 38717
diff changeset
78
40641
85b14f0dc334 doctest: add a `issnapshot` method to _testrevlog
Boris Feld <boris.feld@octobus.net>
parents: 40608
diff changeset
79 def issnapshot(self, rev):
41079
88d813cd9acd revlog: fix pure python slicing test when chain contains nullrev
Boris Feld <boris.feld@octobus.net>
parents: 41033
diff changeset
80 if rev == nullrev:
88d813cd9acd revlog: fix pure python slicing test when chain contains nullrev
Boris Feld <boris.feld@octobus.net>
parents: 41033
diff changeset
81 return True
40641
85b14f0dc334 doctest: add a `issnapshot` method to _testrevlog
Boris Feld <boris.feld@octobus.net>
parents: 40608
diff changeset
82 return rev in self._snapshot
85b14f0dc334 doctest: add a `issnapshot` method to _testrevlog
Boris Feld <boris.feld@octobus.net>
parents: 40608
diff changeset
83
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
84
40604
3ac23dad6364 sparse-revlog: drop unused deltainfo parameter from _slicechunktodensity
Boris Feld <boris.feld@octobus.net>
parents: 40603
diff changeset
85 def slicechunk(revlog, revs, targetsize=None):
34824
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
86 """slice revs to reduce the amount of unrelated data to be read from disk.
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
87
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
88 ``revs`` is sliced into groups that should be read in one time.
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
89 Assume that revs are sorted.
38640
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
90
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
91 The initial chunk is sliced until the overall density (payload/chunks-span
51062
8c614fa16330 revlog: remove legacy usage of `_srdensitythreshold`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51060
diff changeset
92 ratio) is above `revlog.data_config.sr_density_threshold`. No gap smaller
51063
533d6943f6a3 revlog: remove legacy usage of `_srmingapsize`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51062
diff changeset
93 than `revlog.data_config.sr_min_gap_size` is skipped.
38640
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
94
38643
967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents: 38642
diff changeset
95 If `targetsize` is set, no chunk larger than `targetsize` will be yield.
967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents: 38642
diff changeset
96 For consistency with other slicing choice, this limit won't go lower than
51063
533d6943f6a3 revlog: remove legacy usage of `_srmingapsize`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51062
diff changeset
97 `revlog.data_config.sr_min_gap_size`.
38643
967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents: 38642
diff changeset
98
967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents: 38642
diff changeset
99 If individual revisions chunk are larger than this limit, they will still
967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents: 38642
diff changeset
100 be raised individually.
967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents: 38642
diff changeset
101
40642
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
102 >>> data = [
38640
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
103 ... 5, #00 (5)
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
104 ... 10, #01 (5)
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
105 ... 12, #02 (2)
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
106 ... 12, #03 (empty)
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
107 ... 27, #04 (15)
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
108 ... 31, #05 (4)
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
109 ... 31, #06 (empty)
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
110 ... 42, #07 (11)
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
111 ... 47, #08 (5)
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
112 ... 47, #09 (empty)
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
113 ... 48, #10 (1)
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
114 ... 51, #11 (3)
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
115 ... 74, #12 (23)
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
116 ... 85, #13 (11)
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
117 ... 86, #14 (1)
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
118 ... 91, #15 (5)
40642
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
119 ... ]
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
120 >>> revlog = _testrevlog(data, snapshot=range(16))
38640
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
121
39330
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
122 >>> list(slicechunk(revlog, list(range(16))))
38640
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
123 [[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]]
39330
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
124 >>> list(slicechunk(revlog, [0, 15]))
38640
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
125 [[0], [15]]
39330
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
126 >>> list(slicechunk(revlog, [0, 11, 15]))
38640
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
127 [[0], [11], [15]]
39330
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
128 >>> list(slicechunk(revlog, [0, 11, 13, 15]))
38640
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
129 [[0], [11, 13, 15]]
39330
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
130 >>> list(slicechunk(revlog, [1, 2, 3, 5, 8, 10, 11, 14]))
38640
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
131 [[1, 2], [5, 8, 10, 11], [14]]
38643
967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents: 38642
diff changeset
132
967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents: 38642
diff changeset
133 Slicing with a maximum chunk size
39330
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
134 >>> list(slicechunk(revlog, [0, 11, 13, 15], targetsize=15))
38643
967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents: 38642
diff changeset
135 [[0], [11], [13], [15]]
39330
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
136 >>> list(slicechunk(revlog, [0, 11, 13, 15], targetsize=20))
38643
967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents: 38642
diff changeset
137 [[0], [11], [13, 15]]
41079
88d813cd9acd revlog: fix pure python slicing test when chain contains nullrev
Boris Feld <boris.feld@octobus.net>
parents: 41033
diff changeset
138
88d813cd9acd revlog: fix pure python slicing test when chain contains nullrev
Boris Feld <boris.feld@octobus.net>
parents: 41033
diff changeset
139 Slicing involving nullrev
88d813cd9acd revlog: fix pure python slicing test when chain contains nullrev
Boris Feld <boris.feld@octobus.net>
parents: 41033
diff changeset
140 >>> list(slicechunk(revlog, [-1, 0, 11, 13, 15], targetsize=20))
88d813cd9acd revlog: fix pure python slicing test when chain contains nullrev
Boris Feld <boris.feld@octobus.net>
parents: 41033
diff changeset
141 [[-1, 0], [11], [13, 15]]
88d813cd9acd revlog: fix pure python slicing test when chain contains nullrev
Boris Feld <boris.feld@octobus.net>
parents: 41033
diff changeset
142 >>> list(slicechunk(revlog, [-1, 13, 15], targetsize=5))
88d813cd9acd revlog: fix pure python slicing test when chain contains nullrev
Boris Feld <boris.feld@octobus.net>
parents: 41033
diff changeset
143 [[-1], [13], [15]]
34824
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
144 """
38643
967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents: 38642
diff changeset
145 if targetsize is not None:
51063
533d6943f6a3 revlog: remove legacy usage of `_srmingapsize`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51062
diff changeset
146 targetsize = max(targetsize, revlog.data_config.sr_min_gap_size)
38718
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents: 38717
diff changeset
147 # targetsize should not be specified when evaluating delta candidates:
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents: 38717
diff changeset
148 # * targetsize is used to ensure we stay within specification when reading,
40709
39d29542fe40 sparse-revlog: put the native implementation of slicechunktodensity to use
Boris Feld <boris.feld@octobus.net>
parents: 40657
diff changeset
149 densityslicing = getattr(revlog.index, 'slicechunktodensity', None)
39d29542fe40 sparse-revlog: put the native implementation of slicechunktodensity to use
Boris Feld <boris.feld@octobus.net>
parents: 40657
diff changeset
150 if densityslicing is None:
39d29542fe40 sparse-revlog: put the native implementation of slicechunktodensity to use
Boris Feld <boris.feld@octobus.net>
parents: 40657
diff changeset
151 densityslicing = lambda x, y, z: _slicechunktodensity(revlog, x, y, z)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
152 for chunk in densityslicing(
51063
533d6943f6a3 revlog: remove legacy usage of `_srmingapsize`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51062
diff changeset
153 revs,
533d6943f6a3 revlog: remove legacy usage of `_srmingapsize`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51062
diff changeset
154 revlog.data_config.sr_density_threshold,
533d6943f6a3 revlog: remove legacy usage of `_srmingapsize`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51062
diff changeset
155 revlog.data_config.sr_min_gap_size,
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
156 ):
38643
967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents: 38642
diff changeset
157 for subchunk in _slicechunktosize(revlog, chunk, targetsize):
967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents: 38642
diff changeset
158 yield subchunk
38641
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
159
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
160
38718
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents: 38717
diff changeset
161 def _slicechunktosize(revlog, revs, targetsize=None):
38642
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
162 """slice revs to match the target size
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
163
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
164 This is intended to be used on chunk that density slicing selected by that
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
165 are still too large compared to the read garantee of revlog. This might
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
166 happens when "minimal gap size" interrupted the slicing or when chain are
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
167 built in a way that create large blocks next to each other.
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
168
40642
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
169 >>> data = [
38642
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
170 ... 3, #0 (3)
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
171 ... 5, #1 (2)
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
172 ... 6, #2 (1)
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
173 ... 8, #3 (2)
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
174 ... 8, #4 (empty)
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
175 ... 11, #5 (3)
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
176 ... 12, #6 (1)
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
177 ... 13, #7 (1)
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
178 ... 14, #8 (1)
40642
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
179 ... ]
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
180
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
181 == All snapshots cases ==
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
182 >>> revlog = _testrevlog(data, snapshot=range(9))
38642
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
183
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
184 Cases where chunk is already small enough
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
185 >>> list(_slicechunktosize(revlog, [0], 3))
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
186 [[0]]
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
187 >>> list(_slicechunktosize(revlog, [6, 7], 3))
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
188 [[6, 7]]
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
189 >>> list(_slicechunktosize(revlog, [0], None))
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
190 [[0]]
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
191 >>> list(_slicechunktosize(revlog, [6, 7], None))
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
192 [[6, 7]]
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
193
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
194 cases where we need actual slicing
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
195 >>> list(_slicechunktosize(revlog, [0, 1], 3))
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
196 [[0], [1]]
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
197 >>> list(_slicechunktosize(revlog, [1, 3], 3))
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
198 [[1], [3]]
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
199 >>> list(_slicechunktosize(revlog, [1, 2, 3], 3))
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
200 [[1, 2], [3]]
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
201 >>> list(_slicechunktosize(revlog, [3, 5], 3))
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
202 [[3], [5]]
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
203 >>> list(_slicechunktosize(revlog, [3, 4, 5], 3))
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
204 [[3], [5]]
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
205 >>> list(_slicechunktosize(revlog, [5, 6, 7, 8], 3))
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
206 [[5], [6, 7, 8]]
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
207 >>> list(_slicechunktosize(revlog, [0, 1, 2, 3, 4, 5, 6, 7, 8], 3))
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
208 [[0], [1, 2], [3], [5], [6, 7, 8]]
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
209
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
210 Case with too large individual chunk (must return valid chunk)
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
211 >>> list(_slicechunktosize(revlog, [0, 1], 2))
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
212 [[0], [1]]
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
213 >>> list(_slicechunktosize(revlog, [1, 3], 1))
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
214 [[1], [3]]
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
215 >>> list(_slicechunktosize(revlog, [3, 4, 5], 2))
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
216 [[3], [5]]
40642
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
217
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
218 == No Snapshot cases ==
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
219 >>> revlog = _testrevlog(data)
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
220
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
221 Cases where chunk is already small enough
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
222 >>> list(_slicechunktosize(revlog, [0], 3))
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
223 [[0]]
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
224 >>> list(_slicechunktosize(revlog, [6, 7], 3))
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
225 [[6, 7]]
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
226 >>> list(_slicechunktosize(revlog, [0], None))
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
227 [[0]]
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
228 >>> list(_slicechunktosize(revlog, [6, 7], None))
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
229 [[6, 7]]
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
230
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
231 cases where we need actual slicing
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
232 >>> list(_slicechunktosize(revlog, [0, 1], 3))
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
233 [[0], [1]]
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
234 >>> list(_slicechunktosize(revlog, [1, 3], 3))
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
235 [[1], [3]]
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
236 >>> list(_slicechunktosize(revlog, [1, 2, 3], 3))
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
237 [[1], [2, 3]]
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
238 >>> list(_slicechunktosize(revlog, [3, 5], 3))
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
239 [[3], [5]]
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
240 >>> list(_slicechunktosize(revlog, [3, 4, 5], 3))
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
241 [[3], [4, 5]]
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
242 >>> list(_slicechunktosize(revlog, [5, 6, 7, 8], 3))
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
243 [[5], [6, 7, 8]]
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
244 >>> list(_slicechunktosize(revlog, [0, 1, 2, 3, 4, 5, 6, 7, 8], 3))
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
245 [[0], [1, 2], [3], [5], [6, 7, 8]]
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
246
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
247 Case with too large individual chunk (must return valid chunk)
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
248 >>> list(_slicechunktosize(revlog, [0, 1], 2))
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
249 [[0], [1]]
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
250 >>> list(_slicechunktosize(revlog, [1, 3], 1))
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
251 [[1], [3]]
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
252 >>> list(_slicechunktosize(revlog, [3, 4, 5], 2))
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
253 [[3], [5]]
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
254
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
255 == mixed case ==
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
256 >>> revlog = _testrevlog(data, snapshot=[0, 1, 2])
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
257 >>> list(_slicechunktosize(revlog, list(range(9)), 5))
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
258 [[0, 1], [2], [3, 4, 5], [6, 7, 8]]
38642
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
259 """
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
260 assert targetsize is None or 0 <= targetsize
40642
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
261 startdata = revlog.start(revs[0])
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
262 enddata = revlog.end(revs[-1])
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
263 fullspan = enddata - startdata
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
264 if targetsize is None or fullspan <= targetsize:
38642
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
265 yield revs
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
266 return
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
267
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
268 startrevidx = 0
40657
2eb48aa0acce sparse-revlog: align endrevidx usages in the _slicechunktosize
Boris Feld <boris.feld@octobus.net>
parents: 40654
diff changeset
269 endrevidx = 1
38642
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
270 iterrevs = enumerate(revs)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
271 next(iterrevs) # skip first rev.
40642
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
272 # first step: get snapshots out of the way
38642
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
273 for idx, r in iterrevs:
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
274 span = revlog.end(r) - startdata
40642
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
275 snapshot = revlog.issnapshot(r)
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
276 if span <= targetsize and snapshot:
40657
2eb48aa0acce sparse-revlog: align endrevidx usages in the _slicechunktosize
Boris Feld <boris.feld@octobus.net>
parents: 40654
diff changeset
277 endrevidx = idx + 1
38642
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
278 else:
40657
2eb48aa0acce sparse-revlog: align endrevidx usages in the _slicechunktosize
Boris Feld <boris.feld@octobus.net>
parents: 40654
diff changeset
279 chunk = _trimchunk(revlog, revs, startrevidx, endrevidx)
38642
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
280 if chunk:
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
281 yield chunk
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
282 startrevidx = idx
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
283 startdata = revlog.start(r)
40657
2eb48aa0acce sparse-revlog: align endrevidx usages in the _slicechunktosize
Boris Feld <boris.feld@octobus.net>
parents: 40654
diff changeset
284 endrevidx = idx + 1
40642
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
285 if not snapshot:
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
286 break
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
287
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
288 # for the others, we use binary slicing to quickly converge toward valid
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
289 # chunks (otherwise, we might end up looking for start/end of many
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
290 # revisions). This logic is not looking for the perfect slicing point, it
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
291 # focuses on quickly converging toward valid chunks.
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
292 nbitem = len(revs)
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
293 while (enddata - startdata) > targetsize:
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
294 endrevidx = nbitem
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
295 if nbitem - startrevidx <= 1:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
296 break # protect against individual chunk larger than limit
40642
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
297 localenddata = revlog.end(revs[endrevidx - 1])
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
298 span = localenddata - startdata
40654
fd1d41ccbe38 sparse-revlog: use `span` variable as intended
Boris Feld <boris.feld@octobus.net>
parents: 40642
diff changeset
299 while span > targetsize:
40642
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
300 if endrevidx - startrevidx <= 1:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
301 break # protect against individual chunk larger than limit
40642
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
302 endrevidx -= (endrevidx - startrevidx) // 2
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
303 localenddata = revlog.end(revs[endrevidx - 1])
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
304 span = localenddata - startdata
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
305 chunk = _trimchunk(revlog, revs, startrevidx, endrevidx)
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
306 if chunk:
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
307 yield chunk
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
308 startrevidx = endrevidx
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
309 startdata = revlog.start(revs[startrevidx])
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
310
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
311 chunk = _trimchunk(revlog, revs, startrevidx)
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
312 if chunk:
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents: 40641
diff changeset
313 yield chunk
38642
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
314
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
315
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
316 def _slicechunktodensity(revlog, revs, targetdensity=0.5, mingapsize=0):
38641
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
317 """slice revs to reduce the amount of unrelated data to be read from disk.
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
318
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
319 ``revs`` is sliced into groups that should be read in one time.
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
320 Assume that revs are sorted.
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
321
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
322 The initial chunk is sliced until the overall density (payload/chunks-span
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
323 ratio) is above `targetdensity`. No gap smaller than `mingapsize` is
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
324 skipped.
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
325
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
326 >>> revlog = _testrevlog([
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
327 ... 5, #00 (5)
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
328 ... 10, #01 (5)
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
329 ... 12, #02 (2)
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
330 ... 12, #03 (empty)
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
331 ... 27, #04 (15)
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
332 ... 31, #05 (4)
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
333 ... 31, #06 (empty)
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
334 ... 42, #07 (11)
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
335 ... 47, #08 (5)
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
336 ... 47, #09 (empty)
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
337 ... 48, #10 (1)
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
338 ... 51, #11 (3)
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
339 ... 74, #12 (23)
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
340 ... 85, #13 (11)
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
341 ... 86, #14 (1)
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
342 ... 91, #15 (5)
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
343 ... ])
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
344
38655
cd1c484e31e8 revlog: adjust doctest examples to be portable to Python 3
Augie Fackler <augie@google.com>
parents: 38644
diff changeset
345 >>> list(_slicechunktodensity(revlog, list(range(16))))
38641
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
346 [[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]]
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
347 >>> list(_slicechunktodensity(revlog, [0, 15]))
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
348 [[0], [15]]
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
349 >>> list(_slicechunktodensity(revlog, [0, 11, 15]))
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
350 [[0], [11], [15]]
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
351 >>> list(_slicechunktodensity(revlog, [0, 11, 13, 15]))
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
352 [[0], [11, 13, 15]]
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
353 >>> list(_slicechunktodensity(revlog, [1, 2, 3, 5, 8, 10, 11, 14]))
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
354 [[1, 2], [5, 8, 10, 11], [14]]
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
355 >>> list(_slicechunktodensity(revlog, [1, 2, 3, 5, 8, 10, 11, 14],
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
356 ... mingapsize=20))
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
357 [[1, 2, 3, 5, 8, 10, 11], [14]]
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
358 >>> list(_slicechunktodensity(revlog, [1, 2, 3, 5, 8, 10, 11, 14],
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
359 ... targetdensity=0.95))
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
360 [[1, 2], [5], [8, 10, 11], [14]]
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
361 >>> list(_slicechunktodensity(revlog, [1, 2, 3, 5, 8, 10, 11, 14],
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
362 ... targetdensity=0.95, mingapsize=12))
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
363 [[1, 2], [5, 8, 10, 11], [14]]
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
364 """
34824
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
365 start = revlog.start
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
366 length = revlog.length
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
367
34880
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
368 if len(revs) <= 1:
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
369 yield revs
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
370 return
34824
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
371
40604
3ac23dad6364 sparse-revlog: drop unused deltainfo parameter from _slicechunktodensity
Boris Feld <boris.feld@octobus.net>
parents: 40603
diff changeset
372 deltachainspan = segmentspan(revlog, revs)
34880
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
373
38641
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
374 if deltachainspan < mingapsize:
38635
d083ae26c325 revlog: early return in _slicechunk when span is already small enough
Boris Feld <boris.feld@octobus.net>
parents: 38634
diff changeset
375 yield revs
d083ae26c325 revlog: early return in _slicechunk when span is already small enough
Boris Feld <boris.feld@octobus.net>
parents: 38634
diff changeset
376 return
d083ae26c325 revlog: early return in _slicechunk when span is already small enough
Boris Feld <boris.feld@octobus.net>
parents: 38634
diff changeset
377
38718
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents: 38717
diff changeset
378 readdata = deltachainspan
40606
bfbfd15d65bd sparse-revlog: fast-path before computing payload size
Boris Feld <boris.feld@octobus.net>
parents: 40605
diff changeset
379 chainpayload = sum(length(r) for r in revs)
34824
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
380
34880
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
381 if deltachainspan:
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
382 density = chainpayload / float(deltachainspan)
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
383 else:
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
384 density = 1.0
34824
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
385
38641
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
386 if density >= targetdensity:
38634
f0ea8b847831 revlog: early return in _slicechunk when density is already good
Paul Morelle <paul.morelle@octobus.net>
parents: 38632
diff changeset
387 yield revs
f0ea8b847831 revlog: early return in _slicechunk when density is already good
Paul Morelle <paul.morelle@octobus.net>
parents: 38632
diff changeset
388 return
f0ea8b847831 revlog: early return in _slicechunk when density is already good
Paul Morelle <paul.morelle@octobus.net>
parents: 38632
diff changeset
389
34880
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
390 # Store the gaps in a heap to have them sorted by decreasing size
40607
54de23400b2a sparse-revlog: stop using a heap to track gaps
Boris Feld <boris.feld@octobus.net>
parents: 40606
diff changeset
391 gaps = []
34880
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
392 prevend = None
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
393 for i, rev in enumerate(revs):
40604
3ac23dad6364 sparse-revlog: drop unused deltainfo parameter from _slicechunktodensity
Boris Feld <boris.feld@octobus.net>
parents: 40603
diff changeset
394 revstart = start(rev)
3ac23dad6364 sparse-revlog: drop unused deltainfo parameter from _slicechunktodensity
Boris Feld <boris.feld@octobus.net>
parents: 40603
diff changeset
395 revlen = length(rev)
34824
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
396
34898
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents: 34881
diff changeset
397 # Skip empty revisions to form larger holes
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents: 34881
diff changeset
398 if revlen == 0:
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents: 34881
diff changeset
399 continue
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents: 34881
diff changeset
400
34880
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
401 if prevend is not None:
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
402 gapsize = revstart - prevend
34881
8c9b08a0c48c sparse-read: skip gaps too small to be worth splitting
Paul Morelle <paul.morelle@octobus.net>
parents: 34880
diff changeset
403 # only consider holes that are large enough
38641
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
404 if gapsize > mingapsize:
40607
54de23400b2a sparse-revlog: stop using a heap to track gaps
Boris Feld <boris.feld@octobus.net>
parents: 40606
diff changeset
405 gaps.append((gapsize, i))
34880
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
406
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
407 prevend = revstart + revlen
40607
54de23400b2a sparse-revlog: stop using a heap to track gaps
Boris Feld <boris.feld@octobus.net>
parents: 40606
diff changeset
408 # sort the gaps to pop them from largest to small
54de23400b2a sparse-revlog: stop using a heap to track gaps
Boris Feld <boris.feld@octobus.net>
parents: 40606
diff changeset
409 gaps.sort()
34880
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
410
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
411 # Collect the indices of the largest holes until the density is acceptable
40608
526ee887c4d5 sparse-revlog: stop using a heap to track selected gap
Boris Feld <boris.feld@octobus.net>
parents: 40607
diff changeset
412 selected = []
40607
54de23400b2a sparse-revlog: stop using a heap to track gaps
Boris Feld <boris.feld@octobus.net>
parents: 40606
diff changeset
413 while gaps and density < targetdensity:
54de23400b2a sparse-revlog: stop using a heap to track gaps
Boris Feld <boris.feld@octobus.net>
parents: 40606
diff changeset
414 gapsize, gapidx = gaps.pop()
34880
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
415
40608
526ee887c4d5 sparse-revlog: stop using a heap to track selected gap
Boris Feld <boris.feld@octobus.net>
parents: 40607
diff changeset
416 selected.append(gapidx)
34824
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
417
34880
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
418 # the gap sizes are stored as negatives to be sorted decreasingly
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
419 # by the heap
40607
54de23400b2a sparse-revlog: stop using a heap to track gaps
Boris Feld <boris.feld@octobus.net>
parents: 40606
diff changeset
420 readdata -= gapsize
34880
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
421 if readdata > 0:
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
422 density = chainpayload / float(readdata)
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
423 else:
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
424 density = 1.0
40608
526ee887c4d5 sparse-revlog: stop using a heap to track selected gap
Boris Feld <boris.feld@octobus.net>
parents: 40607
diff changeset
425 selected.sort()
34824
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
426
34880
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
427 # Cut the revs at collected indices
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
428 previdx = 0
40608
526ee887c4d5 sparse-revlog: stop using a heap to track selected gap
Boris Feld <boris.feld@octobus.net>
parents: 40607
diff changeset
429 for idx in selected:
34898
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents: 34881
diff changeset
430
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents: 34881
diff changeset
431 chunk = _trimchunk(revlog, revs, previdx, idx)
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents: 34881
diff changeset
432 if chunk:
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents: 34881
diff changeset
433 yield chunk
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents: 34881
diff changeset
434
34880
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
435 previdx = idx
34898
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents: 34881
diff changeset
436
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents: 34881
diff changeset
437 chunk = _trimchunk(revlog, revs, previdx)
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents: 34881
diff changeset
438 if chunk:
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents: 34881
diff changeset
439 yield chunk
34824
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
440
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
441
39330
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
442 def _trimchunk(revlog, revs, startidx, endidx=None):
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
443 """returns revs[startidx:endidx] without empty trailing revs
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
444
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
445 Doctest Setup
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
446 >>> revlog = _testrevlog([
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
447 ... 5, #0
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
448 ... 10, #1
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
449 ... 12, #2
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
450 ... 12, #3 (empty)
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
451 ... 17, #4
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
452 ... 21, #5
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
453 ... 21, #6 (empty)
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
454 ... ])
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
455
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
456 Contiguous cases:
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
457 >>> _trimchunk(revlog, [0, 1, 2, 3, 4, 5, 6], 0)
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
458 [0, 1, 2, 3, 4, 5]
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
459 >>> _trimchunk(revlog, [0, 1, 2, 3, 4, 5, 6], 0, 5)
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
460 [0, 1, 2, 3, 4]
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
461 >>> _trimchunk(revlog, [0, 1, 2, 3, 4, 5, 6], 0, 4)
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
462 [0, 1, 2]
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
463 >>> _trimchunk(revlog, [0, 1, 2, 3, 4, 5, 6], 2, 4)
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
464 [2]
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
465 >>> _trimchunk(revlog, [0, 1, 2, 3, 4, 5, 6], 3)
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
466 [3, 4, 5]
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
467 >>> _trimchunk(revlog, [0, 1, 2, 3, 4, 5, 6], 3, 5)
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
468 [3, 4]
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
469
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
470 Discontiguous cases:
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
471 >>> _trimchunk(revlog, [1, 3, 5, 6], 0)
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
472 [1, 3, 5]
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
473 >>> _trimchunk(revlog, [1, 3, 5, 6], 0, 2)
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
474 [1]
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
475 >>> _trimchunk(revlog, [1, 3, 5, 6], 1, 3)
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
476 [3, 5]
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
477 >>> _trimchunk(revlog, [1, 3, 5, 6], 1)
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
478 [3, 5]
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
479 """
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
480 length = revlog.length
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
481
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
482 if endidx is None:
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
483 endidx = len(revs)
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
484
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
485 # If we have a non-emtpy delta candidate, there are nothing to trim
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
486 if revs[endidx - 1] < len(revlog):
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
487 # Trim empty revs at the end, except the very first revision of a chain
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
488 while (
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
489 endidx > 1 and endidx > startidx and length(revs[endidx - 1]) == 0
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
490 ):
39330
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
491 endidx -= 1
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
492
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
493 return revs[startidx:endidx]
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
494
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
495
40605
a32ccd32982b sparse-revlog: drop unused deltainfo parameter from segmentspan
Boris Feld <boris.feld@octobus.net>
parents: 40604
diff changeset
496 def segmentspan(revlog, revs):
39330
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
497 """Get the byte span of a segment of revisions
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
498
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
499 revs is a sorted array of revision numbers
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
500
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
501 >>> revlog = _testrevlog([
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
502 ... 5, #0
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
503 ... 10, #1
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
504 ... 12, #2
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
505 ... 12, #3 (empty)
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
506 ... 17, #4
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
507 ... ])
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
508
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
509 >>> segmentspan(revlog, [0, 1, 2, 3, 4])
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
510 17
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
511 >>> segmentspan(revlog, [0, 4])
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
512 17
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
513 >>> segmentspan(revlog, [3, 4])
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
514 5
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
515 >>> segmentspan(revlog, [1, 2, 3,])
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
516 7
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
517 >>> segmentspan(revlog, [1, 3])
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
518 7
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
519 """
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
520 if not revs:
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
521 return 0
40605
a32ccd32982b sparse-revlog: drop unused deltainfo parameter from segmentspan
Boris Feld <boris.feld@octobus.net>
parents: 40604
diff changeset
522 end = revlog.end(revs[-1])
39330
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
523 return end - revlog.start(revs[0])
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
524
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
525
51020
509f0f7fc89e delta-computer: stop explicitly taking file handle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50929
diff changeset
526 def _textfromdelta(revlog, baserev, delta, p1, p2, flags, expectednode):
39331
fd0150a3c2fe revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents: 39330
diff changeset
527 """build full text from a (base, delta) pair and other metadata"""
fd0150a3c2fe revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents: 39330
diff changeset
528 # special case deltas which replace entire base; no need to decode
fd0150a3c2fe revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents: 39330
diff changeset
529 # base revision. this neatly avoids censored bases, which throw when
fd0150a3c2fe revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents: 39330
diff changeset
530 # they're decoded.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
531 hlen = struct.calcsize(b">lll")
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
532 if delta[:hlen] == mdiff.replacediffheader(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
533 revlog.rawsize(baserev), len(delta) - hlen
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
534 ):
39331
fd0150a3c2fe revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents: 39330
diff changeset
535 fulltext = delta[hlen:]
fd0150a3c2fe revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents: 39330
diff changeset
536 else:
fd0150a3c2fe revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents: 39330
diff changeset
537 # deltabase is rawtext before changed by flag processors, which is
fd0150a3c2fe revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents: 39330
diff changeset
538 # equivalent to non-raw text
51020
509f0f7fc89e delta-computer: stop explicitly taking file handle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50929
diff changeset
539 basetext = revlog.revision(baserev)
39331
fd0150a3c2fe revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents: 39330
diff changeset
540 fulltext = mdiff.patch(basetext, delta)
fd0150a3c2fe revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents: 39330
diff changeset
541
fd0150a3c2fe revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents: 39330
diff changeset
542 try:
42992
dff95420480f flagprocessors: make `processflagsraw` a module level function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42878
diff changeset
543 validatehash = flagutil.processflagsraw(revlog, fulltext, flags)
39331
fd0150a3c2fe revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents: 39330
diff changeset
544 if validatehash:
fd0150a3c2fe revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents: 39330
diff changeset
545 revlog.checkhash(fulltext, expectednode, p1=p1, p2=p2)
fd0150a3c2fe revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents: 39330
diff changeset
546 if flags & REVIDX_ISCENSORED:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
547 raise error.StorageError(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
548 _(b'node %s is not censored') % expectednode
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
549 )
39774
4a2466b2a434 revlog: drop some more error aliases (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39773
diff changeset
550 except error.CensoredNodeError:
39331
fd0150a3c2fe revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents: 39330
diff changeset
551 # must pass the censored index flag to add censored revisions
fd0150a3c2fe revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents: 39330
diff changeset
552 if not flags & REVIDX_ISCENSORED:
fd0150a3c2fe revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents: 39330
diff changeset
553 raise
fd0150a3c2fe revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents: 39330
diff changeset
554 return fulltext
fd0150a3c2fe revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents: 39330
diff changeset
555
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
556
35638
edc9330acac1 revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents: 35637
diff changeset
557 @attr.s(slots=True, frozen=True)
48946
642e31cb55f0 py3: use class X: instead of class X(object):
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48875
diff changeset
558 class _deltainfo:
35638
edc9330acac1 revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents: 35637
diff changeset
559 distance = attr.ib()
edc9330acac1 revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents: 35637
diff changeset
560 deltalen = attr.ib()
edc9330acac1 revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents: 35637
diff changeset
561 data = attr.ib()
edc9330acac1 revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents: 35637
diff changeset
562 base = attr.ib()
edc9330acac1 revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents: 35637
diff changeset
563 chainbase = attr.ib()
edc9330acac1 revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents: 35637
diff changeset
564 chainlen = attr.ib()
edc9330acac1 revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents: 35637
diff changeset
565 compresseddeltalen = attr.ib()
39154
e0da43e2f71f revlog: compute snapshot depth on delta info
Boris Feld <boris.feld@octobus.net>
parents: 39152
diff changeset
566 snapshotdepth = attr.ib()
35638
edc9330acac1 revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents: 35637
diff changeset
567
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
568
47253
b876f0bf7366 revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46819
diff changeset
569 def drop_u_compression(delta):
b876f0bf7366 revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46819
diff changeset
570 """turn into a "u" (no-compression) into no-compression without header
b876f0bf7366 revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46819
diff changeset
571
b876f0bf7366 revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46819
diff changeset
572 This is useful for revlog format that has better compression method.
b876f0bf7366 revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46819
diff changeset
573 """
b876f0bf7366 revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46819
diff changeset
574 assert delta.data[0] == b'u', delta.data[0]
b876f0bf7366 revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46819
diff changeset
575 return _deltainfo(
b876f0bf7366 revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46819
diff changeset
576 delta.distance,
b876f0bf7366 revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46819
diff changeset
577 delta.deltalen - 1,
b876f0bf7366 revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46819
diff changeset
578 (b'', delta.data[1]),
b876f0bf7366 revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46819
diff changeset
579 delta.base,
b876f0bf7366 revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46819
diff changeset
580 delta.chainbase,
b876f0bf7366 revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46819
diff changeset
581 delta.chainlen,
b876f0bf7366 revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46819
diff changeset
582 delta.compresseddeltalen,
b876f0bf7366 revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46819
diff changeset
583 delta.snapshotdepth,
b876f0bf7366 revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46819
diff changeset
584 )
b876f0bf7366 revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46819
diff changeset
585
b876f0bf7366 revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46819
diff changeset
586
49674
5af4a0a73e4c find-delta: rename _isgooddeltainfo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49657
diff changeset
587 def is_good_delta_info(revlog, deltainfo, revinfo):
39330
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
588 """Returns True if the given delta is good. Good means that it is within
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
589 the disk span, disk size, and chain length bounds that we know to be
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
590 performant."""
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
591 if deltainfo is None:
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
592 return False
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
593
50698
f1b57672cb94 delta-find: remove dead code intended to deal with forced delta reuse
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50686
diff changeset
594 # the DELTA_BASE_REUSE_FORCE case should have been taken care of sooner so
f1b57672cb94 delta-find: remove dead code intended to deal with forced delta reuse
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50686
diff changeset
595 # we should never end up asking such question. Adding the assert as a
f1b57672cb94 delta-find: remove dead code intended to deal with forced delta reuse
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50686
diff changeset
596 # safe-guard to detect anything that would be fishy in this regard.
f1b57672cb94 delta-find: remove dead code intended to deal with forced delta reuse
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50686
diff changeset
597 assert (
f1b57672cb94 delta-find: remove dead code intended to deal with forced delta reuse
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50686
diff changeset
598 revinfo.cachedelta is None
f1b57672cb94 delta-find: remove dead code intended to deal with forced delta reuse
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50686
diff changeset
599 or revinfo.cachedelta[2] != DELTA_BASE_REUSE_FORCE
51046
f636103c4d67 revlog: remove legacy usage of `_generaldelta`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51045
diff changeset
600 or not revlog.delta_config.general_delta
50698
f1b57672cb94 delta-find: remove dead code intended to deal with forced delta reuse
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50686
diff changeset
601 )
49768
bcae90c53def delta-find: add a delta-reuse policy that blindly accepts incoming deltas
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49681
diff changeset
602
39330
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
603 # - 'deltainfo.distance' is the distance from the base revision --
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
604 # bounding it limits the amount of I/O we need to do.
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
605 # - 'deltainfo.compresseddeltalen' is the sum of the total size of
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
606 # deltas we need to apply -- bounding it limits the amount of CPU
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
607 # we consume.
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
608
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
609 textlen = revinfo.textlen
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
610 defaultmax = textlen * 4
51058
8bdb2478c4bc revlog: remove legacy usage of `_maxdeltachainspan`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51055
diff changeset
611 maxdist = revlog.delta_config.max_deltachain_span
39330
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
612 if not maxdist:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
613 maxdist = deltainfo.distance # ensure the conditional pass
39330
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
614 maxdist = max(maxdist, defaultmax)
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
615
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
616 # Bad delta from read span:
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
617 #
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
618 # If the span of data read is larger than the maximum allowed.
40603
2f7e531ef3e7 sparse-revlog: skip the span check in the sparse-revlog case
Boris Feld <boris.feld@octobus.net>
parents: 40451
diff changeset
619 #
2f7e531ef3e7 sparse-revlog: skip the span check in the sparse-revlog case
Boris Feld <boris.feld@octobus.net>
parents: 40451
diff changeset
620 # In the sparse-revlog case, we rely on the associated "sparse reading"
2f7e531ef3e7 sparse-revlog: skip the span check in the sparse-revlog case
Boris Feld <boris.feld@octobus.net>
parents: 40451
diff changeset
621 # to avoid issue related to the span of data. In theory, it would be
2f7e531ef3e7 sparse-revlog: skip the span check in the sparse-revlog case
Boris Feld <boris.feld@octobus.net>
parents: 40451
diff changeset
622 # possible to build pathological revlog where delta pattern would lead
2f7e531ef3e7 sparse-revlog: skip the span check in the sparse-revlog case
Boris Feld <boris.feld@octobus.net>
parents: 40451
diff changeset
623 # to too many reads. However, they do not happen in practice at all. So
2f7e531ef3e7 sparse-revlog: skip the span check in the sparse-revlog case
Boris Feld <boris.feld@octobus.net>
parents: 40451
diff changeset
624 # we skip the span check entirely.
51060
f71f07a679b4 revlog: remove legacy usage of `_sparserevlog`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51058
diff changeset
625 if not revlog.delta_config.sparse_revlog and maxdist < deltainfo.distance:
39330
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
626 return False
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
627
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
628 # Bad delta from new delta size:
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
629 #
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
630 # If the delta size is larger than the target text, storing the
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
631 # delta will be inefficient.
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
632 if textlen < deltainfo.deltalen:
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
633 return False
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
634
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
635 # Bad delta from cumulated payload size:
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
636 #
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
637 # If the sum of delta get larger than K * target text length.
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
638 if textlen * LIMIT_DELTA2TEXT < deltainfo.compresseddeltalen:
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
639 return False
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
640
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
641 # Bad delta from chain length:
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
642 #
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
643 # If the number of delta in the chain gets too high.
51052
e80e2d614c9d revlog: remove legacy usage of `_maxchainlen`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51046
diff changeset
644 if (
e80e2d614c9d revlog: remove legacy usage of `_maxchainlen`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51046
diff changeset
645 revlog.delta_config.max_chain_len
e80e2d614c9d revlog: remove legacy usage of `_maxchainlen`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51046
diff changeset
646 and revlog.delta_config.max_chain_len < deltainfo.chainlen
e80e2d614c9d revlog: remove legacy usage of `_maxchainlen`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51046
diff changeset
647 ):
39330
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
648 return False
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
649
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
650 # bad delta from intermediate snapshot size limit
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
651 #
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
652 # If an intermediate snapshot size is higher than the limit. The
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
653 # limit exist to prevent endless chain of intermediate delta to be
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
654 # created.
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
655 if (
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
656 deltainfo.snapshotdepth is not None
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
657 and (textlen >> deltainfo.snapshotdepth) < deltainfo.deltalen
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
658 ):
39330
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
659 return False
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
660
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
661 # bad delta if new intermediate snapshot is larger than the previous
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
662 # snapshot
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
663 if (
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
664 deltainfo.snapshotdepth
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
665 and revlog.length(deltainfo.base) < deltainfo.deltalen
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
666 ):
39330
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
667 return False
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
668
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
669 return True
655b5b465953 revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents: 39329
diff changeset
670
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
671
40978
42f59d3f714d delta: exclude base candidate much smaller than the target
Boris Feld <boris.feld@octobus.net>
parents: 40957
diff changeset
672 # If a revision's full text is that much bigger than a base candidate full
42f59d3f714d delta: exclude base candidate much smaller than the target
Boris Feld <boris.feld@octobus.net>
parents: 40957
diff changeset
673 # text's, it is very unlikely that it will produce a valid delta. We no longer
42f59d3f714d delta: exclude base candidate much smaller than the target
Boris Feld <boris.feld@octobus.net>
parents: 40957
diff changeset
674 # consider these candidates.
41033
b373477948df revlog: limit base to rev size ratio to 500 instead of 50
Boris Feld <boris.feld@octobus.net>
parents: 40979
diff changeset
675 LIMIT_BASE2TEXT = 500
40978
42f59d3f714d delta: exclude base candidate much smaller than the target
Boris Feld <boris.feld@octobus.net>
parents: 40957
diff changeset
676
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
677
51320
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
678 class _DeltaSearch:
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
679 """perform the search of a good delta for a single revlog revision
50354
ca1522fe4ec8 delta-find: assume the target-rev if not specified
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49776
diff changeset
680
51320
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
681 note: some of the deltacomputer.finddeltainfo logic should probably move
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
682 here.
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
683 """
50355
0232571255d3 delta-find: never do anything fancy when general delta is off
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50354
diff changeset
684
51320
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
685 def __init__(
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
686 self,
49610
35d4c2124073 debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49608
diff changeset
687 revlog,
51320
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
688 textlen,
49610
35d4c2124073 debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49608
diff changeset
689 p1,
35d4c2124073 debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49608
diff changeset
690 p2,
35d4c2124073 debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49608
diff changeset
691 cachedelta,
51320
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
692 excluded_bases=None,
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
693 target_rev=None,
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
694 snapshot_cache=None,
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
695 ):
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
696 self.revlog = revlog
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
697 self.textlen = textlen
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
698 self.p1 = p1
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
699 self.p2 = p2
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
700 self.cachedelta = cachedelta
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
701 self.excluded_bases = excluded_bases
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
702 self.target_rev = target_rev
51323
87fce139e927 delta-find: move snapshot_cache in the _DeltaSearch.__init__
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51322
diff changeset
703 if snapshot_cache is None:
87fce139e927 delta-find: move snapshot_cache in the _DeltaSearch.__init__
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51322
diff changeset
704 # map: base-rev: [snapshot-revs]
87fce139e927 delta-find: move snapshot_cache in the _DeltaSearch.__init__
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51322
diff changeset
705 snapshot_cache = SnapshotCache()
51320
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
706 self.snapshot_cache = snapshot_cache
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
707
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
708 def candidate_groups(self):
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
709 """Provides group of revision to be tested as delta base
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
710
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
711 This top level function focus on emitting groups with unique and
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
712 worthwhile content. See _raw_candidate_groups for details about the
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
713 group order.
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
714 """
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
715 # should we try to build a delta?
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
716 if not (len(self.revlog) and self.revlog._storedeltachains):
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
717 yield None
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
718 return
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
719
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
720 if self.target_rev is None:
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
721 self.target_rev = len(self.revlog)
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
722
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
723 if not self.revlog.delta_config.general_delta:
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
724 # before general delta, there is only one possible delta base
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
725 yield (self.target_rev - 1,)
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
726 yield None
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
727 return
49768
bcae90c53def delta-find: add a delta-reuse policy that blindly accepts incoming deltas
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49681
diff changeset
728
51320
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
729 # the DELTA_BASE_REUSE_FORCE case should have been taken care of sooner
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
730 # so we should never end up asking such question. Adding the assert as
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
731 # a safe-guard to detect anything that would be fishy in this regard.
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
732 assert (
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
733 self.cachedelta is None
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
734 or self.cachedelta[2] != DELTA_BASE_REUSE_FORCE
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
735 or not self.revlog.delta_config.general_delta
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
736 )
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
737
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
738 deltalength = self.revlog.length
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
739 deltaparent = self.revlog.deltaparent
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
740 sparse = self.revlog.delta_config.sparse_revlog
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
741 good = None
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
742
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
743 deltas_limit = self.textlen * LIMIT_DELTA2TEXT
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
744 group_chunk_size = self.revlog.delta_config.candidate_group_chunk_size
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
745
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
746 tested = {nullrev}
51321
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51320
diff changeset
747 candidates = self._refined_groups()
51320
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
748 while True:
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
749 temptative = candidates.send(good)
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
750 if temptative is None:
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
751 break
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
752 group = []
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
753 for rev in temptative:
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
754 # skip over empty delta (no need to include them in a chain)
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
755 while not (rev == nullrev or rev in tested or deltalength(rev)):
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
756 tested.add(rev)
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
757 rev = deltaparent(rev)
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
758 # no need to try a delta against nullrev, this will be done as
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
759 # a last resort.
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
760 if rev == nullrev:
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
761 continue
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
762 # filter out revision we tested already
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
763 if rev in tested:
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
764 continue
49610
35d4c2124073 debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49608
diff changeset
765
51320
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
766 # an higher authority deamed the base unworthy (e.g. censored)
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
767 if (
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
768 self.excluded_bases is not None
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
769 and rev in self.excluded_bases
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
770 ):
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
771 tested.add(rev)
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
772 continue
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
773 # We are in some recomputation cases and that rev is too high
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
774 # in the revlog
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
775 if self.target_rev is not None and rev >= self.target_rev:
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
776 tested.add(rev)
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
777 continue
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
778 # filter out delta base that will never produce good delta
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
779 if deltas_limit < self.revlog.length(rev):
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
780 tested.add(rev)
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
781 continue
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
782 if sparse and self.revlog.rawsize(rev) < (
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
783 self.textlen // LIMIT_BASE2TEXT
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
784 ):
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
785 tested.add(rev)
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
786 continue
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
787 # no delta for rawtext-changing revs (see "candelta" for why)
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
788 if self.revlog.flags(rev) & REVIDX_RAWTEXT_CHANGING_FLAGS:
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
789 tested.add(rev)
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
790 continue
42463
a0b26fc8fbba deltas: skip if projected delta size does not match text size constraint
Valentin Gatien-Baron <vgatien-baron@janestreet.com>, Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42057
diff changeset
791
51320
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
792 # If we reach here, we are about to build and test a delta.
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
793 # The delta building process will compute the chaininfo in all
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
794 # case, since that computation is cached, it is fine to access
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
795 # it here too.
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
796 chainlen, chainsize = self.revlog._chaininfo(rev)
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
797 # if chain will be too long, skip base
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
798 if (
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
799 self.revlog.delta_config.max_chain_len
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
800 and chainlen >= self.revlog.delta_config.max_chain_len
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
801 ):
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
802 tested.add(rev)
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
803 continue
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
804 # if chain already have too much data, skip base
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
805 if deltas_limit < chainsize:
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
806 tested.add(rev)
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
807 continue
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
808 if (
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
809 sparse
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
810 and self.revlog.delta_config.upper_bound_comp is not None
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
811 ):
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
812 maxcomp = self.revlog.delta_config.upper_bound_comp
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
813 basenotsnap = (self.p1, self.p2, nullrev)
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
814 if rev not in basenotsnap and self.revlog.issnapshot(rev):
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
815 snapshotdepth = self.revlog.snapshotdepth(rev)
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
816 # If text is significantly larger than the base, we can
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
817 # expect the resulting delta to be proportional to the size
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
818 # difference
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
819 revsize = self.revlog.rawsize(rev)
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
820 rawsizedistance = max(self.textlen - revsize, 0)
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
821 # use an estimate of the compression upper bound.
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
822 lowestrealisticdeltalen = rawsizedistance // maxcomp
42464
66c27df1be84 deltas: skip if projected delta size is bigger than previous snapshot
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents: 42463
diff changeset
823
51320
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
824 # check the absolute constraint on the delta size
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
825 snapshotlimit = self.textlen >> snapshotdepth
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
826 if snapshotlimit < lowestrealisticdeltalen:
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
827 # delta lower bound is larger than accepted upper
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
828 # bound
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
829 tested.add(rev)
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
830 continue
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
831
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
832 # check the relative constraint on the delta size
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
833 revlength = self.revlog.length(rev)
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
834 if revlength < lowestrealisticdeltalen:
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
835 # delta probable lower bound is larger than target
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
836 # base
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
837 tested.add(rev)
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
838 continue
49657
f5f113f1b011 delta-find: add a way to control the number of bases tested at the same time
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49615
diff changeset
839
51320
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
840 group.append(rev)
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
841 if group:
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
842 # When the size of the candidate group is big, it can result in
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
843 # a quite significant performance impact. To reduce this, we
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
844 # can send them in smaller batches until the new batch does not
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
845 # provide any improvements.
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
846 #
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
847 # This might reduce the overall efficiency of the compression
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
848 # in some corner cases, but that should also prevent very
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
849 # pathological cases from being an issue. (eg. 20 000
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
850 # candidates).
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
851 #
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
852 # XXX note that the ordering of the group becomes important as
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
853 # it now impacts the final result. The current order is
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
854 # unprocessed and can be improved.
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
855 if group_chunk_size == 0:
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
856 tested.update(group)
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
857 good = yield tuple(group)
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
858 else:
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
859 prev_good = good
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
860 for start in range(0, len(group), group_chunk_size):
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
861 sub_group = group[start : start + group_chunk_size]
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
862 tested.update(sub_group)
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
863 good = yield tuple(sub_group)
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
864 if prev_good == good:
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
865 break
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
866
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
867 yield None
39336
1c6ff52fe9cf revlogdeltas: split candidate groups selection from the filtering logic
Boris Feld <boris.feld@octobus.net>
parents: 39335
diff changeset
868
51321
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51320
diff changeset
869 def _refined_groups(self):
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51320
diff changeset
870 good = None
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51320
diff changeset
871 # First we try to reuse a the delta contained in the bundle. (or from
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51320
diff changeset
872 # the source revlog)
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51320
diff changeset
873 #
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51320
diff changeset
874 # This logic only applies to general delta repositories and can be
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51320
diff changeset
875 # disabled through configuration. Disabling reuse source delta is
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51320
diff changeset
876 # useful when we want to make sure we recomputed "optimal" deltas.
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51320
diff changeset
877 debug_info = None
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51320
diff changeset
878 if (
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51320
diff changeset
879 self.cachedelta is not None
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51320
diff changeset
880 and self.cachedelta[2] > DELTA_BASE_REUSE_NO
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51320
diff changeset
881 ):
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51320
diff changeset
882 # Assume what we received from the server is a good choice
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51320
diff changeset
883 # build delta will reuse the cache
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51320
diff changeset
884 if debug_info is not None:
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51320
diff changeset
885 debug_info['cached-delta.tested'] += 1
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51320
diff changeset
886 good = yield (self.cachedelta[0],)
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51320
diff changeset
887 if good is not None:
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51320
diff changeset
888 if debug_info is not None:
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51320
diff changeset
889 debug_info['cached-delta.accepted'] += 1
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51320
diff changeset
890 yield None
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51320
diff changeset
891 return
51322
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
892 groups = self._raw_groups()
51321
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51320
diff changeset
893 for candidates in groups:
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51320
diff changeset
894 good = yield candidates
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51320
diff changeset
895 if good is not None:
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51320
diff changeset
896 break
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
897
51321
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51320
diff changeset
898 # If sparse revlog is enabled, we can try to refine the available
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51320
diff changeset
899 # deltas
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51320
diff changeset
900 if not self.revlog.delta_config.sparse_revlog:
39501
993d7e2c8b79 snapshot: make sure we'll never refine delta base from a reused source
Boris Feld <boris.feld@octobus.net>
parents: 39500
diff changeset
901 yield None
993d7e2c8b79 snapshot: make sure we'll never refine delta base from a reused source
Boris Feld <boris.feld@octobus.net>
parents: 39500
diff changeset
902 return
40428
bafa1c4bb7a8 sparse-revlog: only refine delta candidates in the sparse case (issue6006)
Boris Feld <boris.feld@octobus.net>
parents: 39777
diff changeset
903
51321
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51320
diff changeset
904 # if we have a refinable value, try to refine it
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51320
diff changeset
905 if (
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51320
diff changeset
906 good is not None
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51320
diff changeset
907 and good not in (self.p1, self.p2)
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51320
diff changeset
908 and self.revlog.issnapshot(good)
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51320
diff changeset
909 ):
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51320
diff changeset
910 # refine snapshot down
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51320
diff changeset
911 previous = None
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51320
diff changeset
912 while previous != good:
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51320
diff changeset
913 previous = good
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51320
diff changeset
914 base = self.revlog.deltaparent(good)
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51320
diff changeset
915 if base == nullrev:
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51320
diff changeset
916 break
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51320
diff changeset
917 good = yield (base,)
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51320
diff changeset
918 # refine snapshot up
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51320
diff changeset
919 if not self.snapshot_cache.snapshots:
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51320
diff changeset
920 self.snapshot_cache.update(self.revlog, good + 1)
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51320
diff changeset
921 previous = None
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51320
diff changeset
922 while good != previous:
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51320
diff changeset
923 previous = good
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51320
diff changeset
924 children = tuple(
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51320
diff changeset
925 sorted(c for c in self.snapshot_cache.snapshots[good])
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51320
diff changeset
926 )
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51320
diff changeset
927 good = yield children
39503
5aef5afa8654 snapshot: refine candidate snapshot base upward
Boris Feld <boris.feld@octobus.net>
parents: 39502
diff changeset
928
51321
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51320
diff changeset
929 if debug_info is not None:
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51320
diff changeset
930 if good is None:
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51320
diff changeset
931 debug_info['no-solution'] += 1
49610
35d4c2124073 debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49608
diff changeset
932
51321
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51320
diff changeset
933 yield None
39496
2f9f7889549b snapshot: introduce an intermediate `_refinedgroups` generator
Boris Feld <boris.feld@octobus.net>
parents: 39495
diff changeset
934
51322
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
935 def _raw_groups(self):
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
936 """Provides group of revision to be tested as delta base
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
937
51322
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
938 This lower level function focus on emitting delta theorically
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
939 interresting without looking it any practical details.
39336
1c6ff52fe9cf revlogdeltas: split candidate groups selection from the filtering logic
Boris Feld <boris.feld@octobus.net>
parents: 39335
diff changeset
940
51322
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
941 The group order aims at providing fast or small candidates first.
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
942 """
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
943 # Why search for delta base if we cannot use a delta base ?
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
944 assert self.revlog.delta_config.general_delta
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
945 # also see issue6056
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
946 sparse = self.revlog.delta_config.sparse_revlog
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
947 curr = len(self.revlog)
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
948 prev = curr - 1
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
949 deltachain = lambda rev: self.revlog._deltachain(rev)[0]
39336
1c6ff52fe9cf revlogdeltas: split candidate groups selection from the filtering logic
Boris Feld <boris.feld@octobus.net>
parents: 39335
diff changeset
950
51322
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
951 # exclude already lazy tested base if any
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
952 parents = [p for p in (self.p1, self.p2) if p != nullrev]
39334
507f5b1dd7c8 revlogdeltas: extract _getcandidaterevs in a function
Boris Feld <boris.feld@octobus.net>
parents: 39333
diff changeset
953
51322
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
954 if (
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
955 not self.revlog.delta_config.delta_both_parents
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
956 and len(parents) == 2
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
957 ):
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
958 parents.sort()
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
959 # To minimize the chance of having to build a fulltext,
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
960 # pick first whichever parent is closest to us (max rev)
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
961 yield (parents[1],)
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
962 # then the other one (min rev) if the first did not fit
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
963 yield (parents[0],)
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
964 elif len(parents) > 0:
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
965 # Test all parents (1 or 2), and keep the best candidate
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
966 yield parents
39334
507f5b1dd7c8 revlogdeltas: extract _getcandidaterevs in a function
Boris Feld <boris.feld@octobus.net>
parents: 39333
diff changeset
967
51322
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
968 if sparse and parents:
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
969 # See if we can use an existing snapshot in the parent chains to
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
970 # use as a base for a new intermediate-snapshot
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
971 #
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
972 # search for snapshot in parents delta chain map: snapshot-level:
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
973 # snapshot-rev
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
974 parents_snaps = collections.defaultdict(set)
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
975 candidate_chains = [deltachain(p) for p in parents]
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
976 for chain in candidate_chains:
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
977 for idx, s in enumerate(chain):
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
978 if not self.revlog.issnapshot(s):
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
979 break
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
980 parents_snaps[idx].add(s)
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
981 snapfloor = min(parents_snaps[0]) + 1
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
982 self.snapshot_cache.update(self.revlog, snapfloor)
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
983 # search for the highest "unrelated" revision
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
984 #
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
985 # Adding snapshots used by "unrelated" revision increase the odd we
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
986 # reuse an independant, yet better snapshot chain.
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
987 #
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
988 # XXX instead of building a set of revisions, we could lazily
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
989 # enumerate over the chains. That would be more efficient, however
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
990 # we stick to simple code for now.
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
991 all_revs = set()
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
992 for chain in candidate_chains:
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
993 all_revs.update(chain)
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
994 other = None
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
995 for r in self.revlog.revs(prev, snapfloor):
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
996 if r not in all_revs:
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
997 other = r
39494
e72130f58f5d snapshot: consider all snapshots in the parents' chains
Boris Feld <boris.feld@octobus.net>
parents: 39493
diff changeset
998 break
51322
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
999 if other is not None:
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
1000 # To avoid unfair competition, we won't use unrelated
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
1001 # intermediate snapshot that are deeper than the ones from the
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
1002 # parent delta chain.
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
1003 max_depth = max(parents_snaps.keys())
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
1004 chain = deltachain(other)
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
1005 for depth, s in enumerate(chain):
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
1006 if s < snapfloor:
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
1007 continue
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
1008 if max_depth < depth:
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
1009 break
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
1010 if not self.revlog.issnapshot(s):
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
1011 break
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
1012 parents_snaps[depth].add(s)
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
1013 # Test them as possible intermediate snapshot base We test them
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
1014 # from highest to lowest level. High level one are more likely to
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
1015 # result in small delta
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
1016 floor = None
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
1017 for idx, snaps in sorted(parents_snaps.items(), reverse=True):
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
1018 siblings = set()
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
1019 for s in snaps:
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
1020 siblings.update(self.snapshot_cache.snapshots[s])
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
1021 # Before considering making a new intermediate snapshot, we
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
1022 # check if an existing snapshot, children of base we consider,
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
1023 # would be suitable.
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
1024 #
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
1025 # It give a change to reuse a delta chain "unrelated" to the
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
1026 # current revision instead of starting our own. Without such
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
1027 # re-use, topological branches would keep reopening new chains.
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
1028 # Creating more and more snapshot as the repository grow.
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
1029
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
1030 if floor is not None:
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
1031 # We only do this for siblings created after the one in our
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
1032 # parent's delta chain. Those created before has less
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
1033 # chances to be valid base since our ancestors had to
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
1034 # create a new snapshot.
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
1035 siblings = [r for r in siblings if floor < r]
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
1036 yield tuple(sorted(siblings))
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
1037 # then test the base from our parent's delta chain.
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
1038 yield tuple(sorted(snaps))
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
1039 floor = min(snaps)
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
1040 # No suitable base found in the parent chain, search if any full
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
1041 # snapshots emitted since parent's base would be a suitable base
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
1042 # for an intermediate snapshot.
39495
6a53842727c1 snapshot: consider unrelated snapshots at a similar level first
Boris Feld <boris.feld@octobus.net>
parents: 39494
diff changeset
1043 #
51322
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
1044 # It give a chance to reuse a delta chain unrelated to the current
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
1045 # revisions instead of starting our own. Without such re-use,
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
1046 # topological branches would keep reopening new full chains.
39495
6a53842727c1 snapshot: consider unrelated snapshots at a similar level first
Boris Feld <boris.feld@octobus.net>
parents: 39494
diff changeset
1047 # Creating more and more snapshot as the repository grow.
51322
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
1048 full = [
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
1049 r
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
1050 for r in self.snapshot_cache.snapshots[nullrev]
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
1051 if snapfloor <= r
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
1052 ]
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
1053 yield tuple(sorted(full))
39495
6a53842727c1 snapshot: consider unrelated snapshots at a similar level first
Boris Feld <boris.feld@octobus.net>
parents: 39494
diff changeset
1054
51322
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
1055 if not sparse:
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
1056 # other approach failed try against prev to hopefully save us a
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
1057 # fulltext.
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51321
diff changeset
1058 yield (prev,)
39334
507f5b1dd7c8 revlogdeltas: extract _getcandidaterevs in a function
Boris Feld <boris.feld@octobus.net>
parents: 39333
diff changeset
1059
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
1060
49678
efbbc2f9121e delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49677
diff changeset
1061 class SnapshotCache:
efbbc2f9121e delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49677
diff changeset
1062 __slots__ = ('snapshots', '_start_rev', '_end_rev')
efbbc2f9121e delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49677
diff changeset
1063
efbbc2f9121e delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49677
diff changeset
1064 def __init__(self):
49679
b670eb3dd6c9 delta-find: use sets instead of list in the snapshot cache
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49678
diff changeset
1065 self.snapshots = collections.defaultdict(set)
49678
efbbc2f9121e delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49677
diff changeset
1066 self._start_rev = None
efbbc2f9121e delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49677
diff changeset
1067 self._end_rev = None
efbbc2f9121e delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49677
diff changeset
1068
efbbc2f9121e delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49677
diff changeset
1069 def update(self, revlog, start_rev=0):
efbbc2f9121e delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49677
diff changeset
1070 """find snapshots from start_rev to tip"""
efbbc2f9121e delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49677
diff changeset
1071 nb_revs = len(revlog)
efbbc2f9121e delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49677
diff changeset
1072 end_rev = nb_revs - 1
efbbc2f9121e delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49677
diff changeset
1073 if start_rev > end_rev:
efbbc2f9121e delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49677
diff changeset
1074 return # range is empty
efbbc2f9121e delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49677
diff changeset
1075
efbbc2f9121e delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49677
diff changeset
1076 if self._start_rev is None:
efbbc2f9121e delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49677
diff changeset
1077 assert self._end_rev is None
efbbc2f9121e delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49677
diff changeset
1078 self._update(revlog, start_rev, end_rev)
efbbc2f9121e delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49677
diff changeset
1079 elif not (self._start_rev <= start_rev and end_rev <= self._end_rev):
efbbc2f9121e delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49677
diff changeset
1080 if start_rev < self._start_rev:
efbbc2f9121e delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49677
diff changeset
1081 self._update(revlog, start_rev, self._start_rev - 1)
efbbc2f9121e delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49677
diff changeset
1082 if self._end_rev < end_rev:
efbbc2f9121e delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49677
diff changeset
1083 self._update(revlog, self._end_rev + 1, end_rev)
efbbc2f9121e delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49677
diff changeset
1084
efbbc2f9121e delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49677
diff changeset
1085 if self._start_rev is None:
efbbc2f9121e delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49677
diff changeset
1086 assert self._end_rev is None
efbbc2f9121e delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49677
diff changeset
1087 self._end_rev = end_rev
efbbc2f9121e delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49677
diff changeset
1088 self._start_rev = start_rev
efbbc2f9121e delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49677
diff changeset
1089 else:
efbbc2f9121e delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49677
diff changeset
1090 self._start_rev = min(self._start_rev, start_rev)
efbbc2f9121e delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49677
diff changeset
1091 self._end_rev = max(self._end_rev, end_rev)
efbbc2f9121e delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49677
diff changeset
1092 assert self._start_rev <= self._end_rev, (
efbbc2f9121e delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49677
diff changeset
1093 self._start_rev,
efbbc2f9121e delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49677
diff changeset
1094 self._end_rev,
efbbc2f9121e delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49677
diff changeset
1095 )
efbbc2f9121e delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49677
diff changeset
1096
efbbc2f9121e delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49677
diff changeset
1097 def _update(self, revlog, start_rev, end_rev):
efbbc2f9121e delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49677
diff changeset
1098 """internal method that actually do update content"""
efbbc2f9121e delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49677
diff changeset
1099 assert self._start_rev is None or (
efbbc2f9121e delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49677
diff changeset
1100 start_rev < self._start_rev or start_rev > self._end_rev
efbbc2f9121e delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49677
diff changeset
1101 ), (self._start_rev, self._end_rev, start_rev, end_rev)
efbbc2f9121e delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49677
diff changeset
1102 assert self._start_rev is None or (
efbbc2f9121e delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49677
diff changeset
1103 end_rev < self._start_rev or end_rev > self._end_rev
efbbc2f9121e delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49677
diff changeset
1104 ), (self._start_rev, self._end_rev, start_rev, end_rev)
efbbc2f9121e delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49677
diff changeset
1105 cache = self.snapshots
50928
d718eddf01d9 safehasattr: drop usage in favor of hasattr
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50698
diff changeset
1106 if hasattr(revlog.index, 'findsnapshots'):
49678
efbbc2f9121e delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49677
diff changeset
1107 revlog.index.findsnapshots(cache, start_rev, end_rev)
efbbc2f9121e delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49677
diff changeset
1108 else:
efbbc2f9121e delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49677
diff changeset
1109 deltaparent = revlog.deltaparent
efbbc2f9121e delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49677
diff changeset
1110 issnapshot = revlog.issnapshot
efbbc2f9121e delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49677
diff changeset
1111 for rev in revlog.revs(start_rev, end_rev):
efbbc2f9121e delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49677
diff changeset
1112 if issnapshot(rev):
49679
b670eb3dd6c9 delta-find: use sets instead of list in the snapshot cache
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49678
diff changeset
1113 cache[deltaparent(rev)].add(rev)
49678
efbbc2f9121e delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49677
diff changeset
1114
efbbc2f9121e delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49677
diff changeset
1115
48946
642e31cb55f0 py3: use class X: instead of class X(object):
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48875
diff changeset
1116 class deltacomputer:
51319
da7ecb4deaec delta-find: add a small docstring to deltacomputer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51091
diff changeset
1117 """object capable of computing delta and finding delta for multiple revision
da7ecb4deaec delta-find: add a small docstring to deltacomputer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51091
diff changeset
1118
da7ecb4deaec delta-find: add a small docstring to deltacomputer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51091
diff changeset
1119 This object is meant to compute and find multiple delta applied to the same
da7ecb4deaec delta-find: add a small docstring to deltacomputer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51091
diff changeset
1120 revlog.
da7ecb4deaec delta-find: add a small docstring to deltacomputer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51091
diff changeset
1121 """
da7ecb4deaec delta-find: add a small docstring to deltacomputer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51091
diff changeset
1122
49610
35d4c2124073 debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49608
diff changeset
1123 def __init__(
35d4c2124073 debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49608
diff changeset
1124 self,
35d4c2124073 debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49608
diff changeset
1125 revlog,
35d4c2124073 debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49608
diff changeset
1126 write_debug=None,
35d4c2124073 debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49608
diff changeset
1127 debug_search=False,
35d4c2124073 debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49608
diff changeset
1128 debug_info=None,
35d4c2124073 debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49608
diff changeset
1129 ):
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
1130 self.revlog = revlog
49226
e6b7c6fbeb48 deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48946
diff changeset
1131 self._write_debug = write_debug
50649
c84cc0ac77e4 delta-fine: use the `_debug_search` attribute directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50648
diff changeset
1132 if write_debug is None:
c84cc0ac77e4 delta-fine: use the `_debug_search` attribute directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50648
diff changeset
1133 self._debug_search = False
c84cc0ac77e4 delta-fine: use the `_debug_search` attribute directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50648
diff changeset
1134 else:
c84cc0ac77e4 delta-fine: use the `_debug_search` attribute directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50648
diff changeset
1135 self._debug_search = debug_search
49610
35d4c2124073 debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49608
diff changeset
1136 self._debug_info = debug_info
49681
c261a628e525 delta-find: use a single snapshot cache when applying a group to an object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49680
diff changeset
1137 self._snapshot_cache = SnapshotCache()
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
1138
50648
5d210ff4b657 delta-find: move the `gather_debug` logic in a property
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50357
diff changeset
1139 @property
5d210ff4b657 delta-find: move the `gather_debug` logic in a property
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50357
diff changeset
1140 def _gather_debug(self):
5d210ff4b657 delta-find: move the `gather_debug` logic in a property
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50357
diff changeset
1141 return self._write_debug is not None or self._debug_info is not None
5d210ff4b657 delta-find: move the `gather_debug` logic in a property
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50357
diff changeset
1142
51020
509f0f7fc89e delta-computer: stop explicitly taking file handle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50929
diff changeset
1143 def buildtext(self, revinfo):
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
1144 """Builds a fulltext version of a revision
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
1145
47399
34cc102c73f5 revlog: move `revisioninfo` in `revlogutils`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47253
diff changeset
1146 revinfo: revisioninfo instance that contains all needed info
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
1147 """
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
1148 btext = revinfo.btext
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
1149 if btext[0] is not None:
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
1150 return btext[0]
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
1151
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
1152 revlog = self.revlog
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
1153 cachedelta = revinfo.cachedelta
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
1154 baserev = cachedelta[0]
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
1155 delta = cachedelta[1]
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
1156
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
1157 fulltext = btext[0] = _textfromdelta(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
1158 revlog,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
1159 baserev,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
1160 delta,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
1161 revinfo.p1,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
1162 revinfo.p2,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
1163 revinfo.flags,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
1164 revinfo.node,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
1165 )
39331
fd0150a3c2fe revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents: 39330
diff changeset
1166 return fulltext
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
1167
51020
509f0f7fc89e delta-computer: stop explicitly taking file handle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50929
diff changeset
1168 def _builddeltadiff(self, base, revinfo):
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
1169 revlog = self.revlog
51020
509f0f7fc89e delta-computer: stop explicitly taking file handle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50929
diff changeset
1170 t = self.buildtext(revinfo)
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
1171 if revlog.iscensored(base):
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
1172 # deltas based on a censored revision must replace the
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
1173 # full content in one patch, so delta works everywhere
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
1174 header = mdiff.replacediffheader(revlog.rawsize(base), len(t))
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
1175 delta = header + t
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
1176 else:
51020
509f0f7fc89e delta-computer: stop explicitly taking file handle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50929
diff changeset
1177 ptext = revlog.rawdata(base)
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
1178 delta = mdiff.textdiff(ptext, t)
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
1179
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
1180 return delta
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
1181
51020
509f0f7fc89e delta-computer: stop explicitly taking file handle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50929
diff changeset
1182 def _builddeltainfo(self, revinfo, base, target_rev=None):
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
1183 # can we use the cached delta?
42465
6e9ba867a946 delta: move some delta chain related computation earlier in deltainfo
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents: 42464
diff changeset
1184 revlog = self.revlog
6e9ba867a946 delta: move some delta chain related computation earlier in deltainfo
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents: 42464
diff changeset
1185 chainbase = revlog.chainbase(base)
51046
f636103c4d67 revlog: remove legacy usage of `_generaldelta`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51045
diff changeset
1186 if revlog.delta_config.general_delta:
42465
6e9ba867a946 delta: move some delta chain related computation earlier in deltainfo
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents: 42464
diff changeset
1187 deltabase = base
6e9ba867a946 delta: move some delta chain related computation earlier in deltainfo
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents: 42464
diff changeset
1188 else:
50356
2a6949ab9d23 delta-find: add a simple safeguard to prevent bad non-general-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50355
diff changeset
1189 if target_rev is not None and base != target_rev - 1:
2a6949ab9d23 delta-find: add a simple safeguard to prevent bad non-general-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50355
diff changeset
1190 msg = (
2a6949ab9d23 delta-find: add a simple safeguard to prevent bad non-general-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50355
diff changeset
1191 b'general delta cannot use delta for something else '
2a6949ab9d23 delta-find: add a simple safeguard to prevent bad non-general-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50355
diff changeset
1192 b'than `prev`: %d<-%d'
2a6949ab9d23 delta-find: add a simple safeguard to prevent bad non-general-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50355
diff changeset
1193 )
2a6949ab9d23 delta-find: add a simple safeguard to prevent bad non-general-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50355
diff changeset
1194 msg %= (base, target_rev)
2a6949ab9d23 delta-find: add a simple safeguard to prevent bad non-general-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50355
diff changeset
1195 raise error.ProgrammingError(msg)
42465
6e9ba867a946 delta: move some delta chain related computation earlier in deltainfo
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents: 42464
diff changeset
1196 deltabase = chainbase
6e9ba867a946 delta: move some delta chain related computation earlier in deltainfo
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents: 42464
diff changeset
1197 snapshotdepth = None
51060
f71f07a679b4 revlog: remove legacy usage of `_sparserevlog`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51058
diff changeset
1198 if revlog.delta_config.sparse_revlog and deltabase == nullrev:
42465
6e9ba867a946 delta: move some delta chain related computation earlier in deltainfo
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents: 42464
diff changeset
1199 snapshotdepth = 0
51060
f71f07a679b4 revlog: remove legacy usage of `_sparserevlog`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51058
diff changeset
1200 elif revlog.delta_config.sparse_revlog and revlog.issnapshot(deltabase):
42465
6e9ba867a946 delta: move some delta chain related computation earlier in deltainfo
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents: 42464
diff changeset
1201 # A delta chain should always be one full snapshot,
6e9ba867a946 delta: move some delta chain related computation earlier in deltainfo
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents: 42464
diff changeset
1202 # zero or more semi-snapshots, and zero or more deltas
6e9ba867a946 delta: move some delta chain related computation earlier in deltainfo
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents: 42464
diff changeset
1203 p1, p2 = revlog.rev(revinfo.p1), revlog.rev(revinfo.p2)
6e9ba867a946 delta: move some delta chain related computation earlier in deltainfo
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents: 42464
diff changeset
1204 if deltabase not in (p1, p2) and revlog.issnapshot(deltabase):
6e9ba867a946 delta: move some delta chain related computation earlier in deltainfo
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents: 42464
diff changeset
1205 snapshotdepth = len(revlog._deltachain(deltabase)[0])
39595
a911932d5003 revlog: reuse cached delta for identical base revision (issue5975)
Boris Feld <boris.feld@octobus.net>
parents: 39594
diff changeset
1206 delta = None
a911932d5003 revlog: reuse cached delta for identical base revision (issue5975)
Boris Feld <boris.feld@octobus.net>
parents: 39594
diff changeset
1207 if revinfo.cachedelta:
49675
0fca63953810 find-delta: minor preparatory change
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49674
diff changeset
1208 cachebase = revinfo.cachedelta[0]
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
1209 # check if the diff still apply
39595
a911932d5003 revlog: reuse cached delta for identical base revision (issue5975)
Boris Feld <boris.feld@octobus.net>
parents: 39594
diff changeset
1210 currentbase = cachebase
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
1211 while (
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
1212 currentbase != nullrev
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
1213 and currentbase != base
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
1214 and self.revlog.length(currentbase) == 0
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
1215 ):
39595
a911932d5003 revlog: reuse cached delta for identical base revision (issue5975)
Boris Feld <boris.feld@octobus.net>
parents: 39594
diff changeset
1216 currentbase = self.revlog.deltaparent(currentbase)
51067
26dcdbe15024 revlog: remove legacy usage of `_lazydeltabase`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51063
diff changeset
1217 if self.revlog.delta_config.lazy_delta and currentbase == base:
39595
a911932d5003 revlog: reuse cached delta for identical base revision (issue5975)
Boris Feld <boris.feld@octobus.net>
parents: 39594
diff changeset
1218 delta = revinfo.cachedelta[1]
a911932d5003 revlog: reuse cached delta for identical base revision (issue5975)
Boris Feld <boris.feld@octobus.net>
parents: 39594
diff changeset
1219 if delta is None:
51020
509f0f7fc89e delta-computer: stop explicitly taking file handle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50929
diff changeset
1220 delta = self._builddeltadiff(base, revinfo)
50649
c84cc0ac77e4 delta-fine: use the `_debug_search` attribute directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50648
diff changeset
1221 if self._debug_search:
49228
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1222 msg = b"DBG-DELTAS-SEARCH: uncompressed-delta-size=%d\n"
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1223 msg %= len(delta)
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1224 self._write_debug(msg)
42467
c1c1872d25d1 deltas: skip if projected compressed size does not match text size constraint
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents: 42466
diff changeset
1225 # snapshotdept need to be neither None nor 0 level snapshot
51083
5645524c6b62 revlog: also migrates `revlog.upperboundcomp` to ConfigClass
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51067
diff changeset
1226 if revlog.delta_config.upper_bound_comp is not None and snapshotdepth:
5645524c6b62 revlog: also migrates `revlog.upperboundcomp` to ConfigClass
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51067
diff changeset
1227 lowestrealisticdeltalen = (
5645524c6b62 revlog: also migrates `revlog.upperboundcomp` to ConfigClass
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51067
diff changeset
1228 len(delta) // revlog.delta_config.upper_bound_comp
5645524c6b62 revlog: also migrates `revlog.upperboundcomp` to ConfigClass
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51067
diff changeset
1229 )
42467
c1c1872d25d1 deltas: skip if projected compressed size does not match text size constraint
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents: 42466
diff changeset
1230 snapshotlimit = revinfo.textlen >> snapshotdepth
50649
c84cc0ac77e4 delta-fine: use the `_debug_search` attribute directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50648
diff changeset
1231 if self._debug_search:
49228
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1232 msg = b"DBG-DELTAS-SEARCH: projected-lower-size=%d\n"
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1233 msg %= lowestrealisticdeltalen
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1234 self._write_debug(msg)
42467
c1c1872d25d1 deltas: skip if projected compressed size does not match text size constraint
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents: 42466
diff changeset
1235 if snapshotlimit < lowestrealisticdeltalen:
50649
c84cc0ac77e4 delta-fine: use the `_debug_search` attribute directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50648
diff changeset
1236 if self._debug_search:
49228
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1237 msg = b"DBG-DELTAS-SEARCH: DISCARDED (snapshot limit)\n"
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1238 self._write_debug(msg)
42467
c1c1872d25d1 deltas: skip if projected compressed size does not match text size constraint
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents: 42466
diff changeset
1239 return None
42468
9b5fbe5ead89 deltas: skip if projected compressed size is bigger than previous snapshot
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents: 42467
diff changeset
1240 if revlog.length(base) < lowestrealisticdeltalen:
50649
c84cc0ac77e4 delta-fine: use the `_debug_search` attribute directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50648
diff changeset
1241 if self._debug_search:
49228
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1242 msg = b"DBG-DELTAS-SEARCH: DISCARDED (prev size)\n"
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1243 self._write_debug(msg)
42468
9b5fbe5ead89 deltas: skip if projected compressed size is bigger than previous snapshot
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents: 42467
diff changeset
1244 return None
51091
a82704902db8 revlog: move the compression/decompression logic on the inner object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51083
diff changeset
1245 header, data = revlog._inner.compress(delta)
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
1246 deltalen = len(header) + len(data)
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
1247 offset = revlog.end(len(revlog) - 1)
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
1248 dist = deltalen + offset - revlog.start(chainbase)
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
1249 chainlen, compresseddeltalen = revlog._chaininfo(base)
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
1250 chainlen += 1
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
1251 compresseddeltalen += deltalen
39154
e0da43e2f71f revlog: compute snapshot depth on delta info
Boris Feld <boris.feld@octobus.net>
parents: 39152
diff changeset
1252
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
1253 return _deltainfo(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
1254 dist,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
1255 deltalen,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
1256 (header, data),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
1257 deltabase,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
1258 chainbase,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
1259 chainlen,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
1260 compresseddeltalen,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
1261 snapshotdepth,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
1262 )
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
1263
51020
509f0f7fc89e delta-computer: stop explicitly taking file handle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50929
diff changeset
1264 def _fullsnapshotinfo(self, revinfo, curr):
509f0f7fc89e delta-computer: stop explicitly taking file handle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50929
diff changeset
1265 rawtext = self.buildtext(revinfo)
51091
a82704902db8 revlog: move the compression/decompression logic on the inner object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51083
diff changeset
1266 data = self.revlog._inner.compress(rawtext)
39333
5d343a24bff5 revlogdeltas: always return a delta info object in finddeltainfo
Boris Feld <boris.feld@octobus.net>
parents: 39332
diff changeset
1267 compresseddeltalen = deltalen = dist = len(data[1]) + len(data[0])
5d343a24bff5 revlogdeltas: always return a delta info object in finddeltainfo
Boris Feld <boris.feld@octobus.net>
parents: 39332
diff changeset
1268 deltabase = chainbase = curr
5d343a24bff5 revlogdeltas: always return a delta info object in finddeltainfo
Boris Feld <boris.feld@octobus.net>
parents: 39332
diff changeset
1269 snapshotdepth = 0
5d343a24bff5 revlogdeltas: always return a delta info object in finddeltainfo
Boris Feld <boris.feld@octobus.net>
parents: 39332
diff changeset
1270 chainlen = 1
5d343a24bff5 revlogdeltas: always return a delta info object in finddeltainfo
Boris Feld <boris.feld@octobus.net>
parents: 39332
diff changeset
1271
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
1272 return _deltainfo(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
1273 dist,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
1274 deltalen,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
1275 data,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
1276 deltabase,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
1277 chainbase,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
1278 chainlen,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
1279 compresseddeltalen,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
1280 snapshotdepth,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
1281 )
39333
5d343a24bff5 revlogdeltas: always return a delta info object in finddeltainfo
Boris Feld <boris.feld@octobus.net>
parents: 39332
diff changeset
1282
51020
509f0f7fc89e delta-computer: stop explicitly taking file handle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50929
diff changeset
1283 def finddeltainfo(self, revinfo, excluded_bases=None, target_rev=None):
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
1284 """Find an acceptable delta against a candidate revision
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
1285
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
1286 revinfo: information about the revision (instance of _revisioninfo)
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
1287
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
1288 Returns the first acceptable candidate revision, as ordered by
39334
507f5b1dd7c8 revlogdeltas: extract _getcandidaterevs in a function
Boris Feld <boris.feld@octobus.net>
parents: 39333
diff changeset
1289 _candidategroups
39333
5d343a24bff5 revlogdeltas: always return a delta info object in finddeltainfo
Boris Feld <boris.feld@octobus.net>
parents: 39332
diff changeset
1290
5d343a24bff5 revlogdeltas: always return a delta info object in finddeltainfo
Boris Feld <boris.feld@octobus.net>
parents: 39332
diff changeset
1291 If no suitable deltabase is found, we return delta info for a full
5d343a24bff5 revlogdeltas: always return a delta info object in finddeltainfo
Boris Feld <boris.feld@octobus.net>
parents: 39332
diff changeset
1292 snapshot.
47401
1efe3cdef53a revlog: add a ways to blacklist some revision when searching for a delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47399
diff changeset
1293
1efe3cdef53a revlog: add a ways to blacklist some revision when searching for a delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47399
diff changeset
1294 `excluded_bases` is an optional set of revision that cannot be used as
1efe3cdef53a revlog: add a ways to blacklist some revision when searching for a delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47399
diff changeset
1295 a delta base. Use this to recompute delta suitable in censor or strip
1efe3cdef53a revlog: add a ways to blacklist some revision when searching for a delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47399
diff changeset
1296 context.
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
1297 """
47456
93f4e183b3f5 deltas: at a `target_rev` parameter to finddeltainfo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47452
diff changeset
1298 if target_rev is None:
47457
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47456
diff changeset
1299 target_rev = len(self.revlog)
47456
93f4e183b3f5 deltas: at a `target_rev` parameter to finddeltainfo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47452
diff changeset
1300
50653
f930b1b1d671 delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50652
diff changeset
1301 gather_debug = self._gather_debug
f930b1b1d671 delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50652
diff changeset
1302 cachedelta = revinfo.cachedelta
f930b1b1d671 delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50652
diff changeset
1303 revlog = self.revlog
50657
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50655
diff changeset
1304 p1r = p2r = None
50653
f930b1b1d671 delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50652
diff changeset
1305
50657
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50655
diff changeset
1306 if excluded_bases is None:
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50655
diff changeset
1307 excluded_bases = set()
50653
f930b1b1d671 delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50652
diff changeset
1308
f930b1b1d671 delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50652
diff changeset
1309 if gather_debug:
f930b1b1d671 delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50652
diff changeset
1310 start = util.timer()
f930b1b1d671 delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50652
diff changeset
1311 dbg = self._one_dbg_data()
f930b1b1d671 delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50652
diff changeset
1312 dbg['revision'] = target_rev
f930b1b1d671 delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50652
diff changeset
1313 target_revlog = b"UNKNOWN"
f930b1b1d671 delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50652
diff changeset
1314 target_type = self.revlog.target[0]
f930b1b1d671 delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50652
diff changeset
1315 target_key = self.revlog.target[1]
f930b1b1d671 delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50652
diff changeset
1316 if target_type == KIND_CHANGELOG:
f930b1b1d671 delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50652
diff changeset
1317 target_revlog = b'CHANGELOG:'
f930b1b1d671 delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50652
diff changeset
1318 elif target_type == KIND_MANIFESTLOG:
f930b1b1d671 delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50652
diff changeset
1319 target_revlog = b'MANIFESTLOG:'
f930b1b1d671 delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50652
diff changeset
1320 if target_key:
f930b1b1d671 delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50652
diff changeset
1321 target_revlog += b'%s:' % target_key
f930b1b1d671 delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50652
diff changeset
1322 elif target_type == KIND_FILELOG:
f930b1b1d671 delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50652
diff changeset
1323 target_revlog = b'FILELOG:'
f930b1b1d671 delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50652
diff changeset
1324 if target_key:
f930b1b1d671 delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50652
diff changeset
1325 target_revlog += b'%s:' % target_key
f930b1b1d671 delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50652
diff changeset
1326 dbg['target-revlog'] = target_revlog
f930b1b1d671 delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50652
diff changeset
1327 p1r = revlog.rev(revinfo.p1)
f930b1b1d671 delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50652
diff changeset
1328 p2r = revlog.rev(revinfo.p2)
f930b1b1d671 delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50652
diff changeset
1329 if p1r != nullrev:
f930b1b1d671 delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50652
diff changeset
1330 p1_chain_len = revlog._chaininfo(p1r)[0]
f930b1b1d671 delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50652
diff changeset
1331 else:
f930b1b1d671 delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50652
diff changeset
1332 p1_chain_len = -1
f930b1b1d671 delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50652
diff changeset
1333 if p2r != nullrev:
f930b1b1d671 delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50652
diff changeset
1334 p2_chain_len = revlog._chaininfo(p2r)[0]
f930b1b1d671 delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50652
diff changeset
1335 else:
f930b1b1d671 delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50652
diff changeset
1336 p2_chain_len = -1
f930b1b1d671 delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50652
diff changeset
1337 dbg['p1-chain-len'] = p1_chain_len
f930b1b1d671 delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50652
diff changeset
1338 dbg['p2-chain-len'] = p2_chain_len
f930b1b1d671 delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50652
diff changeset
1339
50654
bfb6404089a5 delta-find: gather the condition to blindly use a full snapshot together
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50653
diff changeset
1340 # 1) if the revision is empty, no amount of delta can beat it
bfb6404089a5 delta-find: gather the condition to blindly use a full snapshot together
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50653
diff changeset
1341 #
bfb6404089a5 delta-find: gather the condition to blindly use a full snapshot together
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50653
diff changeset
1342 # 2) no delta for flag processor revision (see "candelta" for why)
bfb6404089a5 delta-find: gather the condition to blindly use a full snapshot together
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50653
diff changeset
1343 # not calling candelta since only one revision needs test, also to
bfb6404089a5 delta-find: gather the condition to blindly use a full snapshot together
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50653
diff changeset
1344 # avoid overhead fetching flags again.
bfb6404089a5 delta-find: gather the condition to blindly use a full snapshot together
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50653
diff changeset
1345 if not revinfo.textlen or revinfo.flags & REVIDX_RAWTEXT_CHANGING_FLAGS:
51020
509f0f7fc89e delta-computer: stop explicitly taking file handle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50929
diff changeset
1346 deltainfo = self._fullsnapshotinfo(revinfo, target_rev)
50655
485c9410b75a deltafind: issue debug information when we fast-path rivial case too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50654
diff changeset
1347 if gather_debug:
485c9410b75a deltafind: issue debug information when we fast-path rivial case too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50654
diff changeset
1348 end = util.timer()
485c9410b75a deltafind: issue debug information when we fast-path rivial case too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50654
diff changeset
1349 dbg['duration'] = end - start
485c9410b75a deltafind: issue debug information when we fast-path rivial case too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50654
diff changeset
1350 dbg[
485c9410b75a deltafind: issue debug information when we fast-path rivial case too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50654
diff changeset
1351 'delta-base'
485c9410b75a deltafind: issue debug information when we fast-path rivial case too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50654
diff changeset
1352 ] = deltainfo.base # pytype: disable=attribute-error
485c9410b75a deltafind: issue debug information when we fast-path rivial case too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50654
diff changeset
1353 dbg['search_round_count'] = 0
50657
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50655
diff changeset
1354 dbg['using-cached-base'] = False
50655
485c9410b75a deltafind: issue debug information when we fast-path rivial case too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50654
diff changeset
1355 dbg['delta_try_count'] = 0
485c9410b75a deltafind: issue debug information when we fast-path rivial case too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50654
diff changeset
1356 dbg['type'] = b"full"
485c9410b75a deltafind: issue debug information when we fast-path rivial case too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50654
diff changeset
1357 dbg['snapshot-depth'] = 0
485c9410b75a deltafind: issue debug information when we fast-path rivial case too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50654
diff changeset
1358 self._dbg_process_data(dbg)
485c9410b75a deltafind: issue debug information when we fast-path rivial case too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50654
diff changeset
1359 return deltainfo
39085
dbb3e9e44fce revlog: do not search for delta for empty content
Boris Feld <boris.feld@octobus.net>
parents: 39084
diff changeset
1360
50657
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50655
diff changeset
1361 deltainfo = None
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50655
diff changeset
1362
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50655
diff changeset
1363 # If this source delta are to be forcibly reuse, let us comply early.
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50655
diff changeset
1364 if (
51046
f636103c4d67 revlog: remove legacy usage of `_generaldelta`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51045
diff changeset
1365 revlog.delta_config.general_delta
50657
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50655
diff changeset
1366 and revinfo.cachedelta is not None
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50655
diff changeset
1367 and revinfo.cachedelta[2] == DELTA_BASE_REUSE_FORCE
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50655
diff changeset
1368 ):
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50655
diff changeset
1369 base = revinfo.cachedelta[0]
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50655
diff changeset
1370 if base == nullrev:
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50655
diff changeset
1371 dbg_type = b"full"
51020
509f0f7fc89e delta-computer: stop explicitly taking file handle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50929
diff changeset
1372 deltainfo = self._fullsnapshotinfo(revinfo, target_rev)
50657
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50655
diff changeset
1373 if gather_debug:
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50655
diff changeset
1374 snapshotdepth = 0
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50655
diff changeset
1375 elif base not in excluded_bases:
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50655
diff changeset
1376 delta = revinfo.cachedelta[1]
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50655
diff changeset
1377 header, data = revlog.compress(delta)
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50655
diff changeset
1378 deltalen = len(header) + len(data)
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50655
diff changeset
1379 if gather_debug:
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50655
diff changeset
1380 offset = revlog.end(len(revlog) - 1)
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50655
diff changeset
1381 chainbase = revlog.chainbase(base)
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50655
diff changeset
1382 distance = deltalen + offset - revlog.start(chainbase)
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50655
diff changeset
1383 chainlen, compresseddeltalen = revlog._chaininfo(base)
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50655
diff changeset
1384 chainlen += 1
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50655
diff changeset
1385 compresseddeltalen += deltalen
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50655
diff changeset
1386 if base == p1r or base == p2r:
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50655
diff changeset
1387 dbg_type = b"delta"
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50655
diff changeset
1388 snapshotdepth = None
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50655
diff changeset
1389 elif not revlog.issnapshot(base):
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50655
diff changeset
1390 snapshotdepth = None
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50655
diff changeset
1391 else:
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50655
diff changeset
1392 dbg_type = b"snapshot"
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50655
diff changeset
1393 snapshotdepth = revlog.snapshotdepth(base) + 1
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50655
diff changeset
1394 else:
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50655
diff changeset
1395 distance = None
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50655
diff changeset
1396 chainbase = None
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50655
diff changeset
1397 chainlen = None
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50655
diff changeset
1398 compresseddeltalen = None
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50655
diff changeset
1399 snapshotdepth = None
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50655
diff changeset
1400 deltainfo = _deltainfo(
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50655
diff changeset
1401 distance=distance,
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50655
diff changeset
1402 deltalen=deltalen,
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50655
diff changeset
1403 data=(header, data),
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50655
diff changeset
1404 base=base,
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50655
diff changeset
1405 chainbase=chainbase,
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50655
diff changeset
1406 chainlen=chainlen,
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50655
diff changeset
1407 compresseddeltalen=compresseddeltalen,
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50655
diff changeset
1408 snapshotdepth=snapshotdepth,
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50655
diff changeset
1409 )
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50655
diff changeset
1410
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50655
diff changeset
1411 if deltainfo is not None:
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50655
diff changeset
1412 if gather_debug:
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50655
diff changeset
1413 end = util.timer()
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50655
diff changeset
1414 dbg['duration'] = end - start
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50655
diff changeset
1415 dbg[
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50655
diff changeset
1416 'delta-base'
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50655
diff changeset
1417 ] = deltainfo.base # pytype: disable=attribute-error
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50655
diff changeset
1418 dbg['search_round_count'] = 0
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50655
diff changeset
1419 dbg['using-cached-base'] = True
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50655
diff changeset
1420 dbg['delta_try_count'] = 0
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50655
diff changeset
1421 dbg['type'] = b"full"
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50655
diff changeset
1422 if snapshotdepth is None:
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50655
diff changeset
1423 dbg['snapshot-depth'] = 0
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50655
diff changeset
1424 else:
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50655
diff changeset
1425 dbg['snapshot-depth'] = snapshotdepth
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50655
diff changeset
1426 self._dbg_process_data(dbg)
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50655
diff changeset
1427 return deltainfo
47401
1efe3cdef53a revlog: add a ways to blacklist some revision when searching for a delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47399
diff changeset
1428
49226
e6b7c6fbeb48 deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48946
diff changeset
1429 # count the number of different delta we tried (for debug purpose)
e6b7c6fbeb48 deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48946
diff changeset
1430 dbg_try_count = 0
e6b7c6fbeb48 deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48946
diff changeset
1431 # count the number of "search round" we did. (for debug purpose)
e6b7c6fbeb48 deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48946
diff changeset
1432 dbg_try_rounds = 0
e6b7c6fbeb48 deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48946
diff changeset
1433 dbg_type = b'unknown'
e6b7c6fbeb48 deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48946
diff changeset
1434
50653
f930b1b1d671 delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50652
diff changeset
1435 if p1r is None:
f930b1b1d671 delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50652
diff changeset
1436 p1r = revlog.rev(revinfo.p1)
f930b1b1d671 delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50652
diff changeset
1437 p2r = revlog.rev(revinfo.p2)
49226
e6b7c6fbeb48 deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48946
diff changeset
1438
50649
c84cc0ac77e4 delta-fine: use the `_debug_search` attribute directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50648
diff changeset
1439 if self._debug_search:
49228
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1440 msg = b"DBG-DELTAS-SEARCH: SEARCH rev=%d\n"
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1441 msg %= target_rev
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1442 self._write_debug(msg)
49226
e6b7c6fbeb48 deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48946
diff changeset
1443
51320
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
1444 search = _DeltaSearch(
49611
2afee2176775 delta-find: expand a function definition and call before extendin it
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49610
diff changeset
1445 self.revlog,
2afee2176775 delta-find: expand a function definition and call before extendin it
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49610
diff changeset
1446 revinfo.textlen,
2afee2176775 delta-find: expand a function definition and call before extendin it
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49610
diff changeset
1447 p1r,
2afee2176775 delta-find: expand a function definition and call before extendin it
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49610
diff changeset
1448 p2r,
2afee2176775 delta-find: expand a function definition and call before extendin it
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49610
diff changeset
1449 cachedelta,
49612
e706bb41fdb3 delta-find: move pre-filtering with other pre-filtering logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49611
diff changeset
1450 excluded_bases,
e706bb41fdb3 delta-find: move pre-filtering with other pre-filtering logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49611
diff changeset
1451 target_rev,
49681
c261a628e525 delta-find: use a single snapshot cache when applying a group to an object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49680
diff changeset
1452 snapshot_cache=self._snapshot_cache,
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42992
diff changeset
1453 )
51320
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
1454
c82e03b102a6 delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51319
diff changeset
1455 groups = search.candidate_groups()
39497
5b308a4e6d03 snapshot: use None as a stop value when looking for a good delta
Boris Feld <boris.feld@octobus.net>
parents: 39496
diff changeset
1456 candidaterevs = next(groups)
5b308a4e6d03 snapshot: use None as a stop value when looking for a good delta
Boris Feld <boris.feld@octobus.net>
parents: 39496
diff changeset
1457 while candidaterevs is not None:
49226
e6b7c6fbeb48 deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48946
diff changeset
1458 dbg_try_rounds += 1
50649
c84cc0ac77e4 delta-fine: use the `_debug_search` attribute directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50648
diff changeset
1459 if self._debug_search:
49228
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1460 prev = None
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1461 if deltainfo is not None:
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1462 prev = deltainfo.base
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1463
49608
78ba41878f2e delta-find: add debug information about reuse of cached data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49228
diff changeset
1464 if (
78ba41878f2e delta-find: add debug information about reuse of cached data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49228
diff changeset
1465 cachedelta is not None
78ba41878f2e delta-find: add debug information about reuse of cached data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49228
diff changeset
1466 and len(candidaterevs) == 1
78ba41878f2e delta-find: add debug information about reuse of cached data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49228
diff changeset
1467 and cachedelta[0] in candidaterevs
78ba41878f2e delta-find: add debug information about reuse of cached data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49228
diff changeset
1468 ):
78ba41878f2e delta-find: add debug information about reuse of cached data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49228
diff changeset
1469 round_type = b"cached-delta"
50652
ebb292ffdf4d delta-find: fix `parents` round detection
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50651
diff changeset
1470 elif p1r in candidaterevs or p2r in candidaterevs:
49228
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1471 round_type = b"parents"
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1472 elif prev is not None and all(c < prev for c in candidaterevs):
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1473 round_type = b"refine-down"
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1474 elif prev is not None and all(c > prev for c in candidaterevs):
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1475 round_type = b"refine-up"
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1476 else:
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1477 round_type = b"search-down"
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1478 msg = b"DBG-DELTAS-SEARCH: ROUND #%d - %d candidates - %s\n"
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1479 msg %= (dbg_try_rounds, len(candidaterevs), round_type)
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1480 self._write_debug(msg)
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
1481 nominateddeltas = []
39498
04b75f3a3f2a snapshot: add refining logic at the findeltainfo level
Boris Feld <boris.feld@octobus.net>
parents: 39497
diff changeset
1482 if deltainfo is not None:
50649
c84cc0ac77e4 delta-fine: use the `_debug_search` attribute directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50648
diff changeset
1483 if self._debug_search:
49228
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1484 msg = (
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1485 b"DBG-DELTAS-SEARCH: CONTENDER: rev=%d - length=%d\n"
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1486 )
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1487 msg %= (deltainfo.base, deltainfo.deltalen)
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1488 self._write_debug(msg)
39498
04b75f3a3f2a snapshot: add refining logic at the findeltainfo level
Boris Feld <boris.feld@octobus.net>
parents: 39497
diff changeset
1489 # if we already found a good delta,
04b75f3a3f2a snapshot: add refining logic at the findeltainfo level
Boris Feld <boris.feld@octobus.net>
parents: 39497
diff changeset
1490 # challenge it against refined candidates
04b75f3a3f2a snapshot: add refining logic at the findeltainfo level
Boris Feld <boris.feld@octobus.net>
parents: 39497
diff changeset
1491 nominateddeltas.append(deltainfo)
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
1492 for candidaterev in candidaterevs:
50649
c84cc0ac77e4 delta-fine: use the `_debug_search` attribute directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50648
diff changeset
1493 if self._debug_search:
49228
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1494 msg = b"DBG-DELTAS-SEARCH: CANDIDATE: rev=%d\n"
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1495 msg %= candidaterev
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1496 self._write_debug(msg)
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1497 candidate_type = None
50652
ebb292ffdf4d delta-find: fix `parents` round detection
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50651
diff changeset
1498 if candidaterev == p1r:
49228
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1499 candidate_type = b"p1"
50652
ebb292ffdf4d delta-find: fix `parents` round detection
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50651
diff changeset
1500 elif candidaterev == p2r:
49228
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1501 candidate_type = b"p2"
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1502 elif self.revlog.issnapshot(candidaterev):
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1503 candidate_type = b"snapshot-%d"
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1504 candidate_type %= self.revlog.snapshotdepth(
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1505 candidaterev
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1506 )
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1507
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1508 if candidate_type is not None:
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1509 msg = b"DBG-DELTAS-SEARCH: type=%s\n"
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1510 msg %= candidate_type
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1511 self._write_debug(msg)
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1512 msg = b"DBG-DELTAS-SEARCH: size=%d\n"
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1513 msg %= self.revlog.length(candidaterev)
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1514 self._write_debug(msg)
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1515 msg = b"DBG-DELTAS-SEARCH: base=%d\n"
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1516 msg %= self.revlog.deltaparent(candidaterev)
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1517 self._write_debug(msg)
49612
e706bb41fdb3 delta-find: move pre-filtering with other pre-filtering logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49611
diff changeset
1518
49226
e6b7c6fbeb48 deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48946
diff changeset
1519 dbg_try_count += 1
49228
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1520
50649
c84cc0ac77e4 delta-fine: use the `_debug_search` attribute directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50648
diff changeset
1521 if self._debug_search:
49228
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1522 delta_start = util.timer()
50356
2a6949ab9d23 delta-find: add a simple safeguard to prevent bad non-general-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50355
diff changeset
1523 candidatedelta = self._builddeltainfo(
2a6949ab9d23 delta-find: add a simple safeguard to prevent bad non-general-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50355
diff changeset
1524 revinfo,
2a6949ab9d23 delta-find: add a simple safeguard to prevent bad non-general-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50355
diff changeset
1525 candidaterev,
2a6949ab9d23 delta-find: add a simple safeguard to prevent bad non-general-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50355
diff changeset
1526 target_rev=target_rev,
2a6949ab9d23 delta-find: add a simple safeguard to prevent bad non-general-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50355
diff changeset
1527 )
50649
c84cc0ac77e4 delta-fine: use the `_debug_search` attribute directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50648
diff changeset
1528 if self._debug_search:
49228
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1529 delta_end = util.timer()
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1530 msg = b"DBG-DELTAS-SEARCH: delta-search-time=%f\n"
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1531 msg %= delta_end - delta_start
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1532 self._write_debug(msg)
42466
465f2d0df9ae deltas: accept and skip None return for delta info
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents: 42465
diff changeset
1533 if candidatedelta is not None:
49674
5af4a0a73e4c find-delta: rename _isgooddeltainfo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49657
diff changeset
1534 if is_good_delta_info(self.revlog, candidatedelta, revinfo):
50649
c84cc0ac77e4 delta-fine: use the `_debug_search` attribute directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50648
diff changeset
1535 if self._debug_search:
49228
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1536 msg = b"DBG-DELTAS-SEARCH: DELTA: length=%d (GOOD)\n"
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1537 msg %= candidatedelta.deltalen
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1538 self._write_debug(msg)
42466
465f2d0df9ae deltas: accept and skip None return for delta info
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents: 42465
diff changeset
1539 nominateddeltas.append(candidatedelta)
50649
c84cc0ac77e4 delta-fine: use the `_debug_search` attribute directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50648
diff changeset
1540 elif self._debug_search:
49228
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1541 msg = b"DBG-DELTAS-SEARCH: DELTA: length=%d (BAD)\n"
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1542 msg %= candidatedelta.deltalen
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1543 self._write_debug(msg)
50649
c84cc0ac77e4 delta-fine: use the `_debug_search` attribute directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50648
diff changeset
1544 elif self._debug_search:
49228
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1545 msg = b"DBG-DELTAS-SEARCH: NO-DELTA\n"
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1546 self._write_debug(msg)
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
1547 if nominateddeltas:
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
1548 deltainfo = min(nominateddeltas, key=lambda x: x.deltalen)
39498
04b75f3a3f2a snapshot: add refining logic at the findeltainfo level
Boris Feld <boris.feld@octobus.net>
parents: 39497
diff changeset
1549 if deltainfo is not None:
04b75f3a3f2a snapshot: add refining logic at the findeltainfo level
Boris Feld <boris.feld@octobus.net>
parents: 39497
diff changeset
1550 candidaterevs = groups.send(deltainfo.base)
04b75f3a3f2a snapshot: add refining logic at the findeltainfo level
Boris Feld <boris.feld@octobus.net>
parents: 39497
diff changeset
1551 else:
04b75f3a3f2a snapshot: add refining logic at the findeltainfo level
Boris Feld <boris.feld@octobus.net>
parents: 39497
diff changeset
1552 candidaterevs = next(groups)
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
1553
39333
5d343a24bff5 revlogdeltas: always return a delta info object in finddeltainfo
Boris Feld <boris.feld@octobus.net>
parents: 39332
diff changeset
1554 if deltainfo is None:
49226
e6b7c6fbeb48 deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48946
diff changeset
1555 dbg_type = b"full"
51020
509f0f7fc89e delta-computer: stop explicitly taking file handle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50929
diff changeset
1556 deltainfo = self._fullsnapshotinfo(revinfo, target_rev)
49228
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1557 elif deltainfo.snapshotdepth: # pytype: disable=attribute-error
49226
e6b7c6fbeb48 deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48946
diff changeset
1558 dbg_type = b"snapshot"
e6b7c6fbeb48 deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48946
diff changeset
1559 else:
e6b7c6fbeb48 deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48946
diff changeset
1560 dbg_type = b"delta"
e6b7c6fbeb48 deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48946
diff changeset
1561
49610
35d4c2124073 debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49608
diff changeset
1562 if gather_debug:
49226
e6b7c6fbeb48 deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48946
diff changeset
1563 end = util.timer()
49775
d57b966cdeb1 delta-find: properly report full snapshot used from cache as such
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49768
diff changeset
1564 if dbg_type == b'full':
d57b966cdeb1 delta-find: properly report full snapshot used from cache as such
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49768
diff changeset
1565 used_cached = (
d57b966cdeb1 delta-find: properly report full snapshot used from cache as such
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49768
diff changeset
1566 cachedelta is not None
d57b966cdeb1 delta-find: properly report full snapshot used from cache as such
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49768
diff changeset
1567 and dbg_try_rounds == 0
d57b966cdeb1 delta-find: properly report full snapshot used from cache as such
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49768
diff changeset
1568 and dbg_try_count == 0
d57b966cdeb1 delta-find: properly report full snapshot used from cache as such
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49768
diff changeset
1569 and cachedelta[0] == nullrev
d57b966cdeb1 delta-find: properly report full snapshot used from cache as such
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49768
diff changeset
1570 )
d57b966cdeb1 delta-find: properly report full snapshot used from cache as such
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49768
diff changeset
1571 else:
d57b966cdeb1 delta-find: properly report full snapshot used from cache as such
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49768
diff changeset
1572 used_cached = (
d57b966cdeb1 delta-find: properly report full snapshot used from cache as such
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49768
diff changeset
1573 cachedelta is not None
d57b966cdeb1 delta-find: properly report full snapshot used from cache as such
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49768
diff changeset
1574 and dbg_try_rounds == 1
d57b966cdeb1 delta-find: properly report full snapshot used from cache as such
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49768
diff changeset
1575 and dbg_try_count == 1
d57b966cdeb1 delta-find: properly report full snapshot used from cache as such
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49768
diff changeset
1576 and deltainfo.base == cachedelta[0]
d57b966cdeb1 delta-find: properly report full snapshot used from cache as such
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49768
diff changeset
1577 )
50651
d1dc4a03125e delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50650
diff changeset
1578 dbg['duration'] = end - start
d1dc4a03125e delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50650
diff changeset
1579 dbg[
d1dc4a03125e delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50650
diff changeset
1580 'delta-base'
d1dc4a03125e delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50650
diff changeset
1581 ] = deltainfo.base # pytype: disable=attribute-error
d1dc4a03125e delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50650
diff changeset
1582 dbg['search_round_count'] = dbg_try_rounds
d1dc4a03125e delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50650
diff changeset
1583 dbg['using-cached-base'] = used_cached
d1dc4a03125e delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50650
diff changeset
1584 dbg['delta_try_count'] = dbg_try_count
d1dc4a03125e delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50650
diff changeset
1585 dbg['type'] = dbg_type
49228
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1586 if (
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1587 deltainfo.snapshotdepth # pytype: disable=attribute-error
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1588 is not None
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1589 ):
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1590 dbg[
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1591 'snapshot-depth'
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49226
diff changeset
1592 ] = deltainfo.snapshotdepth # pytype: disable=attribute-error
49226
e6b7c6fbeb48 deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48946
diff changeset
1593 else:
e6b7c6fbeb48 deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48946
diff changeset
1594 dbg['snapshot-depth'] = 0
50650
4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50649
diff changeset
1595 self._dbg_process_data(dbg)
4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50649
diff changeset
1596 return deltainfo
49226
e6b7c6fbeb48 deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48946
diff changeset
1597
50651
d1dc4a03125e delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50650
diff changeset
1598 def _one_dbg_data(self):
d1dc4a03125e delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50650
diff changeset
1599 return {
d1dc4a03125e delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50650
diff changeset
1600 'duration': None,
d1dc4a03125e delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50650
diff changeset
1601 'revision': None,
d1dc4a03125e delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50650
diff changeset
1602 'delta-base': None,
d1dc4a03125e delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50650
diff changeset
1603 'search_round_count': None,
d1dc4a03125e delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50650
diff changeset
1604 'using-cached-base': None,
d1dc4a03125e delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50650
diff changeset
1605 'delta_try_count': None,
d1dc4a03125e delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50650
diff changeset
1606 'type': None,
d1dc4a03125e delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50650
diff changeset
1607 'p1-chain-len': None,
d1dc4a03125e delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50650
diff changeset
1608 'p2-chain-len': None,
d1dc4a03125e delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50650
diff changeset
1609 'snapshot-depth': None,
d1dc4a03125e delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50650
diff changeset
1610 'target-revlog': None,
d1dc4a03125e delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50650
diff changeset
1611 }
d1dc4a03125e delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50650
diff changeset
1612
50650
4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50649
diff changeset
1613 def _dbg_process_data(self, dbg):
4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50649
diff changeset
1614 if self._debug_info is not None:
4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50649
diff changeset
1615 self._debug_info.append(dbg)
49610
35d4c2124073 debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49608
diff changeset
1616
50650
4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50649
diff changeset
1617 if self._write_debug is not None:
4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50649
diff changeset
1618 msg = (
4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50649
diff changeset
1619 b"DBG-DELTAS:"
4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50649
diff changeset
1620 b" %-12s"
4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50649
diff changeset
1621 b" rev=%d:"
4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50649
diff changeset
1622 b" delta-base=%d"
4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50649
diff changeset
1623 b" is-cached=%d"
4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50649
diff changeset
1624 b" - search-rounds=%d"
4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50649
diff changeset
1625 b" try-count=%d"
4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50649
diff changeset
1626 b" - delta-type=%-6s"
4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50649
diff changeset
1627 b" snap-depth=%d"
4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50649
diff changeset
1628 b" - p1-chain-length=%d"
4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50649
diff changeset
1629 b" p2-chain-length=%d"
4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50649
diff changeset
1630 b" - duration=%f"
4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50649
diff changeset
1631 b"\n"
4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50649
diff changeset
1632 )
4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50649
diff changeset
1633 msg %= (
4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50649
diff changeset
1634 dbg["target-revlog"],
4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50649
diff changeset
1635 dbg["revision"],
4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50649
diff changeset
1636 dbg["delta-base"],
4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50649
diff changeset
1637 dbg["using-cached-base"],
4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50649
diff changeset
1638 dbg["search_round_count"],
4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50649
diff changeset
1639 dbg["delta_try_count"],
4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50649
diff changeset
1640 dbg["type"],
4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50649
diff changeset
1641 dbg["snapshot-depth"],
4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50649
diff changeset
1642 dbg["p1-chain-len"],
4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50649
diff changeset
1643 dbg["p2-chain-len"],
4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50649
diff changeset
1644 dbg["duration"],
4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50649
diff changeset
1645 )
4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50649
diff changeset
1646 self._write_debug(msg)
47452
c6844912c327 revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47401
diff changeset
1647
c6844912c327 revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47401
diff changeset
1648
c6844912c327 revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47401
diff changeset
1649 def delta_compression(default_compression_header, deltainfo):
c6844912c327 revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47401
diff changeset
1650 """return (COMPRESSION_MODE, deltainfo)
c6844912c327 revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47401
diff changeset
1651
c6844912c327 revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47401
diff changeset
1652 used by revlog v2+ format to dispatch between PLAIN and DEFAULT
c6844912c327 revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47401
diff changeset
1653 compression.
c6844912c327 revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47401
diff changeset
1654 """
c6844912c327 revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47401
diff changeset
1655 h, d = deltainfo.data
c6844912c327 revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47401
diff changeset
1656 compression_mode = COMP_MODE_INLINE
c6844912c327 revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47401
diff changeset
1657 if not h and not d:
c6844912c327 revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47401
diff changeset
1658 # not data to store at all... declare them uncompressed
c6844912c327 revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47401
diff changeset
1659 compression_mode = COMP_MODE_PLAIN
c6844912c327 revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47401
diff changeset
1660 elif not h:
c6844912c327 revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47401
diff changeset
1661 t = d[0:1]
c6844912c327 revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47401
diff changeset
1662 if t == b'\0':
c6844912c327 revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47401
diff changeset
1663 compression_mode = COMP_MODE_PLAIN
c6844912c327 revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47401
diff changeset
1664 elif t == default_compression_header:
c6844912c327 revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47401
diff changeset
1665 compression_mode = COMP_MODE_DEFAULT
c6844912c327 revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47401
diff changeset
1666 elif h == b'u':
c6844912c327 revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47401
diff changeset
1667 # we have a more efficient way to declare uncompressed
c6844912c327 revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47401
diff changeset
1668 h = b''
c6844912c327 revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47401
diff changeset
1669 compression_mode = COMP_MODE_PLAIN
c6844912c327 revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47401
diff changeset
1670 deltainfo = drop_u_compression(deltainfo)
c6844912c327 revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47401
diff changeset
1671 return compression_mode, deltainfo