Mercurial > hg
annotate mercurial/revlogutils/deltas.py @ 51630:513597087b89
branching: merge stable into default
author | Pierre-Yves David <pierre-yves.david@octobus.net> |
---|---|
date | Mon, 10 Jun 2024 12:12:56 +0200 |
parents | e7bbeaa4f0a7 |
children | 493034cc3265 |
rev | line source |
---|---|
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
1 # revlogdeltas.py - Logic around delta computation for revlog |
8226
8b2cd04a6e97
put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents:
8225
diff
changeset
|
2 # |
46819
d4ba4d51f85f
contributor: change mentions of mpm to olivia
Raphaël Gomès <rgomes@octobus.net>
parents:
43089
diff
changeset
|
3 # Copyright 2005-2007 Olivia Mackall <olivia@selenic.com> |
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
4 # Copyright 2018 Octobus <contact@octobus.net> |
8226
8b2cd04a6e97
put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents:
8225
diff
changeset
|
5 # |
8b2cd04a6e97
put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents:
8225
diff
changeset
|
6 # This software may be used and distributed according to the terms of the |
10263 | 7 # GNU General Public License version 2 or any later version. |
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
8 """Helper class to compute deltas stored inside revlogs""" |
8227
0a9542703300
turn some comments back into module docstrings
Martin Geisler <mg@lazybytes.net>
parents:
8226
diff
changeset
|
9 |
27361
29f50344fa83
revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27251
diff
changeset
|
10 |
51346
2a333d791ecf
delta-find: introduce a base class for _DeltaSearch
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51345
diff
changeset
|
11 import abc |
39493
3ca144f1c8dd
snapshot: search for unrelated but reusable full-snapshot
Boris Feld <boris.feld@octobus.net>
parents:
39492
diff
changeset
|
12 import collections |
27361
29f50344fa83
revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27251
diff
changeset
|
13 import struct |
29f50344fa83
revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27251
diff
changeset
|
14 |
29f50344fa83
revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27251
diff
changeset
|
15 # import stuff from node for others to import from revlog |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
16 from ..node import nullrev |
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
17 from ..i18n import _ |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
18 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
19 from .constants import ( |
47452
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
20 COMP_MODE_DEFAULT, |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
21 COMP_MODE_INLINE, |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
22 COMP_MODE_PLAIN, |
49768
bcae90c53def
delta-find: add a delta-reuse policy that blindly accepts incoming deltas
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49681
diff
changeset
|
23 DELTA_BASE_REUSE_FORCE, |
49677
05db41701ece
find-delta: pass the cache-delta usage policy alongside the cache-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49675
diff
changeset
|
24 DELTA_BASE_REUSE_NO, |
49226
e6b7c6fbeb48
deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
25 KIND_CHANGELOG, |
e6b7c6fbeb48
deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
26 KIND_FILELOG, |
e6b7c6fbeb48
deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
27 KIND_MANIFESTLOG, |
39329
729082bb9938
revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents:
39232
diff
changeset
|
28 REVIDX_ISCENSORED, |
729082bb9938
revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents:
39232
diff
changeset
|
29 REVIDX_RAWTEXT_CHANGING_FLAGS, |
729082bb9938
revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents:
39232
diff
changeset
|
30 ) |
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
31 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
32 from ..thirdparty import attr |
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
33 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
34 from .. import ( |
27361
29f50344fa83
revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27251
diff
changeset
|
35 error, |
29f50344fa83
revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27251
diff
changeset
|
36 mdiff, |
41108
38e88450138c
delta: have a native implementation of _findsnapshot
Boris Feld <boris.feld@octobus.net>
parents:
41079
diff
changeset
|
37 util, |
27361
29f50344fa83
revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27251
diff
changeset
|
38 ) |
10913
f2ecc5733c89
revlog: factor out _maxinline global.
Greg Ward <greg-hg@gerg.ca>
parents:
10404
diff
changeset
|
39 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
40 from . import flagutil |
42992
dff95420480f
flagprocessors: make `processflagsraw` a module level function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42878
diff
changeset
|
41 |
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
42 # maximum <delta-chain-data>/<revision-text-length> ratio |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
43 LIMIT_DELTA2TEXT = 2 |
1091
d62130f99a73
Move hash function back to revlog from node
mpm@selenic.com
parents:
1089
diff
changeset
|
44 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
45 |
48946
642e31cb55f0
py3: use class X: instead of class X(object):
Gregory Szorc <gregory.szorc@gmail.com>
parents:
48875
diff
changeset
|
46 class _testrevlog: |
38637
e33f784f2a44
revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents:
38636
diff
changeset
|
47 """minimalist fake revlog to use in doctests""" |
e33f784f2a44
revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents:
38636
diff
changeset
|
48 |
40641
85b14f0dc334
doctest: add a `issnapshot` method to _testrevlog
Boris Feld <boris.feld@octobus.net>
parents:
40608
diff
changeset
|
49 def __init__(self, data, density=0.5, mingap=0, snapshot=()): |
38637
e33f784f2a44
revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents:
38636
diff
changeset
|
50 """data is an list of revision payload boundaries""" |
51045
127656e0b97b
revlog: use the new Config classes in _testrevlog
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51020
diff
changeset
|
51 from .. import revlog |
127656e0b97b
revlog: use the new Config classes in _testrevlog
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51020
diff
changeset
|
52 |
38637
e33f784f2a44
revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents:
38636
diff
changeset
|
53 self._data = data |
51045
127656e0b97b
revlog: use the new Config classes in _testrevlog
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51020
diff
changeset
|
54 self.data_config = revlog.DataConfig() |
127656e0b97b
revlog: use the new Config classes in _testrevlog
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51020
diff
changeset
|
55 self.data_config.sr_density_threshold = density |
127656e0b97b
revlog: use the new Config classes in _testrevlog
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51020
diff
changeset
|
56 self.data_config.sr_min_gap_size = mingap |
127656e0b97b
revlog: use the new Config classes in _testrevlog
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51020
diff
changeset
|
57 self.delta_config = revlog.DeltaConfig() |
127656e0b97b
revlog: use the new Config classes in _testrevlog
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51020
diff
changeset
|
58 self.feature_config = revlog.FeatureConfig() |
40641
85b14f0dc334
doctest: add a `issnapshot` method to _testrevlog
Boris Feld <boris.feld@octobus.net>
parents:
40608
diff
changeset
|
59 self._snapshot = set(snapshot) |
40709
39d29542fe40
sparse-revlog: put the native implementation of slicechunktodensity to use
Boris Feld <boris.feld@octobus.net>
parents:
40657
diff
changeset
|
60 self.index = None |
38637
e33f784f2a44
revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents:
38636
diff
changeset
|
61 |
e33f784f2a44
revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents:
38636
diff
changeset
|
62 def start(self, rev): |
41079
88d813cd9acd
revlog: fix pure python slicing test when chain contains nullrev
Boris Feld <boris.feld@octobus.net>
parents:
41033
diff
changeset
|
63 if rev == nullrev: |
88d813cd9acd
revlog: fix pure python slicing test when chain contains nullrev
Boris Feld <boris.feld@octobus.net>
parents:
41033
diff
changeset
|
64 return 0 |
38637
e33f784f2a44
revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents:
38636
diff
changeset
|
65 if rev == 0: |
e33f784f2a44
revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents:
38636
diff
changeset
|
66 return 0 |
e33f784f2a44
revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents:
38636
diff
changeset
|
67 return self._data[rev - 1] |
e33f784f2a44
revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents:
38636
diff
changeset
|
68 |
e33f784f2a44
revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents:
38636
diff
changeset
|
69 def end(self, rev): |
41079
88d813cd9acd
revlog: fix pure python slicing test when chain contains nullrev
Boris Feld <boris.feld@octobus.net>
parents:
41033
diff
changeset
|
70 if rev == nullrev: |
88d813cd9acd
revlog: fix pure python slicing test when chain contains nullrev
Boris Feld <boris.feld@octobus.net>
parents:
41033
diff
changeset
|
71 return 0 |
38637
e33f784f2a44
revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents:
38636
diff
changeset
|
72 return self._data[rev] |
e33f784f2a44
revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents:
38636
diff
changeset
|
73 |
e33f784f2a44
revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents:
38636
diff
changeset
|
74 def length(self, rev): |
e33f784f2a44
revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents:
38636
diff
changeset
|
75 return self.end(rev) - self.start(rev) |
e33f784f2a44
revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents:
38636
diff
changeset
|
76 |
38718
f8762ea73e0d
sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents:
38717
diff
changeset
|
77 def __len__(self): |
f8762ea73e0d
sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents:
38717
diff
changeset
|
78 return len(self._data) |
f8762ea73e0d
sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents:
38717
diff
changeset
|
79 |
40641
85b14f0dc334
doctest: add a `issnapshot` method to _testrevlog
Boris Feld <boris.feld@octobus.net>
parents:
40608
diff
changeset
|
80 def issnapshot(self, rev): |
41079
88d813cd9acd
revlog: fix pure python slicing test when chain contains nullrev
Boris Feld <boris.feld@octobus.net>
parents:
41033
diff
changeset
|
81 if rev == nullrev: |
88d813cd9acd
revlog: fix pure python slicing test when chain contains nullrev
Boris Feld <boris.feld@octobus.net>
parents:
41033
diff
changeset
|
82 return True |
40641
85b14f0dc334
doctest: add a `issnapshot` method to _testrevlog
Boris Feld <boris.feld@octobus.net>
parents:
40608
diff
changeset
|
83 return rev in self._snapshot |
85b14f0dc334
doctest: add a `issnapshot` method to _testrevlog
Boris Feld <boris.feld@octobus.net>
parents:
40608
diff
changeset
|
84 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
85 |
40604
3ac23dad6364
sparse-revlog: drop unused deltainfo parameter from _slicechunktodensity
Boris Feld <boris.feld@octobus.net>
parents:
40603
diff
changeset
|
86 def slicechunk(revlog, revs, targetsize=None): |
34824
e2ad93bcc084
revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents:
34823
diff
changeset
|
87 """slice revs to reduce the amount of unrelated data to be read from disk. |
e2ad93bcc084
revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents:
34823
diff
changeset
|
88 |
e2ad93bcc084
revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents:
34823
diff
changeset
|
89 ``revs`` is sliced into groups that should be read in one time. |
e2ad93bcc084
revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents:
34823
diff
changeset
|
90 Assume that revs are sorted. |
38640
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
91 |
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
92 The initial chunk is sliced until the overall density (payload/chunks-span |
51062
8c614fa16330
revlog: remove legacy usage of `_srdensitythreshold`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51060
diff
changeset
|
93 ratio) is above `revlog.data_config.sr_density_threshold`. No gap smaller |
51063
533d6943f6a3
revlog: remove legacy usage of `_srmingapsize`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51062
diff
changeset
|
94 than `revlog.data_config.sr_min_gap_size` is skipped. |
38640
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
95 |
38643
967fee55e8d9
revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents:
38642
diff
changeset
|
96 If `targetsize` is set, no chunk larger than `targetsize` will be yield. |
967fee55e8d9
revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents:
38642
diff
changeset
|
97 For consistency with other slicing choice, this limit won't go lower than |
51063
533d6943f6a3
revlog: remove legacy usage of `_srmingapsize`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51062
diff
changeset
|
98 `revlog.data_config.sr_min_gap_size`. |
38643
967fee55e8d9
revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents:
38642
diff
changeset
|
99 |
967fee55e8d9
revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents:
38642
diff
changeset
|
100 If individual revisions chunk are larger than this limit, they will still |
967fee55e8d9
revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents:
38642
diff
changeset
|
101 be raised individually. |
967fee55e8d9
revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents:
38642
diff
changeset
|
102 |
40642
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
103 >>> data = [ |
38640
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
104 ... 5, #00 (5) |
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
105 ... 10, #01 (5) |
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
106 ... 12, #02 (2) |
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
107 ... 12, #03 (empty) |
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
108 ... 27, #04 (15) |
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
109 ... 31, #05 (4) |
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
110 ... 31, #06 (empty) |
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
111 ... 42, #07 (11) |
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
112 ... 47, #08 (5) |
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
113 ... 47, #09 (empty) |
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
114 ... 48, #10 (1) |
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
115 ... 51, #11 (3) |
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
116 ... 74, #12 (23) |
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
117 ... 85, #13 (11) |
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
118 ... 86, #14 (1) |
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
119 ... 91, #15 (5) |
40642
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
120 ... ] |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
121 >>> revlog = _testrevlog(data, snapshot=range(16)) |
38640
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
122 |
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
123 >>> list(slicechunk(revlog, list(range(16)))) |
38640
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
124 [[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]] |
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
125 >>> list(slicechunk(revlog, [0, 15])) |
38640
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
126 [[0], [15]] |
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
127 >>> list(slicechunk(revlog, [0, 11, 15])) |
38640
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
128 [[0], [11], [15]] |
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
129 >>> list(slicechunk(revlog, [0, 11, 13, 15])) |
38640
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
130 [[0], [11, 13, 15]] |
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
131 >>> list(slicechunk(revlog, [1, 2, 3, 5, 8, 10, 11, 14])) |
38640
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
132 [[1, 2], [5, 8, 10, 11], [14]] |
38643
967fee55e8d9
revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents:
38642
diff
changeset
|
133 |
967fee55e8d9
revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents:
38642
diff
changeset
|
134 Slicing with a maximum chunk size |
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
135 >>> list(slicechunk(revlog, [0, 11, 13, 15], targetsize=15)) |
38643
967fee55e8d9
revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents:
38642
diff
changeset
|
136 [[0], [11], [13], [15]] |
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
137 >>> list(slicechunk(revlog, [0, 11, 13, 15], targetsize=20)) |
38643
967fee55e8d9
revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents:
38642
diff
changeset
|
138 [[0], [11], [13, 15]] |
41079
88d813cd9acd
revlog: fix pure python slicing test when chain contains nullrev
Boris Feld <boris.feld@octobus.net>
parents:
41033
diff
changeset
|
139 |
88d813cd9acd
revlog: fix pure python slicing test when chain contains nullrev
Boris Feld <boris.feld@octobus.net>
parents:
41033
diff
changeset
|
140 Slicing involving nullrev |
88d813cd9acd
revlog: fix pure python slicing test when chain contains nullrev
Boris Feld <boris.feld@octobus.net>
parents:
41033
diff
changeset
|
141 >>> list(slicechunk(revlog, [-1, 0, 11, 13, 15], targetsize=20)) |
88d813cd9acd
revlog: fix pure python slicing test when chain contains nullrev
Boris Feld <boris.feld@octobus.net>
parents:
41033
diff
changeset
|
142 [[-1, 0], [11], [13, 15]] |
88d813cd9acd
revlog: fix pure python slicing test when chain contains nullrev
Boris Feld <boris.feld@octobus.net>
parents:
41033
diff
changeset
|
143 >>> list(slicechunk(revlog, [-1, 13, 15], targetsize=5)) |
88d813cd9acd
revlog: fix pure python slicing test when chain contains nullrev
Boris Feld <boris.feld@octobus.net>
parents:
41033
diff
changeset
|
144 [[-1], [13], [15]] |
34824
e2ad93bcc084
revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents:
34823
diff
changeset
|
145 """ |
38643
967fee55e8d9
revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents:
38642
diff
changeset
|
146 if targetsize is not None: |
51063
533d6943f6a3
revlog: remove legacy usage of `_srmingapsize`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51062
diff
changeset
|
147 targetsize = max(targetsize, revlog.data_config.sr_min_gap_size) |
38718
f8762ea73e0d
sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents:
38717
diff
changeset
|
148 # targetsize should not be specified when evaluating delta candidates: |
f8762ea73e0d
sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents:
38717
diff
changeset
|
149 # * targetsize is used to ensure we stay within specification when reading, |
40709
39d29542fe40
sparse-revlog: put the native implementation of slicechunktodensity to use
Boris Feld <boris.feld@octobus.net>
parents:
40657
diff
changeset
|
150 densityslicing = getattr(revlog.index, 'slicechunktodensity', None) |
39d29542fe40
sparse-revlog: put the native implementation of slicechunktodensity to use
Boris Feld <boris.feld@octobus.net>
parents:
40657
diff
changeset
|
151 if densityslicing is None: |
39d29542fe40
sparse-revlog: put the native implementation of slicechunktodensity to use
Boris Feld <boris.feld@octobus.net>
parents:
40657
diff
changeset
|
152 densityslicing = lambda x, y, z: _slicechunktodensity(revlog, x, y, z) |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
153 for chunk in densityslicing( |
51063
533d6943f6a3
revlog: remove legacy usage of `_srmingapsize`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51062
diff
changeset
|
154 revs, |
533d6943f6a3
revlog: remove legacy usage of `_srmingapsize`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51062
diff
changeset
|
155 revlog.data_config.sr_density_threshold, |
533d6943f6a3
revlog: remove legacy usage of `_srmingapsize`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51062
diff
changeset
|
156 revlog.data_config.sr_min_gap_size, |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
157 ): |
38643
967fee55e8d9
revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents:
38642
diff
changeset
|
158 for subchunk in _slicechunktosize(revlog, chunk, targetsize): |
967fee55e8d9
revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents:
38642
diff
changeset
|
159 yield subchunk |
38641
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
160 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
161 |
38718
f8762ea73e0d
sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents:
38717
diff
changeset
|
162 def _slicechunktosize(revlog, revs, targetsize=None): |
38642
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
163 """slice revs to match the target size |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
164 |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
165 This is intended to be used on chunk that density slicing selected by that |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
166 are still too large compared to the read garantee of revlog. This might |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
167 happens when "minimal gap size" interrupted the slicing or when chain are |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
168 built in a way that create large blocks next to each other. |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
169 |
40642
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
170 >>> data = [ |
38642
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
171 ... 3, #0 (3) |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
172 ... 5, #1 (2) |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
173 ... 6, #2 (1) |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
174 ... 8, #3 (2) |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
175 ... 8, #4 (empty) |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
176 ... 11, #5 (3) |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
177 ... 12, #6 (1) |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
178 ... 13, #7 (1) |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
179 ... 14, #8 (1) |
40642
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
180 ... ] |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
181 |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
182 == All snapshots cases == |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
183 >>> revlog = _testrevlog(data, snapshot=range(9)) |
38642
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
184 |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
185 Cases where chunk is already small enough |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
186 >>> list(_slicechunktosize(revlog, [0], 3)) |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
187 [[0]] |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
188 >>> list(_slicechunktosize(revlog, [6, 7], 3)) |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
189 [[6, 7]] |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
190 >>> list(_slicechunktosize(revlog, [0], None)) |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
191 [[0]] |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
192 >>> list(_slicechunktosize(revlog, [6, 7], None)) |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
193 [[6, 7]] |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
194 |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
195 cases where we need actual slicing |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
196 >>> list(_slicechunktosize(revlog, [0, 1], 3)) |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
197 [[0], [1]] |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
198 >>> list(_slicechunktosize(revlog, [1, 3], 3)) |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
199 [[1], [3]] |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
200 >>> list(_slicechunktosize(revlog, [1, 2, 3], 3)) |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
201 [[1, 2], [3]] |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
202 >>> list(_slicechunktosize(revlog, [3, 5], 3)) |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
203 [[3], [5]] |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
204 >>> list(_slicechunktosize(revlog, [3, 4, 5], 3)) |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
205 [[3], [5]] |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
206 >>> list(_slicechunktosize(revlog, [5, 6, 7, 8], 3)) |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
207 [[5], [6, 7, 8]] |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
208 >>> list(_slicechunktosize(revlog, [0, 1, 2, 3, 4, 5, 6, 7, 8], 3)) |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
209 [[0], [1, 2], [3], [5], [6, 7, 8]] |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
210 |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
211 Case with too large individual chunk (must return valid chunk) |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
212 >>> list(_slicechunktosize(revlog, [0, 1], 2)) |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
213 [[0], [1]] |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
214 >>> list(_slicechunktosize(revlog, [1, 3], 1)) |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
215 [[1], [3]] |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
216 >>> list(_slicechunktosize(revlog, [3, 4, 5], 2)) |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
217 [[3], [5]] |
40642
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
218 |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
219 == No Snapshot cases == |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
220 >>> revlog = _testrevlog(data) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
221 |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
222 Cases where chunk is already small enough |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
223 >>> list(_slicechunktosize(revlog, [0], 3)) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
224 [[0]] |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
225 >>> list(_slicechunktosize(revlog, [6, 7], 3)) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
226 [[6, 7]] |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
227 >>> list(_slicechunktosize(revlog, [0], None)) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
228 [[0]] |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
229 >>> list(_slicechunktosize(revlog, [6, 7], None)) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
230 [[6, 7]] |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
231 |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
232 cases where we need actual slicing |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
233 >>> list(_slicechunktosize(revlog, [0, 1], 3)) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
234 [[0], [1]] |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
235 >>> list(_slicechunktosize(revlog, [1, 3], 3)) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
236 [[1], [3]] |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
237 >>> list(_slicechunktosize(revlog, [1, 2, 3], 3)) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
238 [[1], [2, 3]] |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
239 >>> list(_slicechunktosize(revlog, [3, 5], 3)) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
240 [[3], [5]] |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
241 >>> list(_slicechunktosize(revlog, [3, 4, 5], 3)) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
242 [[3], [4, 5]] |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
243 >>> list(_slicechunktosize(revlog, [5, 6, 7, 8], 3)) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
244 [[5], [6, 7, 8]] |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
245 >>> list(_slicechunktosize(revlog, [0, 1, 2, 3, 4, 5, 6, 7, 8], 3)) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
246 [[0], [1, 2], [3], [5], [6, 7, 8]] |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
247 |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
248 Case with too large individual chunk (must return valid chunk) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
249 >>> list(_slicechunktosize(revlog, [0, 1], 2)) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
250 [[0], [1]] |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
251 >>> list(_slicechunktosize(revlog, [1, 3], 1)) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
252 [[1], [3]] |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
253 >>> list(_slicechunktosize(revlog, [3, 4, 5], 2)) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
254 [[3], [5]] |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
255 |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
256 == mixed case == |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
257 >>> revlog = _testrevlog(data, snapshot=[0, 1, 2]) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
258 >>> list(_slicechunktosize(revlog, list(range(9)), 5)) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
259 [[0, 1], [2], [3, 4, 5], [6, 7, 8]] |
38642
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
260 """ |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
261 assert targetsize is None or 0 <= targetsize |
40642
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
262 startdata = revlog.start(revs[0]) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
263 enddata = revlog.end(revs[-1]) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
264 fullspan = enddata - startdata |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
265 if targetsize is None or fullspan <= targetsize: |
38642
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
266 yield revs |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
267 return |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
268 |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
269 startrevidx = 0 |
40657
2eb48aa0acce
sparse-revlog: align endrevidx usages in the _slicechunktosize
Boris Feld <boris.feld@octobus.net>
parents:
40654
diff
changeset
|
270 endrevidx = 1 |
38642
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
271 iterrevs = enumerate(revs) |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
272 next(iterrevs) # skip first rev. |
40642
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
273 # first step: get snapshots out of the way |
38642
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
274 for idx, r in iterrevs: |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
275 span = revlog.end(r) - startdata |
40642
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
276 snapshot = revlog.issnapshot(r) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
277 if span <= targetsize and snapshot: |
40657
2eb48aa0acce
sparse-revlog: align endrevidx usages in the _slicechunktosize
Boris Feld <boris.feld@octobus.net>
parents:
40654
diff
changeset
|
278 endrevidx = idx + 1 |
38642
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
279 else: |
40657
2eb48aa0acce
sparse-revlog: align endrevidx usages in the _slicechunktosize
Boris Feld <boris.feld@octobus.net>
parents:
40654
diff
changeset
|
280 chunk = _trimchunk(revlog, revs, startrevidx, endrevidx) |
38642
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
281 if chunk: |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
282 yield chunk |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
283 startrevidx = idx |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
284 startdata = revlog.start(r) |
40657
2eb48aa0acce
sparse-revlog: align endrevidx usages in the _slicechunktosize
Boris Feld <boris.feld@octobus.net>
parents:
40654
diff
changeset
|
285 endrevidx = idx + 1 |
40642
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
286 if not snapshot: |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
287 break |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
288 |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
289 # for the others, we use binary slicing to quickly converge toward valid |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
290 # chunks (otherwise, we might end up looking for start/end of many |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
291 # revisions). This logic is not looking for the perfect slicing point, it |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
292 # focuses on quickly converging toward valid chunks. |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
293 nbitem = len(revs) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
294 while (enddata - startdata) > targetsize: |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
295 endrevidx = nbitem |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
296 if nbitem - startrevidx <= 1: |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
297 break # protect against individual chunk larger than limit |
40642
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
298 localenddata = revlog.end(revs[endrevidx - 1]) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
299 span = localenddata - startdata |
40654
fd1d41ccbe38
sparse-revlog: use `span` variable as intended
Boris Feld <boris.feld@octobus.net>
parents:
40642
diff
changeset
|
300 while span > targetsize: |
40642
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
301 if endrevidx - startrevidx <= 1: |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
302 break # protect against individual chunk larger than limit |
40642
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
303 endrevidx -= (endrevidx - startrevidx) // 2 |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
304 localenddata = revlog.end(revs[endrevidx - 1]) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
305 span = localenddata - startdata |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
306 chunk = _trimchunk(revlog, revs, startrevidx, endrevidx) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
307 if chunk: |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
308 yield chunk |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
309 startrevidx = endrevidx |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
310 startdata = revlog.start(revs[startrevidx]) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
311 |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
312 chunk = _trimchunk(revlog, revs, startrevidx) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
313 if chunk: |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
314 yield chunk |
38642
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
315 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
316 |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
317 def _slicechunktodensity(revlog, revs, targetdensity=0.5, mingapsize=0): |
38641
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
318 """slice revs to reduce the amount of unrelated data to be read from disk. |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
319 |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
320 ``revs`` is sliced into groups that should be read in one time. |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
321 Assume that revs are sorted. |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
322 |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
323 The initial chunk is sliced until the overall density (payload/chunks-span |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
324 ratio) is above `targetdensity`. No gap smaller than `mingapsize` is |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
325 skipped. |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
326 |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
327 >>> revlog = _testrevlog([ |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
328 ... 5, #00 (5) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
329 ... 10, #01 (5) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
330 ... 12, #02 (2) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
331 ... 12, #03 (empty) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
332 ... 27, #04 (15) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
333 ... 31, #05 (4) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
334 ... 31, #06 (empty) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
335 ... 42, #07 (11) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
336 ... 47, #08 (5) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
337 ... 47, #09 (empty) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
338 ... 48, #10 (1) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
339 ... 51, #11 (3) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
340 ... 74, #12 (23) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
341 ... 85, #13 (11) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
342 ... 86, #14 (1) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
343 ... 91, #15 (5) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
344 ... ]) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
345 |
38655
cd1c484e31e8
revlog: adjust doctest examples to be portable to Python 3
Augie Fackler <augie@google.com>
parents:
38644
diff
changeset
|
346 >>> list(_slicechunktodensity(revlog, list(range(16)))) |
38641
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
347 [[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]] |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
348 >>> list(_slicechunktodensity(revlog, [0, 15])) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
349 [[0], [15]] |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
350 >>> list(_slicechunktodensity(revlog, [0, 11, 15])) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
351 [[0], [11], [15]] |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
352 >>> list(_slicechunktodensity(revlog, [0, 11, 13, 15])) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
353 [[0], [11, 13, 15]] |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
354 >>> list(_slicechunktodensity(revlog, [1, 2, 3, 5, 8, 10, 11, 14])) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
355 [[1, 2], [5, 8, 10, 11], [14]] |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
356 >>> list(_slicechunktodensity(revlog, [1, 2, 3, 5, 8, 10, 11, 14], |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
357 ... mingapsize=20)) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
358 [[1, 2, 3, 5, 8, 10, 11], [14]] |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
359 >>> list(_slicechunktodensity(revlog, [1, 2, 3, 5, 8, 10, 11, 14], |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
360 ... targetdensity=0.95)) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
361 [[1, 2], [5], [8, 10, 11], [14]] |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
362 >>> list(_slicechunktodensity(revlog, [1, 2, 3, 5, 8, 10, 11, 14], |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
363 ... targetdensity=0.95, mingapsize=12)) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
364 [[1, 2], [5, 8, 10, 11], [14]] |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
365 """ |
34824
e2ad93bcc084
revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents:
34823
diff
changeset
|
366 start = revlog.start |
e2ad93bcc084
revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents:
34823
diff
changeset
|
367 length = revlog.length |
e2ad93bcc084
revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents:
34823
diff
changeset
|
368 |
34880
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
369 if len(revs) <= 1: |
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
370 yield revs |
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
371 return |
34824
e2ad93bcc084
revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents:
34823
diff
changeset
|
372 |
40604
3ac23dad6364
sparse-revlog: drop unused deltainfo parameter from _slicechunktodensity
Boris Feld <boris.feld@octobus.net>
parents:
40603
diff
changeset
|
373 deltachainspan = segmentspan(revlog, revs) |
34880
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
374 |
38641
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
375 if deltachainspan < mingapsize: |
38635
d083ae26c325
revlog: early return in _slicechunk when span is already small enough
Boris Feld <boris.feld@octobus.net>
parents:
38634
diff
changeset
|
376 yield revs |
d083ae26c325
revlog: early return in _slicechunk when span is already small enough
Boris Feld <boris.feld@octobus.net>
parents:
38634
diff
changeset
|
377 return |
d083ae26c325
revlog: early return in _slicechunk when span is already small enough
Boris Feld <boris.feld@octobus.net>
parents:
38634
diff
changeset
|
378 |
38718
f8762ea73e0d
sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents:
38717
diff
changeset
|
379 readdata = deltachainspan |
40606
bfbfd15d65bd
sparse-revlog: fast-path before computing payload size
Boris Feld <boris.feld@octobus.net>
parents:
40605
diff
changeset
|
380 chainpayload = sum(length(r) for r in revs) |
34824
e2ad93bcc084
revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents:
34823
diff
changeset
|
381 |
34880
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
382 if deltachainspan: |
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
383 density = chainpayload / float(deltachainspan) |
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
384 else: |
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
385 density = 1.0 |
34824
e2ad93bcc084
revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents:
34823
diff
changeset
|
386 |
38641
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
387 if density >= targetdensity: |
38634
f0ea8b847831
revlog: early return in _slicechunk when density is already good
Paul Morelle <paul.morelle@octobus.net>
parents:
38632
diff
changeset
|
388 yield revs |
f0ea8b847831
revlog: early return in _slicechunk when density is already good
Paul Morelle <paul.morelle@octobus.net>
parents:
38632
diff
changeset
|
389 return |
f0ea8b847831
revlog: early return in _slicechunk when density is already good
Paul Morelle <paul.morelle@octobus.net>
parents:
38632
diff
changeset
|
390 |
34880
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
391 # Store the gaps in a heap to have them sorted by decreasing size |
40607
54de23400b2a
sparse-revlog: stop using a heap to track gaps
Boris Feld <boris.feld@octobus.net>
parents:
40606
diff
changeset
|
392 gaps = [] |
34880
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
393 prevend = None |
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
394 for i, rev in enumerate(revs): |
40604
3ac23dad6364
sparse-revlog: drop unused deltainfo parameter from _slicechunktodensity
Boris Feld <boris.feld@octobus.net>
parents:
40603
diff
changeset
|
395 revstart = start(rev) |
3ac23dad6364
sparse-revlog: drop unused deltainfo parameter from _slicechunktodensity
Boris Feld <boris.feld@octobus.net>
parents:
40603
diff
changeset
|
396 revlen = length(rev) |
34824
e2ad93bcc084
revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents:
34823
diff
changeset
|
397 |
34898
1bde8e8e5de0
sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents:
34881
diff
changeset
|
398 # Skip empty revisions to form larger holes |
1bde8e8e5de0
sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents:
34881
diff
changeset
|
399 if revlen == 0: |
1bde8e8e5de0
sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents:
34881
diff
changeset
|
400 continue |
1bde8e8e5de0
sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents:
34881
diff
changeset
|
401 |
34880
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
402 if prevend is not None: |
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
403 gapsize = revstart - prevend |
34881
8c9b08a0c48c
sparse-read: skip gaps too small to be worth splitting
Paul Morelle <paul.morelle@octobus.net>
parents:
34880
diff
changeset
|
404 # only consider holes that are large enough |
38641
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
405 if gapsize > mingapsize: |
40607
54de23400b2a
sparse-revlog: stop using a heap to track gaps
Boris Feld <boris.feld@octobus.net>
parents:
40606
diff
changeset
|
406 gaps.append((gapsize, i)) |
34880
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
407 |
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
408 prevend = revstart + revlen |
40607
54de23400b2a
sparse-revlog: stop using a heap to track gaps
Boris Feld <boris.feld@octobus.net>
parents:
40606
diff
changeset
|
409 # sort the gaps to pop them from largest to small |
54de23400b2a
sparse-revlog: stop using a heap to track gaps
Boris Feld <boris.feld@octobus.net>
parents:
40606
diff
changeset
|
410 gaps.sort() |
34880
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
411 |
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
412 # Collect the indices of the largest holes until the density is acceptable |
40608
526ee887c4d5
sparse-revlog: stop using a heap to track selected gap
Boris Feld <boris.feld@octobus.net>
parents:
40607
diff
changeset
|
413 selected = [] |
40607
54de23400b2a
sparse-revlog: stop using a heap to track gaps
Boris Feld <boris.feld@octobus.net>
parents:
40606
diff
changeset
|
414 while gaps and density < targetdensity: |
54de23400b2a
sparse-revlog: stop using a heap to track gaps
Boris Feld <boris.feld@octobus.net>
parents:
40606
diff
changeset
|
415 gapsize, gapidx = gaps.pop() |
34880
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
416 |
40608
526ee887c4d5
sparse-revlog: stop using a heap to track selected gap
Boris Feld <boris.feld@octobus.net>
parents:
40607
diff
changeset
|
417 selected.append(gapidx) |
34824
e2ad93bcc084
revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents:
34823
diff
changeset
|
418 |
34880
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
419 # the gap sizes are stored as negatives to be sorted decreasingly |
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
420 # by the heap |
40607
54de23400b2a
sparse-revlog: stop using a heap to track gaps
Boris Feld <boris.feld@octobus.net>
parents:
40606
diff
changeset
|
421 readdata -= gapsize |
34880
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
422 if readdata > 0: |
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
423 density = chainpayload / float(readdata) |
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
424 else: |
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
425 density = 1.0 |
40608
526ee887c4d5
sparse-revlog: stop using a heap to track selected gap
Boris Feld <boris.feld@octobus.net>
parents:
40607
diff
changeset
|
426 selected.sort() |
34824
e2ad93bcc084
revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents:
34823
diff
changeset
|
427 |
34880
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
428 # Cut the revs at collected indices |
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
429 previdx = 0 |
40608
526ee887c4d5
sparse-revlog: stop using a heap to track selected gap
Boris Feld <boris.feld@octobus.net>
parents:
40607
diff
changeset
|
430 for idx in selected: |
34898
1bde8e8e5de0
sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents:
34881
diff
changeset
|
431 |
1bde8e8e5de0
sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents:
34881
diff
changeset
|
432 chunk = _trimchunk(revlog, revs, previdx, idx) |
1bde8e8e5de0
sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents:
34881
diff
changeset
|
433 if chunk: |
1bde8e8e5de0
sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents:
34881
diff
changeset
|
434 yield chunk |
1bde8e8e5de0
sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents:
34881
diff
changeset
|
435 |
34880
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
436 previdx = idx |
34898
1bde8e8e5de0
sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents:
34881
diff
changeset
|
437 |
1bde8e8e5de0
sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents:
34881
diff
changeset
|
438 chunk = _trimchunk(revlog, revs, previdx) |
1bde8e8e5de0
sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents:
34881
diff
changeset
|
439 if chunk: |
1bde8e8e5de0
sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents:
34881
diff
changeset
|
440 yield chunk |
34824
e2ad93bcc084
revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents:
34823
diff
changeset
|
441 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
442 |
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
443 def _trimchunk(revlog, revs, startidx, endidx=None): |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
444 """returns revs[startidx:endidx] without empty trailing revs |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
445 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
446 Doctest Setup |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
447 >>> revlog = _testrevlog([ |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
448 ... 5, #0 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
449 ... 10, #1 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
450 ... 12, #2 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
451 ... 12, #3 (empty) |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
452 ... 17, #4 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
453 ... 21, #5 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
454 ... 21, #6 (empty) |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
455 ... ]) |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
456 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
457 Contiguous cases: |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
458 >>> _trimchunk(revlog, [0, 1, 2, 3, 4, 5, 6], 0) |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
459 [0, 1, 2, 3, 4, 5] |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
460 >>> _trimchunk(revlog, [0, 1, 2, 3, 4, 5, 6], 0, 5) |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
461 [0, 1, 2, 3, 4] |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
462 >>> _trimchunk(revlog, [0, 1, 2, 3, 4, 5, 6], 0, 4) |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
463 [0, 1, 2] |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
464 >>> _trimchunk(revlog, [0, 1, 2, 3, 4, 5, 6], 2, 4) |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
465 [2] |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
466 >>> _trimchunk(revlog, [0, 1, 2, 3, 4, 5, 6], 3) |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
467 [3, 4, 5] |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
468 >>> _trimchunk(revlog, [0, 1, 2, 3, 4, 5, 6], 3, 5) |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
469 [3, 4] |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
470 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
471 Discontiguous cases: |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
472 >>> _trimchunk(revlog, [1, 3, 5, 6], 0) |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
473 [1, 3, 5] |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
474 >>> _trimchunk(revlog, [1, 3, 5, 6], 0, 2) |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
475 [1] |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
476 >>> _trimchunk(revlog, [1, 3, 5, 6], 1, 3) |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
477 [3, 5] |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
478 >>> _trimchunk(revlog, [1, 3, 5, 6], 1) |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
479 [3, 5] |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
480 """ |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
481 length = revlog.length |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
482 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
483 if endidx is None: |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
484 endidx = len(revs) |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
485 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
486 # If we have a non-emtpy delta candidate, there are nothing to trim |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
487 if revs[endidx - 1] < len(revlog): |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
488 # Trim empty revs at the end, except the very first revision of a chain |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
489 while ( |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
490 endidx > 1 and endidx > startidx and length(revs[endidx - 1]) == 0 |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
491 ): |
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
492 endidx -= 1 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
493 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
494 return revs[startidx:endidx] |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
495 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
496 |
40605
a32ccd32982b
sparse-revlog: drop unused deltainfo parameter from segmentspan
Boris Feld <boris.feld@octobus.net>
parents:
40604
diff
changeset
|
497 def segmentspan(revlog, revs): |
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
498 """Get the byte span of a segment of revisions |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
499 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
500 revs is a sorted array of revision numbers |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
501 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
502 >>> revlog = _testrevlog([ |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
503 ... 5, #0 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
504 ... 10, #1 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
505 ... 12, #2 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
506 ... 12, #3 (empty) |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
507 ... 17, #4 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
508 ... ]) |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
509 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
510 >>> segmentspan(revlog, [0, 1, 2, 3, 4]) |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
511 17 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
512 >>> segmentspan(revlog, [0, 4]) |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
513 17 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
514 >>> segmentspan(revlog, [3, 4]) |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
515 5 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
516 >>> segmentspan(revlog, [1, 2, 3,]) |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
517 7 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
518 >>> segmentspan(revlog, [1, 3]) |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
519 7 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
520 """ |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
521 if not revs: |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
522 return 0 |
40605
a32ccd32982b
sparse-revlog: drop unused deltainfo parameter from segmentspan
Boris Feld <boris.feld@octobus.net>
parents:
40604
diff
changeset
|
523 end = revlog.end(revs[-1]) |
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
524 return end - revlog.start(revs[0]) |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
525 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
526 |
51020
509f0f7fc89e
delta-computer: stop explicitly taking file handle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50929
diff
changeset
|
527 def _textfromdelta(revlog, baserev, delta, p1, p2, flags, expectednode): |
39331
fd0150a3c2fe
revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents:
39330
diff
changeset
|
528 """build full text from a (base, delta) pair and other metadata""" |
fd0150a3c2fe
revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents:
39330
diff
changeset
|
529 # special case deltas which replace entire base; no need to decode |
fd0150a3c2fe
revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents:
39330
diff
changeset
|
530 # base revision. this neatly avoids censored bases, which throw when |
fd0150a3c2fe
revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents:
39330
diff
changeset
|
531 # they're decoded. |
43077
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
532 hlen = struct.calcsize(b">lll") |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
533 if delta[:hlen] == mdiff.replacediffheader( |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
534 revlog.rawsize(baserev), len(delta) - hlen |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
535 ): |
39331
fd0150a3c2fe
revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents:
39330
diff
changeset
|
536 fulltext = delta[hlen:] |
fd0150a3c2fe
revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents:
39330
diff
changeset
|
537 else: |
fd0150a3c2fe
revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents:
39330
diff
changeset
|
538 # deltabase is rawtext before changed by flag processors, which is |
fd0150a3c2fe
revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents:
39330
diff
changeset
|
539 # equivalent to non-raw text |
51020
509f0f7fc89e
delta-computer: stop explicitly taking file handle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50929
diff
changeset
|
540 basetext = revlog.revision(baserev) |
39331
fd0150a3c2fe
revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents:
39330
diff
changeset
|
541 fulltext = mdiff.patch(basetext, delta) |
fd0150a3c2fe
revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents:
39330
diff
changeset
|
542 |
fd0150a3c2fe
revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents:
39330
diff
changeset
|
543 try: |
42992
dff95420480f
flagprocessors: make `processflagsraw` a module level function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42878
diff
changeset
|
544 validatehash = flagutil.processflagsraw(revlog, fulltext, flags) |
39331
fd0150a3c2fe
revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents:
39330
diff
changeset
|
545 if validatehash: |
fd0150a3c2fe
revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents:
39330
diff
changeset
|
546 revlog.checkhash(fulltext, expectednode, p1=p1, p2=p2) |
fd0150a3c2fe
revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents:
39330
diff
changeset
|
547 if flags & REVIDX_ISCENSORED: |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
548 raise error.StorageError( |
43077
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
549 _(b'node %s is not censored') % expectednode |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
550 ) |
39774
4a2466b2a434
revlog: drop some more error aliases (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39773
diff
changeset
|
551 except error.CensoredNodeError: |
39331
fd0150a3c2fe
revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents:
39330
diff
changeset
|
552 # must pass the censored index flag to add censored revisions |
fd0150a3c2fe
revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents:
39330
diff
changeset
|
553 if not flags & REVIDX_ISCENSORED: |
fd0150a3c2fe
revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents:
39330
diff
changeset
|
554 raise |
fd0150a3c2fe
revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents:
39330
diff
changeset
|
555 return fulltext |
fd0150a3c2fe
revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents:
39330
diff
changeset
|
556 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
557 |
35638
edc9330acac1
revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents:
35637
diff
changeset
|
558 @attr.s(slots=True, frozen=True) |
48946
642e31cb55f0
py3: use class X: instead of class X(object):
Gregory Szorc <gregory.szorc@gmail.com>
parents:
48875
diff
changeset
|
559 class _deltainfo: |
35638
edc9330acac1
revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents:
35637
diff
changeset
|
560 distance = attr.ib() |
edc9330acac1
revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents:
35637
diff
changeset
|
561 deltalen = attr.ib() |
edc9330acac1
revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents:
35637
diff
changeset
|
562 data = attr.ib() |
edc9330acac1
revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents:
35637
diff
changeset
|
563 base = attr.ib() |
edc9330acac1
revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents:
35637
diff
changeset
|
564 chainbase = attr.ib() |
edc9330acac1
revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents:
35637
diff
changeset
|
565 chainlen = attr.ib() |
edc9330acac1
revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents:
35637
diff
changeset
|
566 compresseddeltalen = attr.ib() |
39154
e0da43e2f71f
revlog: compute snapshot depth on delta info
Boris Feld <boris.feld@octobus.net>
parents:
39152
diff
changeset
|
567 snapshotdepth = attr.ib() |
35638
edc9330acac1
revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents:
35637
diff
changeset
|
568 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
569 |
47253
b876f0bf7366
revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46819
diff
changeset
|
570 def drop_u_compression(delta): |
b876f0bf7366
revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46819
diff
changeset
|
571 """turn into a "u" (no-compression) into no-compression without header |
b876f0bf7366
revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46819
diff
changeset
|
572 |
b876f0bf7366
revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46819
diff
changeset
|
573 This is useful for revlog format that has better compression method. |
b876f0bf7366
revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46819
diff
changeset
|
574 """ |
b876f0bf7366
revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46819
diff
changeset
|
575 assert delta.data[0] == b'u', delta.data[0] |
b876f0bf7366
revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46819
diff
changeset
|
576 return _deltainfo( |
b876f0bf7366
revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46819
diff
changeset
|
577 delta.distance, |
b876f0bf7366
revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46819
diff
changeset
|
578 delta.deltalen - 1, |
b876f0bf7366
revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46819
diff
changeset
|
579 (b'', delta.data[1]), |
b876f0bf7366
revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46819
diff
changeset
|
580 delta.base, |
b876f0bf7366
revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46819
diff
changeset
|
581 delta.chainbase, |
b876f0bf7366
revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46819
diff
changeset
|
582 delta.chainlen, |
b876f0bf7366
revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46819
diff
changeset
|
583 delta.compresseddeltalen, |
b876f0bf7366
revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46819
diff
changeset
|
584 delta.snapshotdepth, |
b876f0bf7366
revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46819
diff
changeset
|
585 ) |
b876f0bf7366
revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46819
diff
changeset
|
586 |
b876f0bf7366
revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46819
diff
changeset
|
587 |
40978
42f59d3f714d
delta: exclude base candidate much smaller than the target
Boris Feld <boris.feld@octobus.net>
parents:
40957
diff
changeset
|
588 # If a revision's full text is that much bigger than a base candidate full |
42f59d3f714d
delta: exclude base candidate much smaller than the target
Boris Feld <boris.feld@octobus.net>
parents:
40957
diff
changeset
|
589 # text's, it is very unlikely that it will produce a valid delta. We no longer |
42f59d3f714d
delta: exclude base candidate much smaller than the target
Boris Feld <boris.feld@octobus.net>
parents:
40957
diff
changeset
|
590 # consider these candidates. |
41033
b373477948df
revlog: limit base to rev size ratio to 500 instead of 50
Boris Feld <boris.feld@octobus.net>
parents:
40979
diff
changeset
|
591 LIMIT_BASE2TEXT = 500 |
40978
42f59d3f714d
delta: exclude base candidate much smaller than the target
Boris Feld <boris.feld@octobus.net>
parents:
40957
diff
changeset
|
592 |
51349
cc806f20d756
delta-find: explicitly track stage of the search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51348
diff
changeset
|
593 ### stage of the search, used for debug and to select and to adjust some logic. |
cc806f20d756
delta-find: explicitly track stage of the search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51348
diff
changeset
|
594 # initial stage, next step is unknown |
cc806f20d756
delta-find: explicitly track stage of the search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51348
diff
changeset
|
595 _STAGE_UNSPECIFIED = "unspecified" |
cc806f20d756
delta-find: explicitly track stage of the search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51348
diff
changeset
|
596 # trying the cached delta |
cc806f20d756
delta-find: explicitly track stage of the search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51348
diff
changeset
|
597 _STAGE_CACHED = "cached" |
cc806f20d756
delta-find: explicitly track stage of the search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51348
diff
changeset
|
598 # trying delta based on parents |
cc806f20d756
delta-find: explicitly track stage of the search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51348
diff
changeset
|
599 _STAGE_PARENTS = "parents" |
cc806f20d756
delta-find: explicitly track stage of the search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51348
diff
changeset
|
600 # trying to build a valid snapshot of any level |
cc806f20d756
delta-find: explicitly track stage of the search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51348
diff
changeset
|
601 _STAGE_SNAPSHOT = "snapshot" |
cc806f20d756
delta-find: explicitly track stage of the search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51348
diff
changeset
|
602 # trying to build a delta based of the previous revision |
cc806f20d756
delta-find: explicitly track stage of the search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51348
diff
changeset
|
603 _STAGE_PREV = "prev" |
cc806f20d756
delta-find: explicitly track stage of the search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51348
diff
changeset
|
604 # trying to build a full snapshot |
cc806f20d756
delta-find: explicitly track stage of the search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51348
diff
changeset
|
605 _STAGE_FULL = "full" |
cc806f20d756
delta-find: explicitly track stage of the search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51348
diff
changeset
|
606 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
607 |
51346
2a333d791ecf
delta-find: introduce a base class for _DeltaSearch
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51345
diff
changeset
|
608 class _BaseDeltaSearch(abc.ABC): |
51320
c82e03b102a6
delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51319
diff
changeset
|
609 """perform the search of a good delta for a single revlog revision |
50354
ca1522fe4ec8
delta-find: assume the target-rev if not specified
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49776
diff
changeset
|
610 |
51320
c82e03b102a6
delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51319
diff
changeset
|
611 note: some of the deltacomputer.finddeltainfo logic should probably move |
c82e03b102a6
delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51319
diff
changeset
|
612 here. |
c82e03b102a6
delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51319
diff
changeset
|
613 """ |
50355
0232571255d3
delta-find: never do anything fancy when general delta is off
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50354
diff
changeset
|
614 |
51320
c82e03b102a6
delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51319
diff
changeset
|
615 def __init__( |
c82e03b102a6
delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51319
diff
changeset
|
616 self, |
49610
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
617 revlog, |
51330
49401b7dec0c
delta-find: feed revinfo to _DeltaSearch
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51328
diff
changeset
|
618 revinfo, |
49610
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
619 p1, |
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
620 p2, |
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
621 cachedelta, |
51320
c82e03b102a6
delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51319
diff
changeset
|
622 excluded_bases=None, |
c82e03b102a6
delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51319
diff
changeset
|
623 target_rev=None, |
c82e03b102a6
delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51319
diff
changeset
|
624 snapshot_cache=None, |
c82e03b102a6
delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51319
diff
changeset
|
625 ): |
51325
c9ab452e5611
delta-find: check DELTA_BASE_REUSE_FORCE in the _DeltaSearch.__init__
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51324
diff
changeset
|
626 # the DELTA_BASE_REUSE_FORCE case should have been taken care of sooner |
c9ab452e5611
delta-find: check DELTA_BASE_REUSE_FORCE in the _DeltaSearch.__init__
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51324
diff
changeset
|
627 # so we should never end up asking such question. Adding the assert as |
c9ab452e5611
delta-find: check DELTA_BASE_REUSE_FORCE in the _DeltaSearch.__init__
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51324
diff
changeset
|
628 # a safe-guard to detect anything that would be fishy in this regard. |
c9ab452e5611
delta-find: check DELTA_BASE_REUSE_FORCE in the _DeltaSearch.__init__
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51324
diff
changeset
|
629 assert ( |
c9ab452e5611
delta-find: check DELTA_BASE_REUSE_FORCE in the _DeltaSearch.__init__
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51324
diff
changeset
|
630 cachedelta is None |
c9ab452e5611
delta-find: check DELTA_BASE_REUSE_FORCE in the _DeltaSearch.__init__
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51324
diff
changeset
|
631 or cachedelta[2] != DELTA_BASE_REUSE_FORCE |
c9ab452e5611
delta-find: check DELTA_BASE_REUSE_FORCE in the _DeltaSearch.__init__
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51324
diff
changeset
|
632 or not revlog.delta_config.general_delta |
c9ab452e5611
delta-find: check DELTA_BASE_REUSE_FORCE in the _DeltaSearch.__init__
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51324
diff
changeset
|
633 ) |
51320
c82e03b102a6
delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51319
diff
changeset
|
634 self.revlog = revlog |
51330
49401b7dec0c
delta-find: feed revinfo to _DeltaSearch
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51328
diff
changeset
|
635 self.revinfo = revinfo |
49401b7dec0c
delta-find: feed revinfo to _DeltaSearch
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51328
diff
changeset
|
636 self.textlen = revinfo.textlen |
51320
c82e03b102a6
delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51319
diff
changeset
|
637 self.p1 = p1 |
c82e03b102a6
delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51319
diff
changeset
|
638 self.p2 = p2 |
c82e03b102a6
delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51319
diff
changeset
|
639 self.cachedelta = cachedelta |
c82e03b102a6
delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51319
diff
changeset
|
640 self.excluded_bases = excluded_bases |
51324
047358417d01
delta-find: move target_rev in the _DeltaSearch.__init__
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51323
diff
changeset
|
641 if target_rev is None: |
047358417d01
delta-find: move target_rev in the _DeltaSearch.__init__
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51323
diff
changeset
|
642 self.target_rev = len(self.revlog) |
51320
c82e03b102a6
delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51319
diff
changeset
|
643 self.target_rev = target_rev |
51323
87fce139e927
delta-find: move snapshot_cache in the _DeltaSearch.__init__
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51322
diff
changeset
|
644 if snapshot_cache is None: |
87fce139e927
delta-find: move snapshot_cache in the _DeltaSearch.__init__
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51322
diff
changeset
|
645 # map: base-rev: [snapshot-revs] |
87fce139e927
delta-find: move snapshot_cache in the _DeltaSearch.__init__
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51322
diff
changeset
|
646 snapshot_cache = SnapshotCache() |
51320
c82e03b102a6
delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51319
diff
changeset
|
647 self.snapshot_cache = snapshot_cache |
c82e03b102a6
delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51319
diff
changeset
|
648 |
51326
01db705bbe32
delta-find: move tested in the _DeltaSearch.__init__
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51325
diff
changeset
|
649 self.tested = {nullrev} |
01db705bbe32
delta-find: move tested in the _DeltaSearch.__init__
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51325
diff
changeset
|
650 |
51349
cc806f20d756
delta-find: explicitly track stage of the search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51348
diff
changeset
|
651 self.current_stage = _STAGE_UNSPECIFIED |
51346
2a333d791ecf
delta-find: introduce a base class for _DeltaSearch
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51345
diff
changeset
|
652 self.current_group = None |
2a333d791ecf
delta-find: introduce a base class for _DeltaSearch
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51345
diff
changeset
|
653 self._init_group() |
51334
d0d869fccd20
delta-find: move away from the generator API for _DeltaSearch
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51333
diff
changeset
|
654 |
51344
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
655 def is_good_delta_info(self, deltainfo): |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
656 """Returns True if the given delta is good. |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
657 |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
658 Good means that it is within the disk span, disk size, and chain length |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
659 bounds that we know to be performant. |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
660 """ |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
661 if not self._is_good_delta_info_universal(deltainfo): |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
662 return False |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
663 if not self._is_good_delta_info_chain_quality(deltainfo): |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
664 return False |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
665 return True |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
666 |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
667 def _is_good_delta_info_universal(self, deltainfo): |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
668 """Returns True if the given delta is good. |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
669 |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
670 This performs generic checks needed by all format variants. |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
671 |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
672 This is used by is_good_delta_info. |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
673 """ |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
674 |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
675 if deltainfo is None: |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
676 return False |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
677 |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
678 # the DELTA_BASE_REUSE_FORCE case should have been taken care of sooner |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
679 # so we should never end up asking such question. Adding the assert as |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
680 # a safe-guard to detect anything that would be fishy in this regard. |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
681 assert ( |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
682 self.revinfo.cachedelta is None |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
683 or self.revinfo.cachedelta[2] != DELTA_BASE_REUSE_FORCE |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
684 or not self.revlog.delta_config.general_delta |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
685 ) |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
686 |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
687 # Bad delta from new delta size: |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
688 # |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
689 # If the delta size is larger than the target text, storing the delta |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
690 # will be inefficient. |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
691 if self.revinfo.textlen < deltainfo.deltalen: |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
692 return False |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
693 |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
694 return True |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
695 |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
696 def _is_good_delta_info_chain_quality(self, deltainfo): |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
697 """Returns True if the chain associated with the delta is good. |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
698 |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
699 This performs checks for format that use delta chains. |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
700 |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
701 This is used by is_good_delta_info. |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
702 """ |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
703 # - 'deltainfo.distance' is the distance from the base revision -- |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
704 # bounding it limits the amount of I/O we need to do. |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
705 |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
706 defaultmax = self.revinfo.textlen * 4 |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
707 maxdist = self.revlog.delta_config.max_deltachain_span |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
708 if not maxdist: |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
709 maxdist = deltainfo.distance # ensure the conditional pass |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
710 maxdist = max(maxdist, defaultmax) |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
711 |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
712 # Bad delta from read span: |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
713 # |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
714 # If the span of data read is larger than the maximum allowed. |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
715 # |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
716 # In the sparse-revlog case, we rely on the associated "sparse |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
717 # reading" to avoid issue related to the span of data. In theory, it |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
718 # would be possible to build pathological revlog where delta pattern |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
719 # would lead to too many reads. However, they do not happen in |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
720 # practice at all. So we skip the span check entirely. |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
721 if ( |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
722 not self.revlog.delta_config.sparse_revlog |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
723 and maxdist < deltainfo.distance |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
724 ): |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
725 return False |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
726 |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
727 # Bad delta from cumulated payload size: |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
728 # |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
729 # - 'deltainfo.compresseddeltalen' is the sum of the total size of |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
730 # deltas we need to apply -- bounding it limits the amount of CPU |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
731 # we consume. |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
732 max_chain_data = self.revinfo.textlen * LIMIT_DELTA2TEXT |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
733 # If the sum of delta get larger than K * target text length. |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
734 if max_chain_data < deltainfo.compresseddeltalen: |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
735 return False |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
736 |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
737 # Bad delta from chain length: |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
738 # |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
739 # If the number of delta in the chain gets too high. |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
740 if ( |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
741 self.revlog.delta_config.max_chain_len |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
742 and self.revlog.delta_config.max_chain_len < deltainfo.chainlen |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
743 ): |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
744 return False |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
745 return True |
f3f35b37f4b2
delta-find: move good delta code earlier in the class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51343
diff
changeset
|
746 |
51334
d0d869fccd20
delta-find: move away from the generator API for _DeltaSearch
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51333
diff
changeset
|
747 @property |
d0d869fccd20
delta-find: move away from the generator API for _DeltaSearch
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51333
diff
changeset
|
748 def done(self): |
d0d869fccd20
delta-find: move away from the generator API for _DeltaSearch
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51333
diff
changeset
|
749 """True when all possible candidate have been tested""" |
d0d869fccd20
delta-find: move away from the generator API for _DeltaSearch
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51333
diff
changeset
|
750 return self.current_group is None |
d0d869fccd20
delta-find: move away from the generator API for _DeltaSearch
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51333
diff
changeset
|
751 |
51346
2a333d791ecf
delta-find: introduce a base class for _DeltaSearch
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51345
diff
changeset
|
752 @abc.abstractmethod |
51334
d0d869fccd20
delta-find: move away from the generator API for _DeltaSearch
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51333
diff
changeset
|
753 def next_group(self, good_delta=None): |
d0d869fccd20
delta-find: move away from the generator API for _DeltaSearch
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51333
diff
changeset
|
754 """move to the next group to test |
d0d869fccd20
delta-find: move away from the generator API for _DeltaSearch
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51333
diff
changeset
|
755 |
d0d869fccd20
delta-find: move away from the generator API for _DeltaSearch
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51333
diff
changeset
|
756 The group of revision to test will be available in |
d0d869fccd20
delta-find: move away from the generator API for _DeltaSearch
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51333
diff
changeset
|
757 `self.current_group`. If the previous group had any good delta, the |
d0d869fccd20
delta-find: move away from the generator API for _DeltaSearch
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51333
diff
changeset
|
758 best one can be passed as the `good_delta` parameter to help selecting |
d0d869fccd20
delta-find: move away from the generator API for _DeltaSearch
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51333
diff
changeset
|
759 the next group. |
d0d869fccd20
delta-find: move away from the generator API for _DeltaSearch
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51333
diff
changeset
|
760 |
d0d869fccd20
delta-find: move away from the generator API for _DeltaSearch
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51333
diff
changeset
|
761 If not revision remains to be, `self.done` will be True and |
d0d869fccd20
delta-find: move away from the generator API for _DeltaSearch
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51333
diff
changeset
|
762 `self.current_group` will be None. |
d0d869fccd20
delta-find: move away from the generator API for _DeltaSearch
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51333
diff
changeset
|
763 """ |
51346
2a333d791ecf
delta-find: introduce a base class for _DeltaSearch
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51345
diff
changeset
|
764 pass |
2a333d791ecf
delta-find: introduce a base class for _DeltaSearch
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51345
diff
changeset
|
765 |
2a333d791ecf
delta-find: introduce a base class for _DeltaSearch
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51345
diff
changeset
|
766 @abc.abstractmethod |
2a333d791ecf
delta-find: introduce a base class for _DeltaSearch
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51345
diff
changeset
|
767 def _init_group(self): |
2a333d791ecf
delta-find: introduce a base class for _DeltaSearch
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51345
diff
changeset
|
768 pass |
2a333d791ecf
delta-find: introduce a base class for _DeltaSearch
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51345
diff
changeset
|
769 |
2a333d791ecf
delta-find: introduce a base class for _DeltaSearch
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51345
diff
changeset
|
770 |
51347
555826073625
delta-find: introduce and use specialized _DeltaSearch class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51346
diff
changeset
|
771 class _NoDeltaSearch(_BaseDeltaSearch): |
555826073625
delta-find: introduce and use specialized _DeltaSearch class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51346
diff
changeset
|
772 """Search for no delta. |
555826073625
delta-find: introduce and use specialized _DeltaSearch class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51346
diff
changeset
|
773 |
555826073625
delta-find: introduce and use specialized _DeltaSearch class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51346
diff
changeset
|
774 This search variant is to be used in case where we should not store delta. |
555826073625
delta-find: introduce and use specialized _DeltaSearch class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51346
diff
changeset
|
775 """ |
555826073625
delta-find: introduce and use specialized _DeltaSearch class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51346
diff
changeset
|
776 |
555826073625
delta-find: introduce and use specialized _DeltaSearch class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51346
diff
changeset
|
777 def _init_group(self): |
51349
cc806f20d756
delta-find: explicitly track stage of the search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51348
diff
changeset
|
778 self.current_stage = _STAGE_FULL |
51347
555826073625
delta-find: introduce and use specialized _DeltaSearch class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51346
diff
changeset
|
779 |
555826073625
delta-find: introduce and use specialized _DeltaSearch class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51346
diff
changeset
|
780 def next_group(self, good_delta=None): |
555826073625
delta-find: introduce and use specialized _DeltaSearch class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51346
diff
changeset
|
781 pass |
555826073625
delta-find: introduce and use specialized _DeltaSearch class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51346
diff
changeset
|
782 |
555826073625
delta-find: introduce and use specialized _DeltaSearch class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51346
diff
changeset
|
783 |
555826073625
delta-find: introduce and use specialized _DeltaSearch class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51346
diff
changeset
|
784 class _PrevDeltaSearch(_BaseDeltaSearch): |
555826073625
delta-find: introduce and use specialized _DeltaSearch class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51346
diff
changeset
|
785 """Search for delta against the previous revision only |
555826073625
delta-find: introduce and use specialized _DeltaSearch class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51346
diff
changeset
|
786 |
555826073625
delta-find: introduce and use specialized _DeltaSearch class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51346
diff
changeset
|
787 This search variant is to be used when the format does not allow for delta |
555826073625
delta-find: introduce and use specialized _DeltaSearch class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51346
diff
changeset
|
788 against arbitrary bases. |
555826073625
delta-find: introduce and use specialized _DeltaSearch class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51346
diff
changeset
|
789 """ |
555826073625
delta-find: introduce and use specialized _DeltaSearch class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51346
diff
changeset
|
790 |
555826073625
delta-find: introduce and use specialized _DeltaSearch class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51346
diff
changeset
|
791 def _init_group(self): |
51349
cc806f20d756
delta-find: explicitly track stage of the search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51348
diff
changeset
|
792 self.current_stage = _STAGE_PREV |
51347
555826073625
delta-find: introduce and use specialized _DeltaSearch class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51346
diff
changeset
|
793 self.current_group = [self.target_rev - 1] |
51351
bbbfa00d1e67
delta-find: remove the "candidate groups" layer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51350
diff
changeset
|
794 self.tested.update(self.current_group) |
51347
555826073625
delta-find: introduce and use specialized _DeltaSearch class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51346
diff
changeset
|
795 |
555826073625
delta-find: introduce and use specialized _DeltaSearch class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51346
diff
changeset
|
796 def next_group(self, good_delta=None): |
51349
cc806f20d756
delta-find: explicitly track stage of the search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51348
diff
changeset
|
797 self.current_stage = _STAGE_FULL |
51347
555826073625
delta-find: introduce and use specialized _DeltaSearch class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51346
diff
changeset
|
798 self.current_group = None |
555826073625
delta-find: introduce and use specialized _DeltaSearch class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51346
diff
changeset
|
799 |
555826073625
delta-find: introduce and use specialized _DeltaSearch class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51346
diff
changeset
|
800 |
51357
d7e2acdd50ba
delta-find: split the _DeltaSearch class in two
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51356
diff
changeset
|
801 class _GeneralDeltaSearch(_BaseDeltaSearch): |
d7e2acdd50ba
delta-find: split the _DeltaSearch class in two
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51356
diff
changeset
|
802 """Delta search variant for general-delta repository""" |
51347
555826073625
delta-find: introduce and use specialized _DeltaSearch class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51346
diff
changeset
|
803 |
51346
2a333d791ecf
delta-find: introduce a base class for _DeltaSearch
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51345
diff
changeset
|
804 def _init_group(self): |
51347
555826073625
delta-find: introduce and use specialized _DeltaSearch class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51346
diff
changeset
|
805 # Why search for delta base if we cannot use a delta base ? |
555826073625
delta-find: introduce and use specialized _DeltaSearch class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51346
diff
changeset
|
806 # also see issue6056 |
555826073625
delta-find: introduce and use specialized _DeltaSearch class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51346
diff
changeset
|
807 assert self.revlog.delta_config.general_delta |
51356
701caeabbee7
delta-find: finish reworking the snapshot logic and drop more layer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51355
diff
changeset
|
808 self._candidates_iterator = self._iter_groups() |
51346
2a333d791ecf
delta-find: introduce a base class for _DeltaSearch
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51345
diff
changeset
|
809 self._last_good = None |
51352
7083b33a2699
delta-find: explicitly deal with usage of the cached revision
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51351
diff
changeset
|
810 if ( |
7083b33a2699
delta-find: explicitly deal with usage of the cached revision
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51351
diff
changeset
|
811 self.cachedelta is not None |
7083b33a2699
delta-find: explicitly deal with usage of the cached revision
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51351
diff
changeset
|
812 and self.cachedelta[2] > DELTA_BASE_REUSE_NO |
7083b33a2699
delta-find: explicitly deal with usage of the cached revision
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51351
diff
changeset
|
813 and self._pre_filter_rev(self.cachedelta[0]) |
7083b33a2699
delta-find: explicitly deal with usage of the cached revision
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51351
diff
changeset
|
814 ): |
7083b33a2699
delta-find: explicitly deal with usage of the cached revision
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51351
diff
changeset
|
815 # First we try to reuse a the delta contained in the bundle. (or from |
7083b33a2699
delta-find: explicitly deal with usage of the cached revision
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51351
diff
changeset
|
816 # the source revlog) |
7083b33a2699
delta-find: explicitly deal with usage of the cached revision
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51351
diff
changeset
|
817 # |
7083b33a2699
delta-find: explicitly deal with usage of the cached revision
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51351
diff
changeset
|
818 # This logic only applies to general delta repositories and can be |
7083b33a2699
delta-find: explicitly deal with usage of the cached revision
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51351
diff
changeset
|
819 # disabled through configuration. Disabling reuse source delta is |
7083b33a2699
delta-find: explicitly deal with usage of the cached revision
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51351
diff
changeset
|
820 # useful when we want to make sure we recomputed "optimal" deltas. |
7083b33a2699
delta-find: explicitly deal with usage of the cached revision
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51351
diff
changeset
|
821 self.current_stage = _STAGE_CACHED |
7083b33a2699
delta-find: explicitly deal with usage of the cached revision
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51351
diff
changeset
|
822 self._internal_group = (self.cachedelta[0],) |
7083b33a2699
delta-find: explicitly deal with usage of the cached revision
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51351
diff
changeset
|
823 self._internal_idx = 0 |
7083b33a2699
delta-find: explicitly deal with usage of the cached revision
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51351
diff
changeset
|
824 self.current_group = self._internal_group |
7083b33a2699
delta-find: explicitly deal with usage of the cached revision
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51351
diff
changeset
|
825 self.tested.update(self.current_group) |
7083b33a2699
delta-find: explicitly deal with usage of the cached revision
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51351
diff
changeset
|
826 else: |
7083b33a2699
delta-find: explicitly deal with usage of the cached revision
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51351
diff
changeset
|
827 self._next_internal_group() |
51351
bbbfa00d1e67
delta-find: remove the "candidate groups" layer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51350
diff
changeset
|
828 |
bbbfa00d1e67
delta-find: remove the "candidate groups" layer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51350
diff
changeset
|
829 def _next_internal_group(self): |
bbbfa00d1e67
delta-find: remove the "candidate groups" layer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51350
diff
changeset
|
830 # self._internal_group can be larger than self.current_group |
bbbfa00d1e67
delta-find: remove the "candidate groups" layer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51350
diff
changeset
|
831 self._internal_idx = 0 |
bbbfa00d1e67
delta-find: remove the "candidate groups" layer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51350
diff
changeset
|
832 group = self._candidates_iterator.send(self._last_good) |
bbbfa00d1e67
delta-find: remove the "candidate groups" layer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51350
diff
changeset
|
833 if group is not None: |
bbbfa00d1e67
delta-find: remove the "candidate groups" layer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51350
diff
changeset
|
834 group = self._pre_filter_candidate_revs(group) |
bbbfa00d1e67
delta-find: remove the "candidate groups" layer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51350
diff
changeset
|
835 self._internal_group = group |
bbbfa00d1e67
delta-find: remove the "candidate groups" layer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51350
diff
changeset
|
836 if self._internal_group is None: |
bbbfa00d1e67
delta-find: remove the "candidate groups" layer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51350
diff
changeset
|
837 self.current_group = None |
bbbfa00d1e67
delta-find: remove the "candidate groups" layer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51350
diff
changeset
|
838 elif len(self._internal_group) == 0: |
bbbfa00d1e67
delta-find: remove the "candidate groups" layer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51350
diff
changeset
|
839 self.next_group() |
bbbfa00d1e67
delta-find: remove the "candidate groups" layer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51350
diff
changeset
|
840 else: |
bbbfa00d1e67
delta-find: remove the "candidate groups" layer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51350
diff
changeset
|
841 chunk_size = self.revlog.delta_config.candidate_group_chunk_size |
bbbfa00d1e67
delta-find: remove the "candidate groups" layer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51350
diff
changeset
|
842 if chunk_size > 0: |
bbbfa00d1e67
delta-find: remove the "candidate groups" layer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51350
diff
changeset
|
843 self.current_group = self._internal_group[:chunk_size] |
bbbfa00d1e67
delta-find: remove the "candidate groups" layer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51350
diff
changeset
|
844 self._internal_idx += chunk_size |
bbbfa00d1e67
delta-find: remove the "candidate groups" layer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51350
diff
changeset
|
845 else: |
bbbfa00d1e67
delta-find: remove the "candidate groups" layer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51350
diff
changeset
|
846 self.current_group = self._internal_group |
bbbfa00d1e67
delta-find: remove the "candidate groups" layer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51350
diff
changeset
|
847 self._internal_idx += len(self.current_group) |
bbbfa00d1e67
delta-find: remove the "candidate groups" layer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51350
diff
changeset
|
848 |
bbbfa00d1e67
delta-find: remove the "candidate groups" layer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51350
diff
changeset
|
849 self.tested.update(self.current_group) |
51346
2a333d791ecf
delta-find: introduce a base class for _DeltaSearch
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51345
diff
changeset
|
850 |
2a333d791ecf
delta-find: introduce a base class for _DeltaSearch
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51345
diff
changeset
|
851 def next_group(self, good_delta=None): |
51351
bbbfa00d1e67
delta-find: remove the "candidate groups" layer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51350
diff
changeset
|
852 old_good = self._last_good |
51334
d0d869fccd20
delta-find: move away from the generator API for _DeltaSearch
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51333
diff
changeset
|
853 if good_delta is not None: |
51360
99869dcf3ba0
delta-find: pass the full deltainfo to the _DeltaSearch class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51359
diff
changeset
|
854 self._last_good = good_delta |
51352
7083b33a2699
delta-find: explicitly deal with usage of the cached revision
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51351
diff
changeset
|
855 if self.current_stage == _STAGE_CACHED and good_delta is not None: |
7083b33a2699
delta-find: explicitly deal with usage of the cached revision
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51351
diff
changeset
|
856 # the cache is good, let us use the cache as requested |
7083b33a2699
delta-find: explicitly deal with usage of the cached revision
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51351
diff
changeset
|
857 self._candidates_iterator = None |
7083b33a2699
delta-find: explicitly deal with usage of the cached revision
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51351
diff
changeset
|
858 self._internal_group = None |
7083b33a2699
delta-find: explicitly deal with usage of the cached revision
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51351
diff
changeset
|
859 self._internal_idx = None |
7083b33a2699
delta-find: explicitly deal with usage of the cached revision
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51351
diff
changeset
|
860 self.current_group = None |
7083b33a2699
delta-find: explicitly deal with usage of the cached revision
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51351
diff
changeset
|
861 return |
7083b33a2699
delta-find: explicitly deal with usage of the cached revision
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51351
diff
changeset
|
862 |
51351
bbbfa00d1e67
delta-find: remove the "candidate groups" layer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51350
diff
changeset
|
863 if (self._internal_idx < len(self._internal_group)) and ( |
bbbfa00d1e67
delta-find: remove the "candidate groups" layer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51350
diff
changeset
|
864 old_good != good_delta |
bbbfa00d1e67
delta-find: remove the "candidate groups" layer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51350
diff
changeset
|
865 ): |
bbbfa00d1e67
delta-find: remove the "candidate groups" layer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51350
diff
changeset
|
866 # When the size of the candidate group is big, it can result in |
bbbfa00d1e67
delta-find: remove the "candidate groups" layer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51350
diff
changeset
|
867 # a quite significant performance impact. To reduce this, we |
bbbfa00d1e67
delta-find: remove the "candidate groups" layer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51350
diff
changeset
|
868 # can send them in smaller batches until the new batch does not |
bbbfa00d1e67
delta-find: remove the "candidate groups" layer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51350
diff
changeset
|
869 # provide any improvements. |
bbbfa00d1e67
delta-find: remove the "candidate groups" layer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51350
diff
changeset
|
870 # |
bbbfa00d1e67
delta-find: remove the "candidate groups" layer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51350
diff
changeset
|
871 # This might reduce the overall efficiency of the compression |
bbbfa00d1e67
delta-find: remove the "candidate groups" layer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51350
diff
changeset
|
872 # in some corner cases, but that should also prevent very |
bbbfa00d1e67
delta-find: remove the "candidate groups" layer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51350
diff
changeset
|
873 # pathological cases from being an issue. (eg. 20 000 |
bbbfa00d1e67
delta-find: remove the "candidate groups" layer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51350
diff
changeset
|
874 # candidates). |
bbbfa00d1e67
delta-find: remove the "candidate groups" layer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51350
diff
changeset
|
875 # |
bbbfa00d1e67
delta-find: remove the "candidate groups" layer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51350
diff
changeset
|
876 # XXX note that the ordering of the group becomes important as |
bbbfa00d1e67
delta-find: remove the "candidate groups" layer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51350
diff
changeset
|
877 # it now impacts the final result. The current order is |
bbbfa00d1e67
delta-find: remove the "candidate groups" layer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51350
diff
changeset
|
878 # unprocessed and can be improved. |
51509
e7bbeaa4f0a7
delta-search: fix crash caused by unbound variable
Raphaël Gomès <rgomes@octobus.net>
parents:
51360
diff
changeset
|
879 chunk_size = self.revlog.delta_config.candidate_group_chunk_size |
e7bbeaa4f0a7
delta-search: fix crash caused by unbound variable
Raphaël Gomès <rgomes@octobus.net>
parents:
51360
diff
changeset
|
880 next_idx = self._internal_idx + chunk_size |
51351
bbbfa00d1e67
delta-find: remove the "candidate groups" layer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51350
diff
changeset
|
881 self.current_group = self._internal_group[ |
bbbfa00d1e67
delta-find: remove the "candidate groups" layer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51350
diff
changeset
|
882 self._internal_idx : next_idx |
bbbfa00d1e67
delta-find: remove the "candidate groups" layer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51350
diff
changeset
|
883 ] |
bbbfa00d1e67
delta-find: remove the "candidate groups" layer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51350
diff
changeset
|
884 self.tested.update(self.current_group) |
bbbfa00d1e67
delta-find: remove the "candidate groups" layer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51350
diff
changeset
|
885 self._internal_idx = next_idx |
bbbfa00d1e67
delta-find: remove the "candidate groups" layer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51350
diff
changeset
|
886 else: |
bbbfa00d1e67
delta-find: remove the "candidate groups" layer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51350
diff
changeset
|
887 self._next_internal_group() |
39336
1c6ff52fe9cf
revlogdeltas: split candidate groups selection from the filtering logic
Boris Feld <boris.feld@octobus.net>
parents:
39335
diff
changeset
|
888 |
51335
02cc19f4f091
delta-find: move pre-filtering of candidates in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51334
diff
changeset
|
889 def _pre_filter_candidate_revs(self, temptative): |
02cc19f4f091
delta-find: move pre-filtering of candidates in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51334
diff
changeset
|
890 """filter possible candidate before computing a delta |
02cc19f4f091
delta-find: move pre-filtering of candidates in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51334
diff
changeset
|
891 |
02cc19f4f091
delta-find: move pre-filtering of candidates in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51334
diff
changeset
|
892 This function use various criteria to pre-filter candidate delta base |
02cc19f4f091
delta-find: move pre-filtering of candidates in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51334
diff
changeset
|
893 before we compute a delta and evaluate its quality. |
02cc19f4f091
delta-find: move pre-filtering of candidates in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51334
diff
changeset
|
894 |
02cc19f4f091
delta-find: move pre-filtering of candidates in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51334
diff
changeset
|
895 Such pre-filter limit the number of computed delta, an expensive operation. |
02cc19f4f091
delta-find: move pre-filtering of candidates in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51334
diff
changeset
|
896 |
02cc19f4f091
delta-find: move pre-filtering of candidates in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51334
diff
changeset
|
897 return the updated list of revision to test |
02cc19f4f091
delta-find: move pre-filtering of candidates in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51334
diff
changeset
|
898 """ |
02cc19f4f091
delta-find: move pre-filtering of candidates in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51334
diff
changeset
|
899 deltalength = self.revlog.length |
02cc19f4f091
delta-find: move pre-filtering of candidates in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51334
diff
changeset
|
900 deltaparent = self.revlog.deltaparent |
02cc19f4f091
delta-find: move pre-filtering of candidates in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51334
diff
changeset
|
901 |
02cc19f4f091
delta-find: move pre-filtering of candidates in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51334
diff
changeset
|
902 tested = self.tested |
02cc19f4f091
delta-find: move pre-filtering of candidates in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51334
diff
changeset
|
903 group = [] |
02cc19f4f091
delta-find: move pre-filtering of candidates in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51334
diff
changeset
|
904 for rev in temptative: |
02cc19f4f091
delta-find: move pre-filtering of candidates in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51334
diff
changeset
|
905 # skip over empty delta (no need to include them in a chain) |
02cc19f4f091
delta-find: move pre-filtering of candidates in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51334
diff
changeset
|
906 while not (rev == nullrev or rev in tested or deltalength(rev)): |
02cc19f4f091
delta-find: move pre-filtering of candidates in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51334
diff
changeset
|
907 tested.add(rev) |
02cc19f4f091
delta-find: move pre-filtering of candidates in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51334
diff
changeset
|
908 rev = deltaparent(rev) |
51336
898c212e1b2f
delta-find: move pre-filtering of individual revision in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51335
diff
changeset
|
909 if self._pre_filter_rev(rev): |
898c212e1b2f
delta-find: move pre-filtering of individual revision in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51335
diff
changeset
|
910 group.append(rev) |
898c212e1b2f
delta-find: move pre-filtering of individual revision in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51335
diff
changeset
|
911 else: |
898c212e1b2f
delta-find: move pre-filtering of individual revision in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51335
diff
changeset
|
912 self.tested.add(rev) |
898c212e1b2f
delta-find: move pre-filtering of individual revision in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51335
diff
changeset
|
913 return group |
898c212e1b2f
delta-find: move pre-filtering of individual revision in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51335
diff
changeset
|
914 |
51338
2407af4f2eea
delta-find: split the generic part of `_pre_filter_rev` in a method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51337
diff
changeset
|
915 def _pre_filter_rev_universal(self, rev): |
2407af4f2eea
delta-find: split the generic part of `_pre_filter_rev` in a method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51337
diff
changeset
|
916 """pre filtering that is need in all cases. |
2407af4f2eea
delta-find: split the generic part of `_pre_filter_rev` in a method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51337
diff
changeset
|
917 |
2407af4f2eea
delta-find: split the generic part of `_pre_filter_rev` in a method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51337
diff
changeset
|
918 return True if it seems okay to test a rev, False otherwise. |
2407af4f2eea
delta-find: split the generic part of `_pre_filter_rev` in a method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51337
diff
changeset
|
919 |
2407af4f2eea
delta-find: split the generic part of `_pre_filter_rev` in a method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51337
diff
changeset
|
920 used by _pre_filter_rev. |
2407af4f2eea
delta-find: split the generic part of `_pre_filter_rev` in a method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51337
diff
changeset
|
921 """ |
51337
ac8b798e602b
delta-find: drop the temporary indent
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51336
diff
changeset
|
922 # no need to try a delta against nullrev, this will be done as |
ac8b798e602b
delta-find: drop the temporary indent
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51336
diff
changeset
|
923 # a last resort. |
ac8b798e602b
delta-find: drop the temporary indent
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51336
diff
changeset
|
924 if rev == nullrev: |
ac8b798e602b
delta-find: drop the temporary indent
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51336
diff
changeset
|
925 return False |
ac8b798e602b
delta-find: drop the temporary indent
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51336
diff
changeset
|
926 # filter out revision we tested already |
ac8b798e602b
delta-find: drop the temporary indent
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51336
diff
changeset
|
927 if rev in self.tested: |
ac8b798e602b
delta-find: drop the temporary indent
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51336
diff
changeset
|
928 return False |
51335
02cc19f4f091
delta-find: move pre-filtering of candidates in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51334
diff
changeset
|
929 |
51337
ac8b798e602b
delta-find: drop the temporary indent
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51336
diff
changeset
|
930 # an higher authority deamed the base unworthy (e.g. censored) |
ac8b798e602b
delta-find: drop the temporary indent
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51336
diff
changeset
|
931 if self.excluded_bases is not None and rev in self.excluded_bases: |
ac8b798e602b
delta-find: drop the temporary indent
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51336
diff
changeset
|
932 return False |
ac8b798e602b
delta-find: drop the temporary indent
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51336
diff
changeset
|
933 # We are in some recomputation cases and that rev is too high |
ac8b798e602b
delta-find: drop the temporary indent
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51336
diff
changeset
|
934 # in the revlog |
ac8b798e602b
delta-find: drop the temporary indent
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51336
diff
changeset
|
935 if self.target_rev is not None and rev >= self.target_rev: |
ac8b798e602b
delta-find: drop the temporary indent
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51336
diff
changeset
|
936 return False |
51338
2407af4f2eea
delta-find: split the generic part of `_pre_filter_rev` in a method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51337
diff
changeset
|
937 # no delta for rawtext-changing revs (see "candelta" for why) |
2407af4f2eea
delta-find: split the generic part of `_pre_filter_rev` in a method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51337
diff
changeset
|
938 if self.revlog.flags(rev) & REVIDX_RAWTEXT_CHANGING_FLAGS: |
2407af4f2eea
delta-find: split the generic part of `_pre_filter_rev` in a method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51337
diff
changeset
|
939 return False |
2407af4f2eea
delta-find: split the generic part of `_pre_filter_rev` in a method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51337
diff
changeset
|
940 return True |
2407af4f2eea
delta-find: split the generic part of `_pre_filter_rev` in a method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51337
diff
changeset
|
941 |
51340
1ea56b10dd4a
delta-find: split the delta-chain part of `_pre_filter_rev` in a method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51339
diff
changeset
|
942 def _pre_filter_rev_delta_chain(self, rev): |
1ea56b10dd4a
delta-find: split the delta-chain part of `_pre_filter_rev` in a method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51339
diff
changeset
|
943 """pre filtering that is needed in sparse revlog cases |
51339
9a1239c362ae
delta-find: split the "sparse" part of `_pre_filter_rev` in a method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51338
diff
changeset
|
944 |
51340
1ea56b10dd4a
delta-find: split the delta-chain part of `_pre_filter_rev` in a method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51339
diff
changeset
|
945 return True if it seems okay to test a rev, False otherwise. |
1ea56b10dd4a
delta-find: split the delta-chain part of `_pre_filter_rev` in a method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51339
diff
changeset
|
946 |
1ea56b10dd4a
delta-find: split the delta-chain part of `_pre_filter_rev` in a method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51339
diff
changeset
|
947 used by _pre_filter_rev. |
1ea56b10dd4a
delta-find: split the delta-chain part of `_pre_filter_rev` in a method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51339
diff
changeset
|
948 """ |
51339
9a1239c362ae
delta-find: split the "sparse" part of `_pre_filter_rev` in a method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51338
diff
changeset
|
949 deltas_limit = self.revinfo.textlen * LIMIT_DELTA2TEXT |
9a1239c362ae
delta-find: split the "sparse" part of `_pre_filter_rev` in a method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51338
diff
changeset
|
950 # filter out delta base that will never produce good delta |
9a1239c362ae
delta-find: split the "sparse" part of `_pre_filter_rev` in a method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51338
diff
changeset
|
951 # |
9a1239c362ae
delta-find: split the "sparse" part of `_pre_filter_rev` in a method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51338
diff
changeset
|
952 # if the delta of that base is already bigger than the limit |
9a1239c362ae
delta-find: split the "sparse" part of `_pre_filter_rev` in a method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51338
diff
changeset
|
953 # for the delta chain size, doing a delta is hopeless. |
9a1239c362ae
delta-find: split the "sparse" part of `_pre_filter_rev` in a method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51338
diff
changeset
|
954 if deltas_limit < self.revlog.length(rev): |
9a1239c362ae
delta-find: split the "sparse" part of `_pre_filter_rev` in a method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51338
diff
changeset
|
955 return False |
9a1239c362ae
delta-find: split the "sparse" part of `_pre_filter_rev` in a method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51338
diff
changeset
|
956 |
9a1239c362ae
delta-find: split the "sparse" part of `_pre_filter_rev` in a method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51338
diff
changeset
|
957 # If we reach here, we are about to build and test a delta. |
9a1239c362ae
delta-find: split the "sparse" part of `_pre_filter_rev` in a method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51338
diff
changeset
|
958 # The delta building process will compute the chaininfo in all |
9a1239c362ae
delta-find: split the "sparse" part of `_pre_filter_rev` in a method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51338
diff
changeset
|
959 # case, since that computation is cached, it is fine to access |
9a1239c362ae
delta-find: split the "sparse" part of `_pre_filter_rev` in a method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51338
diff
changeset
|
960 # it here too. |
9a1239c362ae
delta-find: split the "sparse" part of `_pre_filter_rev` in a method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51338
diff
changeset
|
961 chainlen, chainsize = self.revlog._chaininfo(rev) |
9a1239c362ae
delta-find: split the "sparse" part of `_pre_filter_rev` in a method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51338
diff
changeset
|
962 # if chain will be too long, skip base |
9a1239c362ae
delta-find: split the "sparse" part of `_pre_filter_rev` in a method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51338
diff
changeset
|
963 if ( |
9a1239c362ae
delta-find: split the "sparse" part of `_pre_filter_rev` in a method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51338
diff
changeset
|
964 self.revlog.delta_config.max_chain_len |
9a1239c362ae
delta-find: split the "sparse" part of `_pre_filter_rev` in a method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51338
diff
changeset
|
965 and chainlen >= self.revlog.delta_config.max_chain_len |
9a1239c362ae
delta-find: split the "sparse" part of `_pre_filter_rev` in a method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51338
diff
changeset
|
966 ): |
9a1239c362ae
delta-find: split the "sparse" part of `_pre_filter_rev` in a method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51338
diff
changeset
|
967 return False |
9a1239c362ae
delta-find: split the "sparse" part of `_pre_filter_rev` in a method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51338
diff
changeset
|
968 # if chain already have too much data, skip base |
9a1239c362ae
delta-find: split the "sparse" part of `_pre_filter_rev` in a method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51338
diff
changeset
|
969 if deltas_limit < chainsize: |
9a1239c362ae
delta-find: split the "sparse" part of `_pre_filter_rev` in a method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51338
diff
changeset
|
970 return False |
51340
1ea56b10dd4a
delta-find: split the delta-chain part of `_pre_filter_rev` in a method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51339
diff
changeset
|
971 return True |
51339
9a1239c362ae
delta-find: split the "sparse" part of `_pre_filter_rev` in a method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51338
diff
changeset
|
972 |
51340
1ea56b10dd4a
delta-find: split the delta-chain part of `_pre_filter_rev` in a method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51339
diff
changeset
|
973 def _pre_filter_rev(self, rev): |
1ea56b10dd4a
delta-find: split the delta-chain part of `_pre_filter_rev` in a method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51339
diff
changeset
|
974 """return True if it seems okay to test a rev, False otherwise""" |
1ea56b10dd4a
delta-find: split the delta-chain part of `_pre_filter_rev` in a method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51339
diff
changeset
|
975 if not self._pre_filter_rev_universal(rev): |
1ea56b10dd4a
delta-find: split the delta-chain part of `_pre_filter_rev` in a method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51339
diff
changeset
|
976 return False |
1ea56b10dd4a
delta-find: split the delta-chain part of `_pre_filter_rev` in a method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51339
diff
changeset
|
977 if not self._pre_filter_rev_delta_chain(rev): |
1ea56b10dd4a
delta-find: split the delta-chain part of `_pre_filter_rev` in a method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51339
diff
changeset
|
978 return False |
51339
9a1239c362ae
delta-find: split the "sparse" part of `_pre_filter_rev` in a method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51338
diff
changeset
|
979 return True |
9a1239c362ae
delta-find: split the "sparse" part of `_pre_filter_rev` in a method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51338
diff
changeset
|
980 |
51353
5cc04a6da19d
delta-find: move the emotion of parents in a dedicated method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51352
diff
changeset
|
981 def _iter_parents(self): |
5cc04a6da19d
delta-find: move the emotion of parents in a dedicated method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51352
diff
changeset
|
982 # exclude already lazy tested base if any |
5cc04a6da19d
delta-find: move the emotion of parents in a dedicated method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51352
diff
changeset
|
983 parents = [p for p in (self.p1, self.p2) if p != nullrev] |
5cc04a6da19d
delta-find: move the emotion of parents in a dedicated method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51352
diff
changeset
|
984 |
5cc04a6da19d
delta-find: move the emotion of parents in a dedicated method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51352
diff
changeset
|
985 self.current_stage = _STAGE_PARENTS |
5cc04a6da19d
delta-find: move the emotion of parents in a dedicated method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51352
diff
changeset
|
986 if ( |
5cc04a6da19d
delta-find: move the emotion of parents in a dedicated method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51352
diff
changeset
|
987 not self.revlog.delta_config.delta_both_parents |
5cc04a6da19d
delta-find: move the emotion of parents in a dedicated method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51352
diff
changeset
|
988 and len(parents) == 2 |
5cc04a6da19d
delta-find: move the emotion of parents in a dedicated method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51352
diff
changeset
|
989 ): |
5cc04a6da19d
delta-find: move the emotion of parents in a dedicated method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51352
diff
changeset
|
990 parents.sort() |
5cc04a6da19d
delta-find: move the emotion of parents in a dedicated method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51352
diff
changeset
|
991 # To minimize the chance of having to build a fulltext, |
5cc04a6da19d
delta-find: move the emotion of parents in a dedicated method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51352
diff
changeset
|
992 # pick first whichever parent is closest to us (max rev) |
5cc04a6da19d
delta-find: move the emotion of parents in a dedicated method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51352
diff
changeset
|
993 yield (parents[1],) |
5cc04a6da19d
delta-find: move the emotion of parents in a dedicated method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51352
diff
changeset
|
994 # then the other one (min rev) if the first did not fit |
5cc04a6da19d
delta-find: move the emotion of parents in a dedicated method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51352
diff
changeset
|
995 yield (parents[0],) |
5cc04a6da19d
delta-find: move the emotion of parents in a dedicated method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51352
diff
changeset
|
996 elif len(parents) > 0: |
5cc04a6da19d
delta-find: move the emotion of parents in a dedicated method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51352
diff
changeset
|
997 # Test all parents (1 or 2), and keep the best candidate |
5cc04a6da19d
delta-find: move the emotion of parents in a dedicated method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51352
diff
changeset
|
998 yield parents |
5cc04a6da19d
delta-find: move the emotion of parents in a dedicated method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51352
diff
changeset
|
999 |
51354
94fe4474b053
delta-find: move the emotion of prev in a dedicated method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51353
diff
changeset
|
1000 def _iter_prev(self): |
94fe4474b053
delta-find: move the emotion of prev in a dedicated method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51353
diff
changeset
|
1001 # other approach failed try against prev to hopefully save us a |
94fe4474b053
delta-find: move the emotion of prev in a dedicated method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51353
diff
changeset
|
1002 # fulltext. |
94fe4474b053
delta-find: move the emotion of prev in a dedicated method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51353
diff
changeset
|
1003 self.current_stage = _STAGE_PREV |
94fe4474b053
delta-find: move the emotion of prev in a dedicated method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51353
diff
changeset
|
1004 yield (self.target_rev - 1,) |
94fe4474b053
delta-find: move the emotion of prev in a dedicated method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51353
diff
changeset
|
1005 |
51357
d7e2acdd50ba
delta-find: split the _DeltaSearch class in two
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51356
diff
changeset
|
1006 def _iter_groups(self): |
d7e2acdd50ba
delta-find: split the _DeltaSearch class in two
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51356
diff
changeset
|
1007 good = None |
d7e2acdd50ba
delta-find: split the _DeltaSearch class in two
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51356
diff
changeset
|
1008 for group in self._iter_parents(): |
d7e2acdd50ba
delta-find: split the _DeltaSearch class in two
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51356
diff
changeset
|
1009 good = yield group |
d7e2acdd50ba
delta-find: split the _DeltaSearch class in two
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51356
diff
changeset
|
1010 if good is not None: |
d7e2acdd50ba
delta-find: split the _DeltaSearch class in two
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51356
diff
changeset
|
1011 break |
d7e2acdd50ba
delta-find: split the _DeltaSearch class in two
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51356
diff
changeset
|
1012 else: |
d7e2acdd50ba
delta-find: split the _DeltaSearch class in two
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51356
diff
changeset
|
1013 assert good is None |
d7e2acdd50ba
delta-find: split the _DeltaSearch class in two
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51356
diff
changeset
|
1014 yield from self._iter_prev() |
d7e2acdd50ba
delta-find: split the _DeltaSearch class in two
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51356
diff
changeset
|
1015 yield None |
d7e2acdd50ba
delta-find: split the _DeltaSearch class in two
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51356
diff
changeset
|
1016 |
d7e2acdd50ba
delta-find: split the _DeltaSearch class in two
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51356
diff
changeset
|
1017 |
d7e2acdd50ba
delta-find: split the _DeltaSearch class in two
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51356
diff
changeset
|
1018 class _SparseDeltaSearch(_GeneralDeltaSearch): |
d7e2acdd50ba
delta-find: split the _DeltaSearch class in two
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51356
diff
changeset
|
1019 """Delta search variants for sparse-revlog""" |
d7e2acdd50ba
delta-find: split the _DeltaSearch class in two
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51356
diff
changeset
|
1020 |
51358
383e99f6bc99
delta-find: move sparse-revlog delta checks in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51357
diff
changeset
|
1021 def is_good_delta_info(self, deltainfo): |
383e99f6bc99
delta-find: move sparse-revlog delta checks in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51357
diff
changeset
|
1022 """Returns True if the given delta is good. |
383e99f6bc99
delta-find: move sparse-revlog delta checks in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51357
diff
changeset
|
1023 |
383e99f6bc99
delta-find: move sparse-revlog delta checks in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51357
diff
changeset
|
1024 Good means that it is within the disk span, disk size, and chain length |
383e99f6bc99
delta-find: move sparse-revlog delta checks in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51357
diff
changeset
|
1025 bounds that we know to be performant. |
383e99f6bc99
delta-find: move sparse-revlog delta checks in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51357
diff
changeset
|
1026 """ |
383e99f6bc99
delta-find: move sparse-revlog delta checks in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51357
diff
changeset
|
1027 if not self._is_good_delta_info_universal(deltainfo): |
383e99f6bc99
delta-find: move sparse-revlog delta checks in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51357
diff
changeset
|
1028 return False |
383e99f6bc99
delta-find: move sparse-revlog delta checks in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51357
diff
changeset
|
1029 if not self._is_good_delta_info_chain_quality(deltainfo): |
383e99f6bc99
delta-find: move sparse-revlog delta checks in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51357
diff
changeset
|
1030 return False |
383e99f6bc99
delta-find: move sparse-revlog delta checks in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51357
diff
changeset
|
1031 if not self._is_good_delta_info_snapshot_constraints(deltainfo): |
383e99f6bc99
delta-find: move sparse-revlog delta checks in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51357
diff
changeset
|
1032 return False |
383e99f6bc99
delta-find: move sparse-revlog delta checks in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51357
diff
changeset
|
1033 return True |
383e99f6bc99
delta-find: move sparse-revlog delta checks in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51357
diff
changeset
|
1034 |
383e99f6bc99
delta-find: move sparse-revlog delta checks in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51357
diff
changeset
|
1035 def _is_good_delta_info_snapshot_constraints(self, deltainfo): |
383e99f6bc99
delta-find: move sparse-revlog delta checks in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51357
diff
changeset
|
1036 """Returns True if the chain associated with snapshots |
383e99f6bc99
delta-find: move sparse-revlog delta checks in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51357
diff
changeset
|
1037 |
383e99f6bc99
delta-find: move sparse-revlog delta checks in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51357
diff
changeset
|
1038 This performs checks for format that use sparse-revlog and intermediate |
383e99f6bc99
delta-find: move sparse-revlog delta checks in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51357
diff
changeset
|
1039 snapshots. |
383e99f6bc99
delta-find: move sparse-revlog delta checks in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51357
diff
changeset
|
1040 |
383e99f6bc99
delta-find: move sparse-revlog delta checks in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51357
diff
changeset
|
1041 This is used by is_good_delta_info. |
383e99f6bc99
delta-find: move sparse-revlog delta checks in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51357
diff
changeset
|
1042 """ |
383e99f6bc99
delta-find: move sparse-revlog delta checks in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51357
diff
changeset
|
1043 # if not a snapshot, this method has no filtering to do |
383e99f6bc99
delta-find: move sparse-revlog delta checks in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51357
diff
changeset
|
1044 if deltainfo.snapshotdepth is None: |
383e99f6bc99
delta-find: move sparse-revlog delta checks in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51357
diff
changeset
|
1045 return True |
383e99f6bc99
delta-find: move sparse-revlog delta checks in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51357
diff
changeset
|
1046 # bad delta from intermediate snapshot size limit |
383e99f6bc99
delta-find: move sparse-revlog delta checks in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51357
diff
changeset
|
1047 # |
383e99f6bc99
delta-find: move sparse-revlog delta checks in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51357
diff
changeset
|
1048 # If an intermediate snapshot size is higher than the limit. The |
383e99f6bc99
delta-find: move sparse-revlog delta checks in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51357
diff
changeset
|
1049 # limit exist to prevent endless chain of intermediate delta to be |
383e99f6bc99
delta-find: move sparse-revlog delta checks in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51357
diff
changeset
|
1050 # created. |
383e99f6bc99
delta-find: move sparse-revlog delta checks in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51357
diff
changeset
|
1051 if ( |
383e99f6bc99
delta-find: move sparse-revlog delta checks in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51357
diff
changeset
|
1052 self.revinfo.textlen >> deltainfo.snapshotdepth |
383e99f6bc99
delta-find: move sparse-revlog delta checks in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51357
diff
changeset
|
1053 ) < deltainfo.deltalen: |
383e99f6bc99
delta-find: move sparse-revlog delta checks in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51357
diff
changeset
|
1054 return False |
383e99f6bc99
delta-find: move sparse-revlog delta checks in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51357
diff
changeset
|
1055 |
383e99f6bc99
delta-find: move sparse-revlog delta checks in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51357
diff
changeset
|
1056 # bad delta if new intermediate snapshot is larger than the previous |
383e99f6bc99
delta-find: move sparse-revlog delta checks in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51357
diff
changeset
|
1057 # snapshot |
383e99f6bc99
delta-find: move sparse-revlog delta checks in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51357
diff
changeset
|
1058 if self.revlog.length(deltainfo.base) < deltainfo.deltalen: |
383e99f6bc99
delta-find: move sparse-revlog delta checks in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51357
diff
changeset
|
1059 return False |
383e99f6bc99
delta-find: move sparse-revlog delta checks in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51357
diff
changeset
|
1060 |
383e99f6bc99
delta-find: move sparse-revlog delta checks in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51357
diff
changeset
|
1061 return True |
383e99f6bc99
delta-find: move sparse-revlog delta checks in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51357
diff
changeset
|
1062 |
51359
a224ce5694b3
delta-find: move sparse-revlog pre-filtering in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51358
diff
changeset
|
1063 def _pre_filter_rev(self, rev): |
a224ce5694b3
delta-find: move sparse-revlog pre-filtering in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51358
diff
changeset
|
1064 """return True if it seems okay to test a rev, False otherwise""" |
a224ce5694b3
delta-find: move sparse-revlog pre-filtering in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51358
diff
changeset
|
1065 if not self._pre_filter_rev_universal(rev): |
a224ce5694b3
delta-find: move sparse-revlog pre-filtering in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51358
diff
changeset
|
1066 return False |
a224ce5694b3
delta-find: move sparse-revlog pre-filtering in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51358
diff
changeset
|
1067 if not self._pre_filter_rev_delta_chain(rev): |
a224ce5694b3
delta-find: move sparse-revlog pre-filtering in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51358
diff
changeset
|
1068 return False |
a224ce5694b3
delta-find: move sparse-revlog pre-filtering in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51358
diff
changeset
|
1069 if not self._pre_filter_rev_sparse(rev): |
a224ce5694b3
delta-find: move sparse-revlog pre-filtering in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51358
diff
changeset
|
1070 return False |
a224ce5694b3
delta-find: move sparse-revlog pre-filtering in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51358
diff
changeset
|
1071 return True |
a224ce5694b3
delta-find: move sparse-revlog pre-filtering in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51358
diff
changeset
|
1072 |
a224ce5694b3
delta-find: move sparse-revlog pre-filtering in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51358
diff
changeset
|
1073 def _pre_filter_rev_sparse(self, rev): |
a224ce5694b3
delta-find: move sparse-revlog pre-filtering in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51358
diff
changeset
|
1074 """pre filtering that is needed in sparse revlog cases |
a224ce5694b3
delta-find: move sparse-revlog pre-filtering in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51358
diff
changeset
|
1075 |
a224ce5694b3
delta-find: move sparse-revlog pre-filtering in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51358
diff
changeset
|
1076 return True if it seems okay to test a rev, False otherwise. |
a224ce5694b3
delta-find: move sparse-revlog pre-filtering in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51358
diff
changeset
|
1077 |
a224ce5694b3
delta-find: move sparse-revlog pre-filtering in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51358
diff
changeset
|
1078 used by _pre_filter_rev. |
a224ce5694b3
delta-find: move sparse-revlog pre-filtering in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51358
diff
changeset
|
1079 """ |
a224ce5694b3
delta-find: move sparse-revlog pre-filtering in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51358
diff
changeset
|
1080 assert self.revlog.delta_config.sparse_revlog |
a224ce5694b3
delta-find: move sparse-revlog pre-filtering in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51358
diff
changeset
|
1081 # if the revision we test again is too small, the resulting delta |
a224ce5694b3
delta-find: move sparse-revlog pre-filtering in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51358
diff
changeset
|
1082 # will be large anyway as that amount of data to be added is big |
a224ce5694b3
delta-find: move sparse-revlog pre-filtering in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51358
diff
changeset
|
1083 if self.revlog.rawsize(rev) < (self.textlen // LIMIT_BASE2TEXT): |
a224ce5694b3
delta-find: move sparse-revlog pre-filtering in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51358
diff
changeset
|
1084 return False |
a224ce5694b3
delta-find: move sparse-revlog pre-filtering in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51358
diff
changeset
|
1085 |
a224ce5694b3
delta-find: move sparse-revlog pre-filtering in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51358
diff
changeset
|
1086 if self.revlog.delta_config.upper_bound_comp is not None: |
a224ce5694b3
delta-find: move sparse-revlog pre-filtering in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51358
diff
changeset
|
1087 maxcomp = self.revlog.delta_config.upper_bound_comp |
a224ce5694b3
delta-find: move sparse-revlog pre-filtering in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51358
diff
changeset
|
1088 basenotsnap = (self.p1, self.p2, nullrev) |
a224ce5694b3
delta-find: move sparse-revlog pre-filtering in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51358
diff
changeset
|
1089 if rev not in basenotsnap and self.revlog.issnapshot(rev): |
a224ce5694b3
delta-find: move sparse-revlog pre-filtering in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51358
diff
changeset
|
1090 snapshotdepth = self.revlog.snapshotdepth(rev) |
a224ce5694b3
delta-find: move sparse-revlog pre-filtering in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51358
diff
changeset
|
1091 # If text is significantly larger than the base, we can |
a224ce5694b3
delta-find: move sparse-revlog pre-filtering in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51358
diff
changeset
|
1092 # expect the resulting delta to be proportional to the size |
a224ce5694b3
delta-find: move sparse-revlog pre-filtering in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51358
diff
changeset
|
1093 # difference |
a224ce5694b3
delta-find: move sparse-revlog pre-filtering in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51358
diff
changeset
|
1094 revsize = self.revlog.rawsize(rev) |
a224ce5694b3
delta-find: move sparse-revlog pre-filtering in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51358
diff
changeset
|
1095 rawsizedistance = max(self.textlen - revsize, 0) |
a224ce5694b3
delta-find: move sparse-revlog pre-filtering in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51358
diff
changeset
|
1096 # use an estimate of the compression upper bound. |
a224ce5694b3
delta-find: move sparse-revlog pre-filtering in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51358
diff
changeset
|
1097 lowestrealisticdeltalen = rawsizedistance // maxcomp |
a224ce5694b3
delta-find: move sparse-revlog pre-filtering in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51358
diff
changeset
|
1098 |
a224ce5694b3
delta-find: move sparse-revlog pre-filtering in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51358
diff
changeset
|
1099 # check the absolute constraint on the delta size |
a224ce5694b3
delta-find: move sparse-revlog pre-filtering in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51358
diff
changeset
|
1100 snapshotlimit = self.textlen >> snapshotdepth |
a224ce5694b3
delta-find: move sparse-revlog pre-filtering in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51358
diff
changeset
|
1101 if snapshotlimit < lowestrealisticdeltalen: |
a224ce5694b3
delta-find: move sparse-revlog pre-filtering in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51358
diff
changeset
|
1102 # delta lower bound is larger than accepted upper |
a224ce5694b3
delta-find: move sparse-revlog pre-filtering in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51358
diff
changeset
|
1103 # bound |
a224ce5694b3
delta-find: move sparse-revlog pre-filtering in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51358
diff
changeset
|
1104 return False |
a224ce5694b3
delta-find: move sparse-revlog pre-filtering in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51358
diff
changeset
|
1105 |
a224ce5694b3
delta-find: move sparse-revlog pre-filtering in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51358
diff
changeset
|
1106 # check the relative constraint on the delta size |
a224ce5694b3
delta-find: move sparse-revlog pre-filtering in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51358
diff
changeset
|
1107 revlength = self.revlog.length(rev) |
a224ce5694b3
delta-find: move sparse-revlog pre-filtering in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51358
diff
changeset
|
1108 if revlength < lowestrealisticdeltalen: |
a224ce5694b3
delta-find: move sparse-revlog pre-filtering in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51358
diff
changeset
|
1109 # delta probable lower bound is larger than target |
a224ce5694b3
delta-find: move sparse-revlog pre-filtering in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51358
diff
changeset
|
1110 # base |
a224ce5694b3
delta-find: move sparse-revlog pre-filtering in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51358
diff
changeset
|
1111 return False |
a224ce5694b3
delta-find: move sparse-revlog pre-filtering in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51358
diff
changeset
|
1112 return True |
a224ce5694b3
delta-find: move sparse-revlog pre-filtering in the associated class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51358
diff
changeset
|
1113 |
51355
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1114 def _iter_snapshots_base(self): |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1115 assert self.revlog.delta_config.sparse_revlog |
51356
701caeabbee7
delta-find: finish reworking the snapshot logic and drop more layer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51355
diff
changeset
|
1116 assert self.current_stage == _STAGE_SNAPSHOT |
51355
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1117 prev = self.target_rev - 1 |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1118 deltachain = lambda rev: self.revlog._deltachain(rev)[0] |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1119 |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1120 parents = [p for p in (self.p1, self.p2) if p != nullrev] |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1121 if not parents: |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1122 return |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1123 # See if we can use an existing snapshot in the parent chains to |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1124 # use as a base for a new intermediate-snapshot |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1125 # |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1126 # search for snapshot in parents delta chain map: snapshot-level: |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1127 # snapshot-rev |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1128 parents_snaps = collections.defaultdict(set) |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1129 candidate_chains = [deltachain(p) for p in parents] |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1130 for chain in candidate_chains: |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1131 for idx, s in enumerate(chain): |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1132 if not self.revlog.issnapshot(s): |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1133 break |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1134 parents_snaps[idx].add(s) |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1135 snapfloor = min(parents_snaps[0]) + 1 |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1136 self.snapshot_cache.update(self.revlog, snapfloor) |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1137 # search for the highest "unrelated" revision |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1138 # |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1139 # Adding snapshots used by "unrelated" revision increase the odd we |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1140 # reuse an independant, yet better snapshot chain. |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1141 # |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1142 # XXX instead of building a set of revisions, we could lazily |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1143 # enumerate over the chains. That would be more efficient, however |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1144 # we stick to simple code for now. |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1145 all_revs = set() |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1146 for chain in candidate_chains: |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1147 all_revs.update(chain) |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1148 other = None |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1149 for r in self.revlog.revs(prev, snapfloor): |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1150 if r not in all_revs: |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1151 other = r |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1152 break |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1153 if other is not None: |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1154 # To avoid unfair competition, we won't use unrelated |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1155 # intermediate snapshot that are deeper than the ones from the |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1156 # parent delta chain. |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1157 max_depth = max(parents_snaps.keys()) |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1158 chain = deltachain(other) |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1159 for depth, s in enumerate(chain): |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1160 if s < snapfloor: |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1161 continue |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1162 if max_depth < depth: |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1163 break |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1164 if not self.revlog.issnapshot(s): |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1165 break |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1166 parents_snaps[depth].add(s) |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1167 # Test them as possible intermediate snapshot base We test them |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1168 # from highest to lowest level. High level one are more likely to |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1169 # result in small delta |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1170 floor = None |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1171 for idx, snaps in sorted(parents_snaps.items(), reverse=True): |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1172 siblings = set() |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1173 for s in snaps: |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1174 siblings.update(self.snapshot_cache.snapshots[s]) |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1175 # Before considering making a new intermediate snapshot, we |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1176 # check if an existing snapshot, children of base we consider, |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1177 # would be suitable. |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1178 # |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1179 # It give a change to reuse a delta chain "unrelated" to the |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1180 # current revision instead of starting our own. Without such |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1181 # re-use, topological branches would keep reopening new chains. |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1182 # Creating more and more snapshot as the repository grow. |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1183 |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1184 if floor is not None: |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1185 # We only do this for siblings created after the one in our |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1186 # parent's delta chain. Those created before has less |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1187 # chances to be valid base since our ancestors had to |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1188 # create a new snapshot. |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1189 siblings = [r for r in siblings if floor < r] |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1190 yield tuple(sorted(siblings)) |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1191 # then test the base from our parent's delta chain. |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1192 yield tuple(sorted(snaps)) |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1193 floor = min(snaps) |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1194 # No suitable base found in the parent chain, search if any full |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1195 # snapshots emitted since parent's base would be a suitable base |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1196 # for an intermediate snapshot. |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1197 # |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1198 # It give a chance to reuse a delta chain unrelated to the current |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1199 # revisions instead of starting our own. Without such re-use, |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1200 # topological branches would keep reopening new full chains. |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1201 # Creating more and more snapshot as the repository grow. |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1202 full = [ |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1203 r for r in self.snapshot_cache.snapshots[nullrev] if snapfloor <= r |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1204 ] |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1205 yield tuple(sorted(full)) |
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1206 |
51356
701caeabbee7
delta-find: finish reworking the snapshot logic and drop more layer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51355
diff
changeset
|
1207 def _iter_snapshots(self): |
701caeabbee7
delta-find: finish reworking the snapshot logic and drop more layer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51355
diff
changeset
|
1208 assert self.revlog.delta_config.sparse_revlog |
701caeabbee7
delta-find: finish reworking the snapshot logic and drop more layer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51355
diff
changeset
|
1209 self.current_stage = _STAGE_SNAPSHOT |
51321
c83074405276
delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51320
diff
changeset
|
1210 good = None |
51356
701caeabbee7
delta-find: finish reworking the snapshot logic and drop more layer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51355
diff
changeset
|
1211 groups = self._iter_snapshots_base() |
51321
c83074405276
delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51320
diff
changeset
|
1212 for candidates in groups: |
c83074405276
delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51320
diff
changeset
|
1213 good = yield candidates |
c83074405276
delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51320
diff
changeset
|
1214 if good is not None: |
c83074405276
delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51320
diff
changeset
|
1215 break |
c83074405276
delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51320
diff
changeset
|
1216 # if we have a refinable value, try to refine it |
51360
99869dcf3ba0
delta-find: pass the full deltainfo to the _DeltaSearch class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51359
diff
changeset
|
1217 if good is not None and good.snapshotdepth is not None: |
51350
670e68729aa7
delta-find: stop using heuristic to determine if we are creating a snapshot
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51349
diff
changeset
|
1218 assert self.current_stage == _STAGE_SNAPSHOT |
51321
c83074405276
delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51320
diff
changeset
|
1219 # refine snapshot down |
c83074405276
delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51320
diff
changeset
|
1220 previous = None |
c83074405276
delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51320
diff
changeset
|
1221 while previous != good: |
c83074405276
delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51320
diff
changeset
|
1222 previous = good |
51360
99869dcf3ba0
delta-find: pass the full deltainfo to the _DeltaSearch class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51359
diff
changeset
|
1223 base = self.revlog.deltaparent(good.base) |
51321
c83074405276
delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51320
diff
changeset
|
1224 if base == nullrev: |
c83074405276
delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51320
diff
changeset
|
1225 break |
c83074405276
delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51320
diff
changeset
|
1226 good = yield (base,) |
c83074405276
delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51320
diff
changeset
|
1227 # refine snapshot up |
c83074405276
delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51320
diff
changeset
|
1228 if not self.snapshot_cache.snapshots: |
51360
99869dcf3ba0
delta-find: pass the full deltainfo to the _DeltaSearch class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51359
diff
changeset
|
1229 self.snapshot_cache.update(self.revlog, good.base + 1) |
51321
c83074405276
delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51320
diff
changeset
|
1230 previous = None |
c83074405276
delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51320
diff
changeset
|
1231 while good != previous: |
c83074405276
delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51320
diff
changeset
|
1232 previous = good |
c83074405276
delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51320
diff
changeset
|
1233 children = tuple( |
51360
99869dcf3ba0
delta-find: pass the full deltainfo to the _DeltaSearch class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51359
diff
changeset
|
1234 sorted(c for c in self.snapshot_cache.snapshots[good.base]) |
51321
c83074405276
delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51320
diff
changeset
|
1235 ) |
c83074405276
delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51320
diff
changeset
|
1236 good = yield children |
c83074405276
delta-find: move `_refinedgroups` on the `_DeltaSearch` object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51320
diff
changeset
|
1237 yield None |
39496
2f9f7889549b
snapshot: introduce an intermediate `_refinedgroups` generator
Boris Feld <boris.feld@octobus.net>
parents:
39495
diff
changeset
|
1238 |
51356
701caeabbee7
delta-find: finish reworking the snapshot logic and drop more layer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51355
diff
changeset
|
1239 def _iter_groups(self): |
701caeabbee7
delta-find: finish reworking the snapshot logic and drop more layer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51355
diff
changeset
|
1240 good = None |
701caeabbee7
delta-find: finish reworking the snapshot logic and drop more layer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51355
diff
changeset
|
1241 for group in self._iter_parents(): |
701caeabbee7
delta-find: finish reworking the snapshot logic and drop more layer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51355
diff
changeset
|
1242 good = yield group |
701caeabbee7
delta-find: finish reworking the snapshot logic and drop more layer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51355
diff
changeset
|
1243 if good is not None: |
701caeabbee7
delta-find: finish reworking the snapshot logic and drop more layer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51355
diff
changeset
|
1244 break |
51355
fac6038b11f5
delta-find: move the base of the delta search in its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51354
diff
changeset
|
1245 else: |
51356
701caeabbee7
delta-find: finish reworking the snapshot logic and drop more layer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51355
diff
changeset
|
1246 assert good is None |
51357
d7e2acdd50ba
delta-find: split the _DeltaSearch class in two
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51356
diff
changeset
|
1247 assert self.revlog.delta_config.sparse_revlog |
d7e2acdd50ba
delta-find: split the _DeltaSearch class in two
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51356
diff
changeset
|
1248 # If sparse revlog is enabled, we can try to refine the |
d7e2acdd50ba
delta-find: split the _DeltaSearch class in two
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51356
diff
changeset
|
1249 # available deltas |
d7e2acdd50ba
delta-find: split the _DeltaSearch class in two
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51356
diff
changeset
|
1250 iter_snap = self._iter_snapshots() |
d7e2acdd50ba
delta-find: split the _DeltaSearch class in two
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51356
diff
changeset
|
1251 group = iter_snap.send(None) |
d7e2acdd50ba
delta-find: split the _DeltaSearch class in two
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51356
diff
changeset
|
1252 while group is not None: |
d7e2acdd50ba
delta-find: split the _DeltaSearch class in two
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51356
diff
changeset
|
1253 good = yield group |
d7e2acdd50ba
delta-find: split the _DeltaSearch class in two
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51356
diff
changeset
|
1254 group = iter_snap.send(good) |
51356
701caeabbee7
delta-find: finish reworking the snapshot logic and drop more layer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51355
diff
changeset
|
1255 yield None |
39334
507f5b1dd7c8
revlogdeltas: extract _getcandidaterevs in a function
Boris Feld <boris.feld@octobus.net>
parents:
39333
diff
changeset
|
1256 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1257 |
49678
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1258 class SnapshotCache: |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1259 __slots__ = ('snapshots', '_start_rev', '_end_rev') |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1260 |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1261 def __init__(self): |
49679
b670eb3dd6c9
delta-find: use sets instead of list in the snapshot cache
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49678
diff
changeset
|
1262 self.snapshots = collections.defaultdict(set) |
49678
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1263 self._start_rev = None |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1264 self._end_rev = None |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1265 |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1266 def update(self, revlog, start_rev=0): |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1267 """find snapshots from start_rev to tip""" |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1268 nb_revs = len(revlog) |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1269 end_rev = nb_revs - 1 |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1270 if start_rev > end_rev: |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1271 return # range is empty |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1272 |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1273 if self._start_rev is None: |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1274 assert self._end_rev is None |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1275 self._update(revlog, start_rev, end_rev) |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1276 elif not (self._start_rev <= start_rev and end_rev <= self._end_rev): |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1277 if start_rev < self._start_rev: |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1278 self._update(revlog, start_rev, self._start_rev - 1) |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1279 if self._end_rev < end_rev: |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1280 self._update(revlog, self._end_rev + 1, end_rev) |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1281 |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1282 if self._start_rev is None: |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1283 assert self._end_rev is None |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1284 self._end_rev = end_rev |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1285 self._start_rev = start_rev |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1286 else: |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1287 self._start_rev = min(self._start_rev, start_rev) |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1288 self._end_rev = max(self._end_rev, end_rev) |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1289 assert self._start_rev <= self._end_rev, ( |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1290 self._start_rev, |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1291 self._end_rev, |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1292 ) |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1293 |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1294 def _update(self, revlog, start_rev, end_rev): |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1295 """internal method that actually do update content""" |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1296 assert self._start_rev is None or ( |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1297 start_rev < self._start_rev or start_rev > self._end_rev |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1298 ), (self._start_rev, self._end_rev, start_rev, end_rev) |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1299 assert self._start_rev is None or ( |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1300 end_rev < self._start_rev or end_rev > self._end_rev |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1301 ), (self._start_rev, self._end_rev, start_rev, end_rev) |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1302 cache = self.snapshots |
50928
d718eddf01d9
safehasattr: drop usage in favor of hasattr
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50698
diff
changeset
|
1303 if hasattr(revlog.index, 'findsnapshots'): |
49678
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1304 revlog.index.findsnapshots(cache, start_rev, end_rev) |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1305 else: |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1306 deltaparent = revlog.deltaparent |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1307 issnapshot = revlog.issnapshot |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1308 for rev in revlog.revs(start_rev, end_rev): |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1309 if issnapshot(rev): |
49679
b670eb3dd6c9
delta-find: use sets instead of list in the snapshot cache
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49678
diff
changeset
|
1310 cache[deltaparent(rev)].add(rev) |
49678
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1311 |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1312 |
48946
642e31cb55f0
py3: use class X: instead of class X(object):
Gregory Szorc <gregory.szorc@gmail.com>
parents:
48875
diff
changeset
|
1313 class deltacomputer: |
51319
da7ecb4deaec
delta-find: add a small docstring to deltacomputer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51091
diff
changeset
|
1314 """object capable of computing delta and finding delta for multiple revision |
da7ecb4deaec
delta-find: add a small docstring to deltacomputer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51091
diff
changeset
|
1315 |
da7ecb4deaec
delta-find: add a small docstring to deltacomputer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51091
diff
changeset
|
1316 This object is meant to compute and find multiple delta applied to the same |
da7ecb4deaec
delta-find: add a small docstring to deltacomputer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51091
diff
changeset
|
1317 revlog. |
da7ecb4deaec
delta-find: add a small docstring to deltacomputer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51091
diff
changeset
|
1318 """ |
da7ecb4deaec
delta-find: add a small docstring to deltacomputer
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51091
diff
changeset
|
1319 |
49610
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
1320 def __init__( |
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
1321 self, |
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
1322 revlog, |
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
1323 write_debug=None, |
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
1324 debug_search=False, |
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
1325 debug_info=None, |
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
1326 ): |
35738
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1327 self.revlog = revlog |
49226
e6b7c6fbeb48
deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
1328 self._write_debug = write_debug |
50649
c84cc0ac77e4
delta-fine: use the `_debug_search` attribute directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50648
diff
changeset
|
1329 if write_debug is None: |
c84cc0ac77e4
delta-fine: use the `_debug_search` attribute directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50648
diff
changeset
|
1330 self._debug_search = False |
c84cc0ac77e4
delta-fine: use the `_debug_search` attribute directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50648
diff
changeset
|
1331 else: |
c84cc0ac77e4
delta-fine: use the `_debug_search` attribute directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50648
diff
changeset
|
1332 self._debug_search = debug_search |
49610
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
1333 self._debug_info = debug_info |
49681
c261a628e525
delta-find: use a single snapshot cache when applying a group to an object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49680
diff
changeset
|
1334 self._snapshot_cache = SnapshotCache() |
35738
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1335 |
50648
5d210ff4b657
delta-find: move the `gather_debug` logic in a property
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50357
diff
changeset
|
1336 @property |
5d210ff4b657
delta-find: move the `gather_debug` logic in a property
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50357
diff
changeset
|
1337 def _gather_debug(self): |
5d210ff4b657
delta-find: move the `gather_debug` logic in a property
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50357
diff
changeset
|
1338 return self._write_debug is not None or self._debug_info is not None |
5d210ff4b657
delta-find: move the `gather_debug` logic in a property
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50357
diff
changeset
|
1339 |
51020
509f0f7fc89e
delta-computer: stop explicitly taking file handle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50929
diff
changeset
|
1340 def buildtext(self, revinfo): |
35738
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1341 """Builds a fulltext version of a revision |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1342 |
47399
34cc102c73f5
revlog: move `revisioninfo` in `revlogutils`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47253
diff
changeset
|
1343 revinfo: revisioninfo instance that contains all needed info |
35738
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1344 """ |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1345 btext = revinfo.btext |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1346 if btext[0] is not None: |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1347 return btext[0] |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1348 |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1349 revlog = self.revlog |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1350 cachedelta = revinfo.cachedelta |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1351 baserev = cachedelta[0] |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1352 delta = cachedelta[1] |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1353 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1354 fulltext = btext[0] = _textfromdelta( |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1355 revlog, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1356 baserev, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1357 delta, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1358 revinfo.p1, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1359 revinfo.p2, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1360 revinfo.flags, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1361 revinfo.node, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1362 ) |
39331
fd0150a3c2fe
revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents:
39330
diff
changeset
|
1363 return fulltext |
35738
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1364 |
51020
509f0f7fc89e
delta-computer: stop explicitly taking file handle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50929
diff
changeset
|
1365 def _builddeltadiff(self, base, revinfo): |
35738
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1366 revlog = self.revlog |
51020
509f0f7fc89e
delta-computer: stop explicitly taking file handle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50929
diff
changeset
|
1367 t = self.buildtext(revinfo) |
35738
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1368 if revlog.iscensored(base): |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1369 # deltas based on a censored revision must replace the |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1370 # full content in one patch, so delta works everywhere |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1371 header = mdiff.replacediffheader(revlog.rawsize(base), len(t)) |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1372 delta = header + t |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1373 else: |
51020
509f0f7fc89e
delta-computer: stop explicitly taking file handle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50929
diff
changeset
|
1374 ptext = revlog.rawdata(base) |
35738
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1375 delta = mdiff.textdiff(ptext, t) |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1376 |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1377 return delta |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1378 |
51350
670e68729aa7
delta-find: stop using heuristic to determine if we are creating a snapshot
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51349
diff
changeset
|
1379 def _builddeltainfo( |
670e68729aa7
delta-find: stop using heuristic to determine if we are creating a snapshot
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51349
diff
changeset
|
1380 self, revinfo, base, target_rev=None, as_snapshot=False |
670e68729aa7
delta-find: stop using heuristic to determine if we are creating a snapshot
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51349
diff
changeset
|
1381 ): |
35738
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1382 # can we use the cached delta? |
42465
6e9ba867a946
delta: move some delta chain related computation earlier in deltainfo
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42464
diff
changeset
|
1383 revlog = self.revlog |
6e9ba867a946
delta: move some delta chain related computation earlier in deltainfo
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42464
diff
changeset
|
1384 chainbase = revlog.chainbase(base) |
51046
f636103c4d67
revlog: remove legacy usage of `_generaldelta`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51045
diff
changeset
|
1385 if revlog.delta_config.general_delta: |
42465
6e9ba867a946
delta: move some delta chain related computation earlier in deltainfo
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42464
diff
changeset
|
1386 deltabase = base |
6e9ba867a946
delta: move some delta chain related computation earlier in deltainfo
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42464
diff
changeset
|
1387 else: |
50356
2a6949ab9d23
delta-find: add a simple safeguard to prevent bad non-general-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50355
diff
changeset
|
1388 if target_rev is not None and base != target_rev - 1: |
2a6949ab9d23
delta-find: add a simple safeguard to prevent bad non-general-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50355
diff
changeset
|
1389 msg = ( |
2a6949ab9d23
delta-find: add a simple safeguard to prevent bad non-general-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50355
diff
changeset
|
1390 b'general delta cannot use delta for something else ' |
2a6949ab9d23
delta-find: add a simple safeguard to prevent bad non-general-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50355
diff
changeset
|
1391 b'than `prev`: %d<-%d' |
2a6949ab9d23
delta-find: add a simple safeguard to prevent bad non-general-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50355
diff
changeset
|
1392 ) |
2a6949ab9d23
delta-find: add a simple safeguard to prevent bad non-general-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50355
diff
changeset
|
1393 msg %= (base, target_rev) |
2a6949ab9d23
delta-find: add a simple safeguard to prevent bad non-general-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50355
diff
changeset
|
1394 raise error.ProgrammingError(msg) |
42465
6e9ba867a946
delta: move some delta chain related computation earlier in deltainfo
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42464
diff
changeset
|
1395 deltabase = chainbase |
6e9ba867a946
delta: move some delta chain related computation earlier in deltainfo
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42464
diff
changeset
|
1396 snapshotdepth = None |
51060
f71f07a679b4
revlog: remove legacy usage of `_sparserevlog`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51058
diff
changeset
|
1397 if revlog.delta_config.sparse_revlog and deltabase == nullrev: |
42465
6e9ba867a946
delta: move some delta chain related computation earlier in deltainfo
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42464
diff
changeset
|
1398 snapshotdepth = 0 |
51350
670e68729aa7
delta-find: stop using heuristic to determine if we are creating a snapshot
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51349
diff
changeset
|
1399 elif revlog.delta_config.sparse_revlog and as_snapshot: |
670e68729aa7
delta-find: stop using heuristic to determine if we are creating a snapshot
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51349
diff
changeset
|
1400 assert revlog.issnapshot(deltabase) |
42465
6e9ba867a946
delta: move some delta chain related computation earlier in deltainfo
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42464
diff
changeset
|
1401 # A delta chain should always be one full snapshot, |
6e9ba867a946
delta: move some delta chain related computation earlier in deltainfo
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42464
diff
changeset
|
1402 # zero or more semi-snapshots, and zero or more deltas |
6e9ba867a946
delta: move some delta chain related computation earlier in deltainfo
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42464
diff
changeset
|
1403 p1, p2 = revlog.rev(revinfo.p1), revlog.rev(revinfo.p2) |
6e9ba867a946
delta: move some delta chain related computation earlier in deltainfo
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42464
diff
changeset
|
1404 if deltabase not in (p1, p2) and revlog.issnapshot(deltabase): |
6e9ba867a946
delta: move some delta chain related computation earlier in deltainfo
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42464
diff
changeset
|
1405 snapshotdepth = len(revlog._deltachain(deltabase)[0]) |
39595
a911932d5003
revlog: reuse cached delta for identical base revision (issue5975)
Boris Feld <boris.feld@octobus.net>
parents:
39594
diff
changeset
|
1406 delta = None |
a911932d5003
revlog: reuse cached delta for identical base revision (issue5975)
Boris Feld <boris.feld@octobus.net>
parents:
39594
diff
changeset
|
1407 if revinfo.cachedelta: |
49675
0fca63953810
find-delta: minor preparatory change
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49674
diff
changeset
|
1408 cachebase = revinfo.cachedelta[0] |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1409 # check if the diff still apply |
39595
a911932d5003
revlog: reuse cached delta for identical base revision (issue5975)
Boris Feld <boris.feld@octobus.net>
parents:
39594
diff
changeset
|
1410 currentbase = cachebase |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1411 while ( |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1412 currentbase != nullrev |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1413 and currentbase != base |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1414 and self.revlog.length(currentbase) == 0 |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1415 ): |
39595
a911932d5003
revlog: reuse cached delta for identical base revision (issue5975)
Boris Feld <boris.feld@octobus.net>
parents:
39594
diff
changeset
|
1416 currentbase = self.revlog.deltaparent(currentbase) |
51067
26dcdbe15024
revlog: remove legacy usage of `_lazydeltabase`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51063
diff
changeset
|
1417 if self.revlog.delta_config.lazy_delta and currentbase == base: |
39595
a911932d5003
revlog: reuse cached delta for identical base revision (issue5975)
Boris Feld <boris.feld@octobus.net>
parents:
39594
diff
changeset
|
1418 delta = revinfo.cachedelta[1] |
a911932d5003
revlog: reuse cached delta for identical base revision (issue5975)
Boris Feld <boris.feld@octobus.net>
parents:
39594
diff
changeset
|
1419 if delta is None: |
51020
509f0f7fc89e
delta-computer: stop explicitly taking file handle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50929
diff
changeset
|
1420 delta = self._builddeltadiff(base, revinfo) |
50649
c84cc0ac77e4
delta-fine: use the `_debug_search` attribute directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50648
diff
changeset
|
1421 if self._debug_search: |
49228
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1422 msg = b"DBG-DELTAS-SEARCH: uncompressed-delta-size=%d\n" |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1423 msg %= len(delta) |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1424 self._write_debug(msg) |
42467
c1c1872d25d1
deltas: skip if projected compressed size does not match text size constraint
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42466
diff
changeset
|
1425 # snapshotdept need to be neither None nor 0 level snapshot |
51083
5645524c6b62
revlog: also migrates `revlog.upperboundcomp` to ConfigClass
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51067
diff
changeset
|
1426 if revlog.delta_config.upper_bound_comp is not None and snapshotdepth: |
5645524c6b62
revlog: also migrates `revlog.upperboundcomp` to ConfigClass
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51067
diff
changeset
|
1427 lowestrealisticdeltalen = ( |
5645524c6b62
revlog: also migrates `revlog.upperboundcomp` to ConfigClass
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51067
diff
changeset
|
1428 len(delta) // revlog.delta_config.upper_bound_comp |
5645524c6b62
revlog: also migrates `revlog.upperboundcomp` to ConfigClass
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51067
diff
changeset
|
1429 ) |
42467
c1c1872d25d1
deltas: skip if projected compressed size does not match text size constraint
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42466
diff
changeset
|
1430 snapshotlimit = revinfo.textlen >> snapshotdepth |
50649
c84cc0ac77e4
delta-fine: use the `_debug_search` attribute directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50648
diff
changeset
|
1431 if self._debug_search: |
49228
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1432 msg = b"DBG-DELTAS-SEARCH: projected-lower-size=%d\n" |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1433 msg %= lowestrealisticdeltalen |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1434 self._write_debug(msg) |
42467
c1c1872d25d1
deltas: skip if projected compressed size does not match text size constraint
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42466
diff
changeset
|
1435 if snapshotlimit < lowestrealisticdeltalen: |
50649
c84cc0ac77e4
delta-fine: use the `_debug_search` attribute directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50648
diff
changeset
|
1436 if self._debug_search: |
49228
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1437 msg = b"DBG-DELTAS-SEARCH: DISCARDED (snapshot limit)\n" |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1438 self._write_debug(msg) |
42467
c1c1872d25d1
deltas: skip if projected compressed size does not match text size constraint
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42466
diff
changeset
|
1439 return None |
42468
9b5fbe5ead89
deltas: skip if projected compressed size is bigger than previous snapshot
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42467
diff
changeset
|
1440 if revlog.length(base) < lowestrealisticdeltalen: |
50649
c84cc0ac77e4
delta-fine: use the `_debug_search` attribute directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50648
diff
changeset
|
1441 if self._debug_search: |
49228
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1442 msg = b"DBG-DELTAS-SEARCH: DISCARDED (prev size)\n" |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1443 self._write_debug(msg) |
42468
9b5fbe5ead89
deltas: skip if projected compressed size is bigger than previous snapshot
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42467
diff
changeset
|
1444 return None |
51091
a82704902db8
revlog: move the compression/decompression logic on the inner object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51083
diff
changeset
|
1445 header, data = revlog._inner.compress(delta) |
35738
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1446 deltalen = len(header) + len(data) |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1447 offset = revlog.end(len(revlog) - 1) |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1448 dist = deltalen + offset - revlog.start(chainbase) |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1449 chainlen, compresseddeltalen = revlog._chaininfo(base) |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1450 chainlen += 1 |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1451 compresseddeltalen += deltalen |
39154
e0da43e2f71f
revlog: compute snapshot depth on delta info
Boris Feld <boris.feld@octobus.net>
parents:
39152
diff
changeset
|
1452 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1453 return _deltainfo( |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1454 dist, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1455 deltalen, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1456 (header, data), |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1457 deltabase, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1458 chainbase, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1459 chainlen, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1460 compresseddeltalen, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1461 snapshotdepth, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1462 ) |
35738
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1463 |
51020
509f0f7fc89e
delta-computer: stop explicitly taking file handle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50929
diff
changeset
|
1464 def _fullsnapshotinfo(self, revinfo, curr): |
509f0f7fc89e
delta-computer: stop explicitly taking file handle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50929
diff
changeset
|
1465 rawtext = self.buildtext(revinfo) |
51091
a82704902db8
revlog: move the compression/decompression logic on the inner object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51083
diff
changeset
|
1466 data = self.revlog._inner.compress(rawtext) |
39333
5d343a24bff5
revlogdeltas: always return a delta info object in finddeltainfo
Boris Feld <boris.feld@octobus.net>
parents:
39332
diff
changeset
|
1467 compresseddeltalen = deltalen = dist = len(data[1]) + len(data[0]) |
5d343a24bff5
revlogdeltas: always return a delta info object in finddeltainfo
Boris Feld <boris.feld@octobus.net>
parents:
39332
diff
changeset
|
1468 deltabase = chainbase = curr |
5d343a24bff5
revlogdeltas: always return a delta info object in finddeltainfo
Boris Feld <boris.feld@octobus.net>
parents:
39332
diff
changeset
|
1469 snapshotdepth = 0 |
5d343a24bff5
revlogdeltas: always return a delta info object in finddeltainfo
Boris Feld <boris.feld@octobus.net>
parents:
39332
diff
changeset
|
1470 chainlen = 1 |
5d343a24bff5
revlogdeltas: always return a delta info object in finddeltainfo
Boris Feld <boris.feld@octobus.net>
parents:
39332
diff
changeset
|
1471 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1472 return _deltainfo( |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1473 dist, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1474 deltalen, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1475 data, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1476 deltabase, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1477 chainbase, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1478 chainlen, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1479 compresseddeltalen, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1480 snapshotdepth, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1481 ) |
39333
5d343a24bff5
revlogdeltas: always return a delta info object in finddeltainfo
Boris Feld <boris.feld@octobus.net>
parents:
39332
diff
changeset
|
1482 |
51020
509f0f7fc89e
delta-computer: stop explicitly taking file handle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50929
diff
changeset
|
1483 def finddeltainfo(self, revinfo, excluded_bases=None, target_rev=None): |
35738
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1484 """Find an acceptable delta against a candidate revision |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1485 |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1486 revinfo: information about the revision (instance of _revisioninfo) |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1487 |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1488 Returns the first acceptable candidate revision, as ordered by |
39334
507f5b1dd7c8
revlogdeltas: extract _getcandidaterevs in a function
Boris Feld <boris.feld@octobus.net>
parents:
39333
diff
changeset
|
1489 _candidategroups |
39333
5d343a24bff5
revlogdeltas: always return a delta info object in finddeltainfo
Boris Feld <boris.feld@octobus.net>
parents:
39332
diff
changeset
|
1490 |
5d343a24bff5
revlogdeltas: always return a delta info object in finddeltainfo
Boris Feld <boris.feld@octobus.net>
parents:
39332
diff
changeset
|
1491 If no suitable deltabase is found, we return delta info for a full |
5d343a24bff5
revlogdeltas: always return a delta info object in finddeltainfo
Boris Feld <boris.feld@octobus.net>
parents:
39332
diff
changeset
|
1492 snapshot. |
47401
1efe3cdef53a
revlog: add a ways to blacklist some revision when searching for a delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47399
diff
changeset
|
1493 |
1efe3cdef53a
revlog: add a ways to blacklist some revision when searching for a delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47399
diff
changeset
|
1494 `excluded_bases` is an optional set of revision that cannot be used as |
1efe3cdef53a
revlog: add a ways to blacklist some revision when searching for a delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47399
diff
changeset
|
1495 a delta base. Use this to recompute delta suitable in censor or strip |
1efe3cdef53a
revlog: add a ways to blacklist some revision when searching for a delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47399
diff
changeset
|
1496 context. |
35738
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1497 """ |
47456
93f4e183b3f5
deltas: at a `target_rev` parameter to finddeltainfo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47452
diff
changeset
|
1498 if target_rev is None: |
47457
f8330a3fc39f
censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47456
diff
changeset
|
1499 target_rev = len(self.revlog) |
47456
93f4e183b3f5
deltas: at a `target_rev` parameter to finddeltainfo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47452
diff
changeset
|
1500 |
50653
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1501 gather_debug = self._gather_debug |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1502 cachedelta = revinfo.cachedelta |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1503 revlog = self.revlog |
50657
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1504 p1r = p2r = None |
50653
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1505 |
50657
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1506 if excluded_bases is None: |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1507 excluded_bases = set() |
50653
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1508 |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1509 if gather_debug: |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1510 start = util.timer() |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1511 dbg = self._one_dbg_data() |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1512 dbg['revision'] = target_rev |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1513 p1r = revlog.rev(revinfo.p1) |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1514 p2r = revlog.rev(revinfo.p2) |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1515 if p1r != nullrev: |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1516 p1_chain_len = revlog._chaininfo(p1r)[0] |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1517 else: |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1518 p1_chain_len = -1 |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1519 if p2r != nullrev: |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1520 p2_chain_len = revlog._chaininfo(p2r)[0] |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1521 else: |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1522 p2_chain_len = -1 |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1523 dbg['p1-chain-len'] = p1_chain_len |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1524 dbg['p2-chain-len'] = p2_chain_len |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1525 |
50654
bfb6404089a5
delta-find: gather the condition to blindly use a full snapshot together
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50653
diff
changeset
|
1526 # 1) if the revision is empty, no amount of delta can beat it |
bfb6404089a5
delta-find: gather the condition to blindly use a full snapshot together
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50653
diff
changeset
|
1527 # |
bfb6404089a5
delta-find: gather the condition to blindly use a full snapshot together
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50653
diff
changeset
|
1528 # 2) no delta for flag processor revision (see "candelta" for why) |
bfb6404089a5
delta-find: gather the condition to blindly use a full snapshot together
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50653
diff
changeset
|
1529 # not calling candelta since only one revision needs test, also to |
bfb6404089a5
delta-find: gather the condition to blindly use a full snapshot together
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50653
diff
changeset
|
1530 # avoid overhead fetching flags again. |
bfb6404089a5
delta-find: gather the condition to blindly use a full snapshot together
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50653
diff
changeset
|
1531 if not revinfo.textlen or revinfo.flags & REVIDX_RAWTEXT_CHANGING_FLAGS: |
51020
509f0f7fc89e
delta-computer: stop explicitly taking file handle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50929
diff
changeset
|
1532 deltainfo = self._fullsnapshotinfo(revinfo, target_rev) |
50655
485c9410b75a
deltafind: issue debug information when we fast-path rivial case too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50654
diff
changeset
|
1533 if gather_debug: |
485c9410b75a
deltafind: issue debug information when we fast-path rivial case too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50654
diff
changeset
|
1534 end = util.timer() |
485c9410b75a
deltafind: issue debug information when we fast-path rivial case too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50654
diff
changeset
|
1535 dbg['duration'] = end - start |
485c9410b75a
deltafind: issue debug information when we fast-path rivial case too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50654
diff
changeset
|
1536 dbg[ |
485c9410b75a
deltafind: issue debug information when we fast-path rivial case too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50654
diff
changeset
|
1537 'delta-base' |
485c9410b75a
deltafind: issue debug information when we fast-path rivial case too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50654
diff
changeset
|
1538 ] = deltainfo.base # pytype: disable=attribute-error |
485c9410b75a
deltafind: issue debug information when we fast-path rivial case too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50654
diff
changeset
|
1539 dbg['search_round_count'] = 0 |
50657
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1540 dbg['using-cached-base'] = False |
50655
485c9410b75a
deltafind: issue debug information when we fast-path rivial case too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50654
diff
changeset
|
1541 dbg['delta_try_count'] = 0 |
485c9410b75a
deltafind: issue debug information when we fast-path rivial case too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50654
diff
changeset
|
1542 dbg['type'] = b"full" |
485c9410b75a
deltafind: issue debug information when we fast-path rivial case too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50654
diff
changeset
|
1543 dbg['snapshot-depth'] = 0 |
485c9410b75a
deltafind: issue debug information when we fast-path rivial case too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50654
diff
changeset
|
1544 self._dbg_process_data(dbg) |
485c9410b75a
deltafind: issue debug information when we fast-path rivial case too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50654
diff
changeset
|
1545 return deltainfo |
39085
dbb3e9e44fce
revlog: do not search for delta for empty content
Boris Feld <boris.feld@octobus.net>
parents:
39084
diff
changeset
|
1546 |
50657
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1547 deltainfo = None |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1548 |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1549 # If this source delta are to be forcibly reuse, let us comply early. |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1550 if ( |
51046
f636103c4d67
revlog: remove legacy usage of `_generaldelta`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51045
diff
changeset
|
1551 revlog.delta_config.general_delta |
50657
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1552 and revinfo.cachedelta is not None |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1553 and revinfo.cachedelta[2] == DELTA_BASE_REUSE_FORCE |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1554 ): |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1555 base = revinfo.cachedelta[0] |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1556 if base == nullrev: |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1557 dbg_type = b"full" |
51020
509f0f7fc89e
delta-computer: stop explicitly taking file handle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50929
diff
changeset
|
1558 deltainfo = self._fullsnapshotinfo(revinfo, target_rev) |
50657
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1559 if gather_debug: |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1560 snapshotdepth = 0 |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1561 elif base not in excluded_bases: |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1562 delta = revinfo.cachedelta[1] |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1563 header, data = revlog.compress(delta) |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1564 deltalen = len(header) + len(data) |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1565 if gather_debug: |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1566 offset = revlog.end(len(revlog) - 1) |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1567 chainbase = revlog.chainbase(base) |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1568 distance = deltalen + offset - revlog.start(chainbase) |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1569 chainlen, compresseddeltalen = revlog._chaininfo(base) |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1570 chainlen += 1 |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1571 compresseddeltalen += deltalen |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1572 if base == p1r or base == p2r: |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1573 dbg_type = b"delta" |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1574 snapshotdepth = None |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1575 elif not revlog.issnapshot(base): |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1576 snapshotdepth = None |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1577 else: |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1578 dbg_type = b"snapshot" |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1579 snapshotdepth = revlog.snapshotdepth(base) + 1 |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1580 else: |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1581 distance = None |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1582 chainbase = None |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1583 chainlen = None |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1584 compresseddeltalen = None |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1585 snapshotdepth = None |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1586 deltainfo = _deltainfo( |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1587 distance=distance, |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1588 deltalen=deltalen, |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1589 data=(header, data), |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1590 base=base, |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1591 chainbase=chainbase, |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1592 chainlen=chainlen, |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1593 compresseddeltalen=compresseddeltalen, |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1594 snapshotdepth=snapshotdepth, |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1595 ) |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1596 |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1597 if deltainfo is not None: |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1598 if gather_debug: |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1599 end = util.timer() |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1600 dbg['duration'] = end - start |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1601 dbg[ |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1602 'delta-base' |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1603 ] = deltainfo.base # pytype: disable=attribute-error |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1604 dbg['search_round_count'] = 0 |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1605 dbg['using-cached-base'] = True |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1606 dbg['delta_try_count'] = 0 |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1607 dbg['type'] = b"full" |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1608 if snapshotdepth is None: |
51333
0d92d62ecde0
delta-find: use "-1" as depth snapshot-dept for non snapshot in debug
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51332
diff
changeset
|
1609 dbg['snapshot-depth'] = -1 |
50657
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1610 else: |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1611 dbg['snapshot-depth'] = snapshotdepth |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1612 self._dbg_process_data(dbg) |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1613 return deltainfo |
47401
1efe3cdef53a
revlog: add a ways to blacklist some revision when searching for a delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47399
diff
changeset
|
1614 |
49226
e6b7c6fbeb48
deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
1615 # count the number of different delta we tried (for debug purpose) |
e6b7c6fbeb48
deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
1616 dbg_try_count = 0 |
e6b7c6fbeb48
deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
1617 # count the number of "search round" we did. (for debug purpose) |
e6b7c6fbeb48
deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
1618 dbg_try_rounds = 0 |
e6b7c6fbeb48
deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
1619 dbg_type = b'unknown' |
e6b7c6fbeb48
deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
1620 |
50653
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1621 if p1r is None: |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1622 p1r = revlog.rev(revinfo.p1) |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1623 p2r = revlog.rev(revinfo.p2) |
49226
e6b7c6fbeb48
deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
1624 |
50649
c84cc0ac77e4
delta-fine: use the `_debug_search` attribute directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50648
diff
changeset
|
1625 if self._debug_search: |
49228
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1626 msg = b"DBG-DELTAS-SEARCH: SEARCH rev=%d\n" |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1627 msg %= target_rev |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1628 self._write_debug(msg) |
49226
e6b7c6fbeb48
deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
1629 |
51347
555826073625
delta-find: introduce and use specialized _DeltaSearch class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51346
diff
changeset
|
1630 # should we try to build a delta? |
555826073625
delta-find: introduce and use specialized _DeltaSearch class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51346
diff
changeset
|
1631 if not (len(self.revlog) and self.revlog._storedeltachains): |
555826073625
delta-find: introduce and use specialized _DeltaSearch class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51346
diff
changeset
|
1632 search_cls = _NoDeltaSearch |
51357
d7e2acdd50ba
delta-find: split the _DeltaSearch class in two
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51356
diff
changeset
|
1633 elif self.revlog.delta_config.sparse_revlog: |
d7e2acdd50ba
delta-find: split the _DeltaSearch class in two
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51356
diff
changeset
|
1634 search_cls = _SparseDeltaSearch |
d7e2acdd50ba
delta-find: split the _DeltaSearch class in two
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51356
diff
changeset
|
1635 elif self.revlog.delta_config.general_delta: |
d7e2acdd50ba
delta-find: split the _DeltaSearch class in two
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51356
diff
changeset
|
1636 search_cls = _GeneralDeltaSearch |
d7e2acdd50ba
delta-find: split the _DeltaSearch class in two
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51356
diff
changeset
|
1637 else: |
51347
555826073625
delta-find: introduce and use specialized _DeltaSearch class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51346
diff
changeset
|
1638 # before general delta, there is only one possible delta base |
555826073625
delta-find: introduce and use specialized _DeltaSearch class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51346
diff
changeset
|
1639 search_cls = _PrevDeltaSearch |
555826073625
delta-find: introduce and use specialized _DeltaSearch class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51346
diff
changeset
|
1640 |
555826073625
delta-find: introduce and use specialized _DeltaSearch class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51346
diff
changeset
|
1641 search = search_cls( |
49611
2afee2176775
delta-find: expand a function definition and call before extendin it
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49610
diff
changeset
|
1642 self.revlog, |
51330
49401b7dec0c
delta-find: feed revinfo to _DeltaSearch
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51328
diff
changeset
|
1643 revinfo, |
49611
2afee2176775
delta-find: expand a function definition and call before extendin it
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49610
diff
changeset
|
1644 p1r, |
2afee2176775
delta-find: expand a function definition and call before extendin it
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49610
diff
changeset
|
1645 p2r, |
2afee2176775
delta-find: expand a function definition and call before extendin it
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49610
diff
changeset
|
1646 cachedelta, |
49612
e706bb41fdb3
delta-find: move pre-filtering with other pre-filtering logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49611
diff
changeset
|
1647 excluded_bases, |
e706bb41fdb3
delta-find: move pre-filtering with other pre-filtering logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49611
diff
changeset
|
1648 target_rev, |
49681
c261a628e525
delta-find: use a single snapshot cache when applying a group to an object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49680
diff
changeset
|
1649 snapshot_cache=self._snapshot_cache, |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1650 ) |
51320
c82e03b102a6
delta-find: introduce a _DeltaSearch object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51319
diff
changeset
|
1651 |
51334
d0d869fccd20
delta-find: move away from the generator API for _DeltaSearch
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51333
diff
changeset
|
1652 while not search.done: |
d0d869fccd20
delta-find: move away from the generator API for _DeltaSearch
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51333
diff
changeset
|
1653 current_group = search.current_group |
d0d869fccd20
delta-find: move away from the generator API for _DeltaSearch
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51333
diff
changeset
|
1654 # current_group can be `None`, but not is search.done is False |
d0d869fccd20
delta-find: move away from the generator API for _DeltaSearch
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51333
diff
changeset
|
1655 # We add this assert to help pytype |
d0d869fccd20
delta-find: move away from the generator API for _DeltaSearch
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51333
diff
changeset
|
1656 assert current_group is not None |
d0d869fccd20
delta-find: move away from the generator API for _DeltaSearch
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51333
diff
changeset
|
1657 candidaterevs = current_group |
49226
e6b7c6fbeb48
deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
1658 dbg_try_rounds += 1 |
50649
c84cc0ac77e4
delta-fine: use the `_debug_search` attribute directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50648
diff
changeset
|
1659 if self._debug_search: |
49228
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1660 prev = None |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1661 if deltainfo is not None: |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1662 prev = deltainfo.base |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1663 |
49608
78ba41878f2e
delta-find: add debug information about reuse of cached data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49228
diff
changeset
|
1664 if ( |
78ba41878f2e
delta-find: add debug information about reuse of cached data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49228
diff
changeset
|
1665 cachedelta is not None |
78ba41878f2e
delta-find: add debug information about reuse of cached data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49228
diff
changeset
|
1666 and len(candidaterevs) == 1 |
78ba41878f2e
delta-find: add debug information about reuse of cached data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49228
diff
changeset
|
1667 and cachedelta[0] in candidaterevs |
78ba41878f2e
delta-find: add debug information about reuse of cached data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49228
diff
changeset
|
1668 ): |
78ba41878f2e
delta-find: add debug information about reuse of cached data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49228
diff
changeset
|
1669 round_type = b"cached-delta" |
50652
ebb292ffdf4d
delta-find: fix `parents` round detection
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50651
diff
changeset
|
1670 elif p1r in candidaterevs or p2r in candidaterevs: |
49228
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1671 round_type = b"parents" |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1672 elif prev is not None and all(c < prev for c in candidaterevs): |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1673 round_type = b"refine-down" |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1674 elif prev is not None and all(c > prev for c in candidaterevs): |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1675 round_type = b"refine-up" |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1676 else: |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1677 round_type = b"search-down" |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1678 msg = b"DBG-DELTAS-SEARCH: ROUND #%d - %d candidates - %s\n" |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1679 msg %= (dbg_try_rounds, len(candidaterevs), round_type) |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1680 self._write_debug(msg) |
35738
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1681 nominateddeltas = [] |
39498
04b75f3a3f2a
snapshot: add refining logic at the findeltainfo level
Boris Feld <boris.feld@octobus.net>
parents:
39497
diff
changeset
|
1682 if deltainfo is not None: |
50649
c84cc0ac77e4
delta-fine: use the `_debug_search` attribute directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50648
diff
changeset
|
1683 if self._debug_search: |
49228
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1684 msg = ( |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1685 b"DBG-DELTAS-SEARCH: CONTENDER: rev=%d - length=%d\n" |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1686 ) |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1687 msg %= (deltainfo.base, deltainfo.deltalen) |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1688 self._write_debug(msg) |
39498
04b75f3a3f2a
snapshot: add refining logic at the findeltainfo level
Boris Feld <boris.feld@octobus.net>
parents:
39497
diff
changeset
|
1689 # if we already found a good delta, |
04b75f3a3f2a
snapshot: add refining logic at the findeltainfo level
Boris Feld <boris.feld@octobus.net>
parents:
39497
diff
changeset
|
1690 # challenge it against refined candidates |
04b75f3a3f2a
snapshot: add refining logic at the findeltainfo level
Boris Feld <boris.feld@octobus.net>
parents:
39497
diff
changeset
|
1691 nominateddeltas.append(deltainfo) |
35738
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1692 for candidaterev in candidaterevs: |
50649
c84cc0ac77e4
delta-fine: use the `_debug_search` attribute directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50648
diff
changeset
|
1693 if self._debug_search: |
49228
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1694 msg = b"DBG-DELTAS-SEARCH: CANDIDATE: rev=%d\n" |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1695 msg %= candidaterev |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1696 self._write_debug(msg) |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1697 candidate_type = None |
50652
ebb292ffdf4d
delta-find: fix `parents` round detection
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50651
diff
changeset
|
1698 if candidaterev == p1r: |
49228
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1699 candidate_type = b"p1" |
50652
ebb292ffdf4d
delta-find: fix `parents` round detection
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50651
diff
changeset
|
1700 elif candidaterev == p2r: |
49228
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1701 candidate_type = b"p2" |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1702 elif self.revlog.issnapshot(candidaterev): |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1703 candidate_type = b"snapshot-%d" |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1704 candidate_type %= self.revlog.snapshotdepth( |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1705 candidaterev |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1706 ) |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1707 |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1708 if candidate_type is not None: |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1709 msg = b"DBG-DELTAS-SEARCH: type=%s\n" |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1710 msg %= candidate_type |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1711 self._write_debug(msg) |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1712 msg = b"DBG-DELTAS-SEARCH: size=%d\n" |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1713 msg %= self.revlog.length(candidaterev) |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1714 self._write_debug(msg) |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1715 msg = b"DBG-DELTAS-SEARCH: base=%d\n" |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1716 msg %= self.revlog.deltaparent(candidaterev) |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1717 self._write_debug(msg) |
49612
e706bb41fdb3
delta-find: move pre-filtering with other pre-filtering logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49611
diff
changeset
|
1718 |
49226
e6b7c6fbeb48
deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
1719 dbg_try_count += 1 |
49228
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1720 |
50649
c84cc0ac77e4
delta-fine: use the `_debug_search` attribute directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50648
diff
changeset
|
1721 if self._debug_search: |
49228
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1722 delta_start = util.timer() |
50356
2a6949ab9d23
delta-find: add a simple safeguard to prevent bad non-general-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50355
diff
changeset
|
1723 candidatedelta = self._builddeltainfo( |
2a6949ab9d23
delta-find: add a simple safeguard to prevent bad non-general-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50355
diff
changeset
|
1724 revinfo, |
2a6949ab9d23
delta-find: add a simple safeguard to prevent bad non-general-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50355
diff
changeset
|
1725 candidaterev, |
2a6949ab9d23
delta-find: add a simple safeguard to prevent bad non-general-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50355
diff
changeset
|
1726 target_rev=target_rev, |
51350
670e68729aa7
delta-find: stop using heuristic to determine if we are creating a snapshot
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51349
diff
changeset
|
1727 as_snapshot=search.current_stage == _STAGE_SNAPSHOT, |
50356
2a6949ab9d23
delta-find: add a simple safeguard to prevent bad non-general-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50355
diff
changeset
|
1728 ) |
50649
c84cc0ac77e4
delta-fine: use the `_debug_search` attribute directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50648
diff
changeset
|
1729 if self._debug_search: |
49228
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1730 delta_end = util.timer() |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1731 msg = b"DBG-DELTAS-SEARCH: delta-search-time=%f\n" |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1732 msg %= delta_end - delta_start |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1733 self._write_debug(msg) |
42466
465f2d0df9ae
deltas: accept and skip None return for delta info
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42465
diff
changeset
|
1734 if candidatedelta is not None: |
51331
7455cae67260
delta-find: move is_good_delta_info on the _DeltaSearch class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51330
diff
changeset
|
1735 if search.is_good_delta_info(candidatedelta): |
50649
c84cc0ac77e4
delta-fine: use the `_debug_search` attribute directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50648
diff
changeset
|
1736 if self._debug_search: |
49228
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1737 msg = b"DBG-DELTAS-SEARCH: DELTA: length=%d (GOOD)\n" |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1738 msg %= candidatedelta.deltalen |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1739 self._write_debug(msg) |
42466
465f2d0df9ae
deltas: accept and skip None return for delta info
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42465
diff
changeset
|
1740 nominateddeltas.append(candidatedelta) |
50649
c84cc0ac77e4
delta-fine: use the `_debug_search` attribute directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50648
diff
changeset
|
1741 elif self._debug_search: |
49228
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1742 msg = b"DBG-DELTAS-SEARCH: DELTA: length=%d (BAD)\n" |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1743 msg %= candidatedelta.deltalen |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1744 self._write_debug(msg) |
50649
c84cc0ac77e4
delta-fine: use the `_debug_search` attribute directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50648
diff
changeset
|
1745 elif self._debug_search: |
49228
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1746 msg = b"DBG-DELTAS-SEARCH: NO-DELTA\n" |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1747 self._write_debug(msg) |
35738
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1748 if nominateddeltas: |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1749 deltainfo = min(nominateddeltas, key=lambda x: x.deltalen) |
51334
d0d869fccd20
delta-find: move away from the generator API for _DeltaSearch
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51333
diff
changeset
|
1750 search.next_group(deltainfo) |
35738
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1751 |
39333
5d343a24bff5
revlogdeltas: always return a delta info object in finddeltainfo
Boris Feld <boris.feld@octobus.net>
parents:
39332
diff
changeset
|
1752 if deltainfo is None: |
49226
e6b7c6fbeb48
deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
1753 dbg_type = b"full" |
51020
509f0f7fc89e
delta-computer: stop explicitly taking file handle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50929
diff
changeset
|
1754 deltainfo = self._fullsnapshotinfo(revinfo, target_rev) |
49228
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1755 elif deltainfo.snapshotdepth: # pytype: disable=attribute-error |
49226
e6b7c6fbeb48
deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
1756 dbg_type = b"snapshot" |
e6b7c6fbeb48
deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
1757 else: |
e6b7c6fbeb48
deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
1758 dbg_type = b"delta" |
e6b7c6fbeb48
deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
1759 |
49610
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
1760 if gather_debug: |
49226
e6b7c6fbeb48
deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
1761 end = util.timer() |
49775
d57b966cdeb1
delta-find: properly report full snapshot used from cache as such
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49768
diff
changeset
|
1762 if dbg_type == b'full': |
d57b966cdeb1
delta-find: properly report full snapshot used from cache as such
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49768
diff
changeset
|
1763 used_cached = ( |
d57b966cdeb1
delta-find: properly report full snapshot used from cache as such
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49768
diff
changeset
|
1764 cachedelta is not None |
d57b966cdeb1
delta-find: properly report full snapshot used from cache as such
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49768
diff
changeset
|
1765 and dbg_try_rounds == 0 |
d57b966cdeb1
delta-find: properly report full snapshot used from cache as such
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49768
diff
changeset
|
1766 and dbg_try_count == 0 |
d57b966cdeb1
delta-find: properly report full snapshot used from cache as such
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49768
diff
changeset
|
1767 and cachedelta[0] == nullrev |
d57b966cdeb1
delta-find: properly report full snapshot used from cache as such
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49768
diff
changeset
|
1768 ) |
d57b966cdeb1
delta-find: properly report full snapshot used from cache as such
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49768
diff
changeset
|
1769 else: |
d57b966cdeb1
delta-find: properly report full snapshot used from cache as such
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49768
diff
changeset
|
1770 used_cached = ( |
d57b966cdeb1
delta-find: properly report full snapshot used from cache as such
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49768
diff
changeset
|
1771 cachedelta is not None |
d57b966cdeb1
delta-find: properly report full snapshot used from cache as such
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49768
diff
changeset
|
1772 and dbg_try_rounds == 1 |
d57b966cdeb1
delta-find: properly report full snapshot used from cache as such
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49768
diff
changeset
|
1773 and dbg_try_count == 1 |
d57b966cdeb1
delta-find: properly report full snapshot used from cache as such
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49768
diff
changeset
|
1774 and deltainfo.base == cachedelta[0] |
d57b966cdeb1
delta-find: properly report full snapshot used from cache as such
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49768
diff
changeset
|
1775 ) |
50651
d1dc4a03125e
delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50650
diff
changeset
|
1776 dbg['duration'] = end - start |
d1dc4a03125e
delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50650
diff
changeset
|
1777 dbg[ |
d1dc4a03125e
delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50650
diff
changeset
|
1778 'delta-base' |
d1dc4a03125e
delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50650
diff
changeset
|
1779 ] = deltainfo.base # pytype: disable=attribute-error |
d1dc4a03125e
delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50650
diff
changeset
|
1780 dbg['search_round_count'] = dbg_try_rounds |
d1dc4a03125e
delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50650
diff
changeset
|
1781 dbg['using-cached-base'] = used_cached |
d1dc4a03125e
delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50650
diff
changeset
|
1782 dbg['delta_try_count'] = dbg_try_count |
d1dc4a03125e
delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50650
diff
changeset
|
1783 dbg['type'] = dbg_type |
49228
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1784 if ( |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1785 deltainfo.snapshotdepth # pytype: disable=attribute-error |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1786 is not None |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1787 ): |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1788 dbg[ |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1789 'snapshot-depth' |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1790 ] = deltainfo.snapshotdepth # pytype: disable=attribute-error |
49226
e6b7c6fbeb48
deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
1791 else: |
51333
0d92d62ecde0
delta-find: use "-1" as depth snapshot-dept for non snapshot in debug
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51332
diff
changeset
|
1792 dbg['snapshot-depth'] = -1 |
50650
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1793 self._dbg_process_data(dbg) |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1794 return deltainfo |
49226
e6b7c6fbeb48
deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
1795 |
50651
d1dc4a03125e
delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50650
diff
changeset
|
1796 def _one_dbg_data(self): |
51328
46415df4eb1c
delta-find: move filing of some debug data in `_one_dbg_data`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51327
diff
changeset
|
1797 dbg = { |
50651
d1dc4a03125e
delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50650
diff
changeset
|
1798 'duration': None, |
d1dc4a03125e
delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50650
diff
changeset
|
1799 'revision': None, |
d1dc4a03125e
delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50650
diff
changeset
|
1800 'delta-base': None, |
d1dc4a03125e
delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50650
diff
changeset
|
1801 'search_round_count': None, |
d1dc4a03125e
delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50650
diff
changeset
|
1802 'using-cached-base': None, |
d1dc4a03125e
delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50650
diff
changeset
|
1803 'delta_try_count': None, |
d1dc4a03125e
delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50650
diff
changeset
|
1804 'type': None, |
d1dc4a03125e
delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50650
diff
changeset
|
1805 'p1-chain-len': None, |
d1dc4a03125e
delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50650
diff
changeset
|
1806 'p2-chain-len': None, |
d1dc4a03125e
delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50650
diff
changeset
|
1807 'snapshot-depth': None, |
d1dc4a03125e
delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50650
diff
changeset
|
1808 'target-revlog': None, |
d1dc4a03125e
delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50650
diff
changeset
|
1809 } |
51328
46415df4eb1c
delta-find: move filing of some debug data in `_one_dbg_data`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51327
diff
changeset
|
1810 target_revlog = b"UNKNOWN" |
46415df4eb1c
delta-find: move filing of some debug data in `_one_dbg_data`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51327
diff
changeset
|
1811 target_type = self.revlog.target[0] |
46415df4eb1c
delta-find: move filing of some debug data in `_one_dbg_data`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51327
diff
changeset
|
1812 target_key = self.revlog.target[1] |
46415df4eb1c
delta-find: move filing of some debug data in `_one_dbg_data`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51327
diff
changeset
|
1813 if target_type == KIND_CHANGELOG: |
46415df4eb1c
delta-find: move filing of some debug data in `_one_dbg_data`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51327
diff
changeset
|
1814 target_revlog = b'CHANGELOG:' |
46415df4eb1c
delta-find: move filing of some debug data in `_one_dbg_data`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51327
diff
changeset
|
1815 elif target_type == KIND_MANIFESTLOG: |
46415df4eb1c
delta-find: move filing of some debug data in `_one_dbg_data`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51327
diff
changeset
|
1816 target_revlog = b'MANIFESTLOG:' |
46415df4eb1c
delta-find: move filing of some debug data in `_one_dbg_data`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51327
diff
changeset
|
1817 if target_key: |
46415df4eb1c
delta-find: move filing of some debug data in `_one_dbg_data`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51327
diff
changeset
|
1818 target_revlog += b'%s:' % target_key |
46415df4eb1c
delta-find: move filing of some debug data in `_one_dbg_data`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51327
diff
changeset
|
1819 elif target_type == KIND_FILELOG: |
46415df4eb1c
delta-find: move filing of some debug data in `_one_dbg_data`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51327
diff
changeset
|
1820 target_revlog = b'FILELOG:' |
46415df4eb1c
delta-find: move filing of some debug data in `_one_dbg_data`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51327
diff
changeset
|
1821 if target_key: |
46415df4eb1c
delta-find: move filing of some debug data in `_one_dbg_data`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51327
diff
changeset
|
1822 target_revlog += b'%s:' % target_key |
46415df4eb1c
delta-find: move filing of some debug data in `_one_dbg_data`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51327
diff
changeset
|
1823 dbg['target-revlog'] = target_revlog |
46415df4eb1c
delta-find: move filing of some debug data in `_one_dbg_data`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51327
diff
changeset
|
1824 return dbg |
50651
d1dc4a03125e
delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50650
diff
changeset
|
1825 |
50650
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1826 def _dbg_process_data(self, dbg): |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1827 if self._debug_info is not None: |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1828 self._debug_info.append(dbg) |
49610
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
1829 |
50650
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1830 if self._write_debug is not None: |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1831 msg = ( |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1832 b"DBG-DELTAS:" |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1833 b" %-12s" |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1834 b" rev=%d:" |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1835 b" delta-base=%d" |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1836 b" is-cached=%d" |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1837 b" - search-rounds=%d" |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1838 b" try-count=%d" |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1839 b" - delta-type=%-6s" |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1840 b" snap-depth=%d" |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1841 b" - p1-chain-length=%d" |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1842 b" p2-chain-length=%d" |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1843 b" - duration=%f" |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1844 b"\n" |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1845 ) |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1846 msg %= ( |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1847 dbg["target-revlog"], |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1848 dbg["revision"], |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1849 dbg["delta-base"], |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1850 dbg["using-cached-base"], |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1851 dbg["search_round_count"], |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1852 dbg["delta_try_count"], |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1853 dbg["type"], |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1854 dbg["snapshot-depth"], |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1855 dbg["p1-chain-len"], |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1856 dbg["p2-chain-len"], |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1857 dbg["duration"], |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1858 ) |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1859 self._write_debug(msg) |
47452
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1860 |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1861 |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1862 def delta_compression(default_compression_header, deltainfo): |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1863 """return (COMPRESSION_MODE, deltainfo) |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1864 |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1865 used by revlog v2+ format to dispatch between PLAIN and DEFAULT |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1866 compression. |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1867 """ |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1868 h, d = deltainfo.data |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1869 compression_mode = COMP_MODE_INLINE |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1870 if not h and not d: |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1871 # not data to store at all... declare them uncompressed |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1872 compression_mode = COMP_MODE_PLAIN |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1873 elif not h: |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1874 t = d[0:1] |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1875 if t == b'\0': |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1876 compression_mode = COMP_MODE_PLAIN |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1877 elif t == default_compression_header: |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1878 compression_mode = COMP_MODE_DEFAULT |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1879 elif h == b'u': |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1880 # we have a more efficient way to declare uncompressed |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1881 h = b'' |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1882 compression_mode = COMP_MODE_PLAIN |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1883 deltainfo = drop_u_compression(deltainfo) |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1884 return compression_mode, deltainfo |