Mercurial > hg
annotate mercurial/revlogutils/deltas.py @ 51310:f0e7d51bb454 stable
pycompat: fix bytestr(bytes) in Python 3.11
In Python 3.10, the `bytes` type itself does not have a `__bytes__`
attribute, but it does in 3.11. Yet `bytes(bytes)` does not give
the wished output, so we have to add an exceptional case.
The added case in the doctest reproduces the problem with Python 3.11.
Impact: error treatment in expressions such as `repo[b'invalid']` gets
broken.
author | Georges Racinet <georges.racinet@octobus.net> |
---|---|
date | Wed, 03 Jan 2024 18:33:39 +0100 |
parents | a82704902db8 |
children | da7ecb4deaec |
rev | line source |
---|---|
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
1 # revlogdeltas.py - Logic around delta computation for revlog |
8226
8b2cd04a6e97
put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents:
8225
diff
changeset
|
2 # |
46819
d4ba4d51f85f
contributor: change mentions of mpm to olivia
Raphaël Gomès <rgomes@octobus.net>
parents:
43089
diff
changeset
|
3 # Copyright 2005-2007 Olivia Mackall <olivia@selenic.com> |
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
4 # Copyright 2018 Octobus <contact@octobus.net> |
8226
8b2cd04a6e97
put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents:
8225
diff
changeset
|
5 # |
8b2cd04a6e97
put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents:
8225
diff
changeset
|
6 # This software may be used and distributed according to the terms of the |
10263 | 7 # GNU General Public License version 2 or any later version. |
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
8 """Helper class to compute deltas stored inside revlogs""" |
8227
0a9542703300
turn some comments back into module docstrings
Martin Geisler <mg@lazybytes.net>
parents:
8226
diff
changeset
|
9 |
27361
29f50344fa83
revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27251
diff
changeset
|
10 |
39493
3ca144f1c8dd
snapshot: search for unrelated but reusable full-snapshot
Boris Feld <boris.feld@octobus.net>
parents:
39492
diff
changeset
|
11 import collections |
27361
29f50344fa83
revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27251
diff
changeset
|
12 import struct |
29f50344fa83
revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27251
diff
changeset
|
13 |
29f50344fa83
revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27251
diff
changeset
|
14 # import stuff from node for others to import from revlog |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
15 from ..node import nullrev |
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
16 from ..i18n import _ |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
17 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
18 from .constants import ( |
47452
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
19 COMP_MODE_DEFAULT, |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
20 COMP_MODE_INLINE, |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
21 COMP_MODE_PLAIN, |
49768
bcae90c53def
delta-find: add a delta-reuse policy that blindly accepts incoming deltas
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49681
diff
changeset
|
22 DELTA_BASE_REUSE_FORCE, |
49677
05db41701ece
find-delta: pass the cache-delta usage policy alongside the cache-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49675
diff
changeset
|
23 DELTA_BASE_REUSE_NO, |
49226
e6b7c6fbeb48
deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
24 KIND_CHANGELOG, |
e6b7c6fbeb48
deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
25 KIND_FILELOG, |
e6b7c6fbeb48
deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
26 KIND_MANIFESTLOG, |
39329
729082bb9938
revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents:
39232
diff
changeset
|
27 REVIDX_ISCENSORED, |
729082bb9938
revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents:
39232
diff
changeset
|
28 REVIDX_RAWTEXT_CHANGING_FLAGS, |
729082bb9938
revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents:
39232
diff
changeset
|
29 ) |
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
30 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
31 from ..thirdparty import attr |
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
32 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
33 from .. import ( |
27361
29f50344fa83
revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27251
diff
changeset
|
34 error, |
29f50344fa83
revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27251
diff
changeset
|
35 mdiff, |
41108
38e88450138c
delta: have a native implementation of _findsnapshot
Boris Feld <boris.feld@octobus.net>
parents:
41079
diff
changeset
|
36 util, |
27361
29f50344fa83
revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27251
diff
changeset
|
37 ) |
10913
f2ecc5733c89
revlog: factor out _maxinline global.
Greg Ward <greg-hg@gerg.ca>
parents:
10404
diff
changeset
|
38 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
39 from . import flagutil |
42992
dff95420480f
flagprocessors: make `processflagsraw` a module level function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42878
diff
changeset
|
40 |
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
41 # maximum <delta-chain-data>/<revision-text-length> ratio |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
42 LIMIT_DELTA2TEXT = 2 |
1091
d62130f99a73
Move hash function back to revlog from node
mpm@selenic.com
parents:
1089
diff
changeset
|
43 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
44 |
48946
642e31cb55f0
py3: use class X: instead of class X(object):
Gregory Szorc <gregory.szorc@gmail.com>
parents:
48875
diff
changeset
|
45 class _testrevlog: |
38637
e33f784f2a44
revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents:
38636
diff
changeset
|
46 """minimalist fake revlog to use in doctests""" |
e33f784f2a44
revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents:
38636
diff
changeset
|
47 |
40641
85b14f0dc334
doctest: add a `issnapshot` method to _testrevlog
Boris Feld <boris.feld@octobus.net>
parents:
40608
diff
changeset
|
48 def __init__(self, data, density=0.5, mingap=0, snapshot=()): |
38637
e33f784f2a44
revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents:
38636
diff
changeset
|
49 """data is an list of revision payload boundaries""" |
51045
127656e0b97b
revlog: use the new Config classes in _testrevlog
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51020
diff
changeset
|
50 from .. import revlog |
127656e0b97b
revlog: use the new Config classes in _testrevlog
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51020
diff
changeset
|
51 |
38637
e33f784f2a44
revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents:
38636
diff
changeset
|
52 self._data = data |
51045
127656e0b97b
revlog: use the new Config classes in _testrevlog
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51020
diff
changeset
|
53 self.data_config = revlog.DataConfig() |
127656e0b97b
revlog: use the new Config classes in _testrevlog
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51020
diff
changeset
|
54 self.data_config.sr_density_threshold = density |
127656e0b97b
revlog: use the new Config classes in _testrevlog
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51020
diff
changeset
|
55 self.data_config.sr_min_gap_size = mingap |
127656e0b97b
revlog: use the new Config classes in _testrevlog
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51020
diff
changeset
|
56 self.delta_config = revlog.DeltaConfig() |
127656e0b97b
revlog: use the new Config classes in _testrevlog
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51020
diff
changeset
|
57 self.feature_config = revlog.FeatureConfig() |
40641
85b14f0dc334
doctest: add a `issnapshot` method to _testrevlog
Boris Feld <boris.feld@octobus.net>
parents:
40608
diff
changeset
|
58 self._snapshot = set(snapshot) |
40709
39d29542fe40
sparse-revlog: put the native implementation of slicechunktodensity to use
Boris Feld <boris.feld@octobus.net>
parents:
40657
diff
changeset
|
59 self.index = None |
38637
e33f784f2a44
revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents:
38636
diff
changeset
|
60 |
e33f784f2a44
revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents:
38636
diff
changeset
|
61 def start(self, rev): |
41079
88d813cd9acd
revlog: fix pure python slicing test when chain contains nullrev
Boris Feld <boris.feld@octobus.net>
parents:
41033
diff
changeset
|
62 if rev == nullrev: |
88d813cd9acd
revlog: fix pure python slicing test when chain contains nullrev
Boris Feld <boris.feld@octobus.net>
parents:
41033
diff
changeset
|
63 return 0 |
38637
e33f784f2a44
revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents:
38636
diff
changeset
|
64 if rev == 0: |
e33f784f2a44
revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents:
38636
diff
changeset
|
65 return 0 |
e33f784f2a44
revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents:
38636
diff
changeset
|
66 return self._data[rev - 1] |
e33f784f2a44
revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents:
38636
diff
changeset
|
67 |
e33f784f2a44
revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents:
38636
diff
changeset
|
68 def end(self, rev): |
41079
88d813cd9acd
revlog: fix pure python slicing test when chain contains nullrev
Boris Feld <boris.feld@octobus.net>
parents:
41033
diff
changeset
|
69 if rev == nullrev: |
88d813cd9acd
revlog: fix pure python slicing test when chain contains nullrev
Boris Feld <boris.feld@octobus.net>
parents:
41033
diff
changeset
|
70 return 0 |
38637
e33f784f2a44
revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents:
38636
diff
changeset
|
71 return self._data[rev] |
e33f784f2a44
revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents:
38636
diff
changeset
|
72 |
e33f784f2a44
revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents:
38636
diff
changeset
|
73 def length(self, rev): |
e33f784f2a44
revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents:
38636
diff
changeset
|
74 return self.end(rev) - self.start(rev) |
e33f784f2a44
revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents:
38636
diff
changeset
|
75 |
38718
f8762ea73e0d
sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents:
38717
diff
changeset
|
76 def __len__(self): |
f8762ea73e0d
sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents:
38717
diff
changeset
|
77 return len(self._data) |
f8762ea73e0d
sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents:
38717
diff
changeset
|
78 |
40641
85b14f0dc334
doctest: add a `issnapshot` method to _testrevlog
Boris Feld <boris.feld@octobus.net>
parents:
40608
diff
changeset
|
79 def issnapshot(self, rev): |
41079
88d813cd9acd
revlog: fix pure python slicing test when chain contains nullrev
Boris Feld <boris.feld@octobus.net>
parents:
41033
diff
changeset
|
80 if rev == nullrev: |
88d813cd9acd
revlog: fix pure python slicing test when chain contains nullrev
Boris Feld <boris.feld@octobus.net>
parents:
41033
diff
changeset
|
81 return True |
40641
85b14f0dc334
doctest: add a `issnapshot` method to _testrevlog
Boris Feld <boris.feld@octobus.net>
parents:
40608
diff
changeset
|
82 return rev in self._snapshot |
85b14f0dc334
doctest: add a `issnapshot` method to _testrevlog
Boris Feld <boris.feld@octobus.net>
parents:
40608
diff
changeset
|
83 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
84 |
40604
3ac23dad6364
sparse-revlog: drop unused deltainfo parameter from _slicechunktodensity
Boris Feld <boris.feld@octobus.net>
parents:
40603
diff
changeset
|
85 def slicechunk(revlog, revs, targetsize=None): |
34824
e2ad93bcc084
revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents:
34823
diff
changeset
|
86 """slice revs to reduce the amount of unrelated data to be read from disk. |
e2ad93bcc084
revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents:
34823
diff
changeset
|
87 |
e2ad93bcc084
revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents:
34823
diff
changeset
|
88 ``revs`` is sliced into groups that should be read in one time. |
e2ad93bcc084
revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents:
34823
diff
changeset
|
89 Assume that revs are sorted. |
38640
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
90 |
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
91 The initial chunk is sliced until the overall density (payload/chunks-span |
51062
8c614fa16330
revlog: remove legacy usage of `_srdensitythreshold`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51060
diff
changeset
|
92 ratio) is above `revlog.data_config.sr_density_threshold`. No gap smaller |
51063
533d6943f6a3
revlog: remove legacy usage of `_srmingapsize`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51062
diff
changeset
|
93 than `revlog.data_config.sr_min_gap_size` is skipped. |
38640
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
94 |
38643
967fee55e8d9
revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents:
38642
diff
changeset
|
95 If `targetsize` is set, no chunk larger than `targetsize` will be yield. |
967fee55e8d9
revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents:
38642
diff
changeset
|
96 For consistency with other slicing choice, this limit won't go lower than |
51063
533d6943f6a3
revlog: remove legacy usage of `_srmingapsize`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51062
diff
changeset
|
97 `revlog.data_config.sr_min_gap_size`. |
38643
967fee55e8d9
revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents:
38642
diff
changeset
|
98 |
967fee55e8d9
revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents:
38642
diff
changeset
|
99 If individual revisions chunk are larger than this limit, they will still |
967fee55e8d9
revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents:
38642
diff
changeset
|
100 be raised individually. |
967fee55e8d9
revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents:
38642
diff
changeset
|
101 |
40642
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
102 >>> data = [ |
38640
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
103 ... 5, #00 (5) |
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
104 ... 10, #01 (5) |
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
105 ... 12, #02 (2) |
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
106 ... 12, #03 (empty) |
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
107 ... 27, #04 (15) |
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
108 ... 31, #05 (4) |
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
109 ... 31, #06 (empty) |
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
110 ... 42, #07 (11) |
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
111 ... 47, #08 (5) |
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
112 ... 47, #09 (empty) |
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
113 ... 48, #10 (1) |
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
114 ... 51, #11 (3) |
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
115 ... 74, #12 (23) |
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
116 ... 85, #13 (11) |
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
117 ... 86, #14 (1) |
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
118 ... 91, #15 (5) |
40642
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
119 ... ] |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
120 >>> revlog = _testrevlog(data, snapshot=range(16)) |
38640
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
121 |
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
122 >>> list(slicechunk(revlog, list(range(16)))) |
38640
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
123 [[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]] |
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
124 >>> list(slicechunk(revlog, [0, 15])) |
38640
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
125 [[0], [15]] |
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
126 >>> list(slicechunk(revlog, [0, 11, 15])) |
38640
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
127 [[0], [11], [15]] |
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
128 >>> list(slicechunk(revlog, [0, 11, 13, 15])) |
38640
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
129 [[0], [11, 13, 15]] |
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
130 >>> list(slicechunk(revlog, [1, 2, 3, 5, 8, 10, 11, 14])) |
38640
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
131 [[1, 2], [5, 8, 10, 11], [14]] |
38643
967fee55e8d9
revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents:
38642
diff
changeset
|
132 |
967fee55e8d9
revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents:
38642
diff
changeset
|
133 Slicing with a maximum chunk size |
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
134 >>> list(slicechunk(revlog, [0, 11, 13, 15], targetsize=15)) |
38643
967fee55e8d9
revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents:
38642
diff
changeset
|
135 [[0], [11], [13], [15]] |
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
136 >>> list(slicechunk(revlog, [0, 11, 13, 15], targetsize=20)) |
38643
967fee55e8d9
revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents:
38642
diff
changeset
|
137 [[0], [11], [13, 15]] |
41079
88d813cd9acd
revlog: fix pure python slicing test when chain contains nullrev
Boris Feld <boris.feld@octobus.net>
parents:
41033
diff
changeset
|
138 |
88d813cd9acd
revlog: fix pure python slicing test when chain contains nullrev
Boris Feld <boris.feld@octobus.net>
parents:
41033
diff
changeset
|
139 Slicing involving nullrev |
88d813cd9acd
revlog: fix pure python slicing test when chain contains nullrev
Boris Feld <boris.feld@octobus.net>
parents:
41033
diff
changeset
|
140 >>> list(slicechunk(revlog, [-1, 0, 11, 13, 15], targetsize=20)) |
88d813cd9acd
revlog: fix pure python slicing test when chain contains nullrev
Boris Feld <boris.feld@octobus.net>
parents:
41033
diff
changeset
|
141 [[-1, 0], [11], [13, 15]] |
88d813cd9acd
revlog: fix pure python slicing test when chain contains nullrev
Boris Feld <boris.feld@octobus.net>
parents:
41033
diff
changeset
|
142 >>> list(slicechunk(revlog, [-1, 13, 15], targetsize=5)) |
88d813cd9acd
revlog: fix pure python slicing test when chain contains nullrev
Boris Feld <boris.feld@octobus.net>
parents:
41033
diff
changeset
|
143 [[-1], [13], [15]] |
34824
e2ad93bcc084
revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents:
34823
diff
changeset
|
144 """ |
38643
967fee55e8d9
revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents:
38642
diff
changeset
|
145 if targetsize is not None: |
51063
533d6943f6a3
revlog: remove legacy usage of `_srmingapsize`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51062
diff
changeset
|
146 targetsize = max(targetsize, revlog.data_config.sr_min_gap_size) |
38718
f8762ea73e0d
sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents:
38717
diff
changeset
|
147 # targetsize should not be specified when evaluating delta candidates: |
f8762ea73e0d
sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents:
38717
diff
changeset
|
148 # * targetsize is used to ensure we stay within specification when reading, |
40709
39d29542fe40
sparse-revlog: put the native implementation of slicechunktodensity to use
Boris Feld <boris.feld@octobus.net>
parents:
40657
diff
changeset
|
149 densityslicing = getattr(revlog.index, 'slicechunktodensity', None) |
39d29542fe40
sparse-revlog: put the native implementation of slicechunktodensity to use
Boris Feld <boris.feld@octobus.net>
parents:
40657
diff
changeset
|
150 if densityslicing is None: |
39d29542fe40
sparse-revlog: put the native implementation of slicechunktodensity to use
Boris Feld <boris.feld@octobus.net>
parents:
40657
diff
changeset
|
151 densityslicing = lambda x, y, z: _slicechunktodensity(revlog, x, y, z) |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
152 for chunk in densityslicing( |
51063
533d6943f6a3
revlog: remove legacy usage of `_srmingapsize`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51062
diff
changeset
|
153 revs, |
533d6943f6a3
revlog: remove legacy usage of `_srmingapsize`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51062
diff
changeset
|
154 revlog.data_config.sr_density_threshold, |
533d6943f6a3
revlog: remove legacy usage of `_srmingapsize`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51062
diff
changeset
|
155 revlog.data_config.sr_min_gap_size, |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
156 ): |
38643
967fee55e8d9
revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents:
38642
diff
changeset
|
157 for subchunk in _slicechunktosize(revlog, chunk, targetsize): |
967fee55e8d9
revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents:
38642
diff
changeset
|
158 yield subchunk |
38641
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
159 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
160 |
38718
f8762ea73e0d
sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents:
38717
diff
changeset
|
161 def _slicechunktosize(revlog, revs, targetsize=None): |
38642
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
162 """slice revs to match the target size |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
163 |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
164 This is intended to be used on chunk that density slicing selected by that |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
165 are still too large compared to the read garantee of revlog. This might |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
166 happens when "minimal gap size" interrupted the slicing or when chain are |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
167 built in a way that create large blocks next to each other. |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
168 |
40642
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
169 >>> data = [ |
38642
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
170 ... 3, #0 (3) |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
171 ... 5, #1 (2) |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
172 ... 6, #2 (1) |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
173 ... 8, #3 (2) |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
174 ... 8, #4 (empty) |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
175 ... 11, #5 (3) |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
176 ... 12, #6 (1) |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
177 ... 13, #7 (1) |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
178 ... 14, #8 (1) |
40642
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
179 ... ] |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
180 |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
181 == All snapshots cases == |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
182 >>> revlog = _testrevlog(data, snapshot=range(9)) |
38642
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
183 |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
184 Cases where chunk is already small enough |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
185 >>> list(_slicechunktosize(revlog, [0], 3)) |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
186 [[0]] |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
187 >>> list(_slicechunktosize(revlog, [6, 7], 3)) |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
188 [[6, 7]] |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
189 >>> list(_slicechunktosize(revlog, [0], None)) |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
190 [[0]] |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
191 >>> list(_slicechunktosize(revlog, [6, 7], None)) |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
192 [[6, 7]] |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
193 |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
194 cases where we need actual slicing |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
195 >>> list(_slicechunktosize(revlog, [0, 1], 3)) |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
196 [[0], [1]] |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
197 >>> list(_slicechunktosize(revlog, [1, 3], 3)) |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
198 [[1], [3]] |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
199 >>> list(_slicechunktosize(revlog, [1, 2, 3], 3)) |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
200 [[1, 2], [3]] |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
201 >>> list(_slicechunktosize(revlog, [3, 5], 3)) |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
202 [[3], [5]] |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
203 >>> list(_slicechunktosize(revlog, [3, 4, 5], 3)) |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
204 [[3], [5]] |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
205 >>> list(_slicechunktosize(revlog, [5, 6, 7, 8], 3)) |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
206 [[5], [6, 7, 8]] |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
207 >>> list(_slicechunktosize(revlog, [0, 1, 2, 3, 4, 5, 6, 7, 8], 3)) |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
208 [[0], [1, 2], [3], [5], [6, 7, 8]] |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
209 |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
210 Case with too large individual chunk (must return valid chunk) |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
211 >>> list(_slicechunktosize(revlog, [0, 1], 2)) |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
212 [[0], [1]] |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
213 >>> list(_slicechunktosize(revlog, [1, 3], 1)) |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
214 [[1], [3]] |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
215 >>> list(_slicechunktosize(revlog, [3, 4, 5], 2)) |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
216 [[3], [5]] |
40642
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
217 |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
218 == No Snapshot cases == |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
219 >>> revlog = _testrevlog(data) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
220 |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
221 Cases where chunk is already small enough |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
222 >>> list(_slicechunktosize(revlog, [0], 3)) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
223 [[0]] |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
224 >>> list(_slicechunktosize(revlog, [6, 7], 3)) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
225 [[6, 7]] |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
226 >>> list(_slicechunktosize(revlog, [0], None)) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
227 [[0]] |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
228 >>> list(_slicechunktosize(revlog, [6, 7], None)) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
229 [[6, 7]] |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
230 |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
231 cases where we need actual slicing |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
232 >>> list(_slicechunktosize(revlog, [0, 1], 3)) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
233 [[0], [1]] |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
234 >>> list(_slicechunktosize(revlog, [1, 3], 3)) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
235 [[1], [3]] |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
236 >>> list(_slicechunktosize(revlog, [1, 2, 3], 3)) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
237 [[1], [2, 3]] |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
238 >>> list(_slicechunktosize(revlog, [3, 5], 3)) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
239 [[3], [5]] |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
240 >>> list(_slicechunktosize(revlog, [3, 4, 5], 3)) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
241 [[3], [4, 5]] |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
242 >>> list(_slicechunktosize(revlog, [5, 6, 7, 8], 3)) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
243 [[5], [6, 7, 8]] |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
244 >>> list(_slicechunktosize(revlog, [0, 1, 2, 3, 4, 5, 6, 7, 8], 3)) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
245 [[0], [1, 2], [3], [5], [6, 7, 8]] |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
246 |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
247 Case with too large individual chunk (must return valid chunk) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
248 >>> list(_slicechunktosize(revlog, [0, 1], 2)) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
249 [[0], [1]] |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
250 >>> list(_slicechunktosize(revlog, [1, 3], 1)) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
251 [[1], [3]] |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
252 >>> list(_slicechunktosize(revlog, [3, 4, 5], 2)) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
253 [[3], [5]] |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
254 |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
255 == mixed case == |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
256 >>> revlog = _testrevlog(data, snapshot=[0, 1, 2]) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
257 >>> list(_slicechunktosize(revlog, list(range(9)), 5)) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
258 [[0, 1], [2], [3, 4, 5], [6, 7, 8]] |
38642
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
259 """ |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
260 assert targetsize is None or 0 <= targetsize |
40642
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
261 startdata = revlog.start(revs[0]) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
262 enddata = revlog.end(revs[-1]) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
263 fullspan = enddata - startdata |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
264 if targetsize is None or fullspan <= targetsize: |
38642
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
265 yield revs |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
266 return |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
267 |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
268 startrevidx = 0 |
40657
2eb48aa0acce
sparse-revlog: align endrevidx usages in the _slicechunktosize
Boris Feld <boris.feld@octobus.net>
parents:
40654
diff
changeset
|
269 endrevidx = 1 |
38642
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
270 iterrevs = enumerate(revs) |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
271 next(iterrevs) # skip first rev. |
40642
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
272 # first step: get snapshots out of the way |
38642
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
273 for idx, r in iterrevs: |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
274 span = revlog.end(r) - startdata |
40642
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
275 snapshot = revlog.issnapshot(r) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
276 if span <= targetsize and snapshot: |
40657
2eb48aa0acce
sparse-revlog: align endrevidx usages in the _slicechunktosize
Boris Feld <boris.feld@octobus.net>
parents:
40654
diff
changeset
|
277 endrevidx = idx + 1 |
38642
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
278 else: |
40657
2eb48aa0acce
sparse-revlog: align endrevidx usages in the _slicechunktosize
Boris Feld <boris.feld@octobus.net>
parents:
40654
diff
changeset
|
279 chunk = _trimchunk(revlog, revs, startrevidx, endrevidx) |
38642
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
280 if chunk: |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
281 yield chunk |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
282 startrevidx = idx |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
283 startdata = revlog.start(r) |
40657
2eb48aa0acce
sparse-revlog: align endrevidx usages in the _slicechunktosize
Boris Feld <boris.feld@octobus.net>
parents:
40654
diff
changeset
|
284 endrevidx = idx + 1 |
40642
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
285 if not snapshot: |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
286 break |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
287 |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
288 # for the others, we use binary slicing to quickly converge toward valid |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
289 # chunks (otherwise, we might end up looking for start/end of many |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
290 # revisions). This logic is not looking for the perfect slicing point, it |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
291 # focuses on quickly converging toward valid chunks. |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
292 nbitem = len(revs) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
293 while (enddata - startdata) > targetsize: |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
294 endrevidx = nbitem |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
295 if nbitem - startrevidx <= 1: |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
296 break # protect against individual chunk larger than limit |
40642
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
297 localenddata = revlog.end(revs[endrevidx - 1]) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
298 span = localenddata - startdata |
40654
fd1d41ccbe38
sparse-revlog: use `span` variable as intended
Boris Feld <boris.feld@octobus.net>
parents:
40642
diff
changeset
|
299 while span > targetsize: |
40642
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
300 if endrevidx - startrevidx <= 1: |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
301 break # protect against individual chunk larger than limit |
40642
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
302 endrevidx -= (endrevidx - startrevidx) // 2 |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
303 localenddata = revlog.end(revs[endrevidx - 1]) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
304 span = localenddata - startdata |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
305 chunk = _trimchunk(revlog, revs, startrevidx, endrevidx) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
306 if chunk: |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
307 yield chunk |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
308 startrevidx = endrevidx |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
309 startdata = revlog.start(revs[startrevidx]) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
310 |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
311 chunk = _trimchunk(revlog, revs, startrevidx) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
312 if chunk: |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
313 yield chunk |
38642
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
314 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
315 |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
316 def _slicechunktodensity(revlog, revs, targetdensity=0.5, mingapsize=0): |
38641
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
317 """slice revs to reduce the amount of unrelated data to be read from disk. |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
318 |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
319 ``revs`` is sliced into groups that should be read in one time. |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
320 Assume that revs are sorted. |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
321 |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
322 The initial chunk is sliced until the overall density (payload/chunks-span |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
323 ratio) is above `targetdensity`. No gap smaller than `mingapsize` is |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
324 skipped. |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
325 |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
326 >>> revlog = _testrevlog([ |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
327 ... 5, #00 (5) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
328 ... 10, #01 (5) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
329 ... 12, #02 (2) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
330 ... 12, #03 (empty) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
331 ... 27, #04 (15) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
332 ... 31, #05 (4) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
333 ... 31, #06 (empty) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
334 ... 42, #07 (11) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
335 ... 47, #08 (5) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
336 ... 47, #09 (empty) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
337 ... 48, #10 (1) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
338 ... 51, #11 (3) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
339 ... 74, #12 (23) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
340 ... 85, #13 (11) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
341 ... 86, #14 (1) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
342 ... 91, #15 (5) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
343 ... ]) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
344 |
38655
cd1c484e31e8
revlog: adjust doctest examples to be portable to Python 3
Augie Fackler <augie@google.com>
parents:
38644
diff
changeset
|
345 >>> list(_slicechunktodensity(revlog, list(range(16)))) |
38641
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
346 [[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]] |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
347 >>> list(_slicechunktodensity(revlog, [0, 15])) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
348 [[0], [15]] |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
349 >>> list(_slicechunktodensity(revlog, [0, 11, 15])) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
350 [[0], [11], [15]] |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
351 >>> list(_slicechunktodensity(revlog, [0, 11, 13, 15])) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
352 [[0], [11, 13, 15]] |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
353 >>> list(_slicechunktodensity(revlog, [1, 2, 3, 5, 8, 10, 11, 14])) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
354 [[1, 2], [5, 8, 10, 11], [14]] |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
355 >>> list(_slicechunktodensity(revlog, [1, 2, 3, 5, 8, 10, 11, 14], |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
356 ... mingapsize=20)) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
357 [[1, 2, 3, 5, 8, 10, 11], [14]] |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
358 >>> list(_slicechunktodensity(revlog, [1, 2, 3, 5, 8, 10, 11, 14], |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
359 ... targetdensity=0.95)) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
360 [[1, 2], [5], [8, 10, 11], [14]] |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
361 >>> list(_slicechunktodensity(revlog, [1, 2, 3, 5, 8, 10, 11, 14], |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
362 ... targetdensity=0.95, mingapsize=12)) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
363 [[1, 2], [5, 8, 10, 11], [14]] |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
364 """ |
34824
e2ad93bcc084
revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents:
34823
diff
changeset
|
365 start = revlog.start |
e2ad93bcc084
revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents:
34823
diff
changeset
|
366 length = revlog.length |
e2ad93bcc084
revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents:
34823
diff
changeset
|
367 |
34880
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
368 if len(revs) <= 1: |
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
369 yield revs |
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
370 return |
34824
e2ad93bcc084
revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents:
34823
diff
changeset
|
371 |
40604
3ac23dad6364
sparse-revlog: drop unused deltainfo parameter from _slicechunktodensity
Boris Feld <boris.feld@octobus.net>
parents:
40603
diff
changeset
|
372 deltachainspan = segmentspan(revlog, revs) |
34880
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
373 |
38641
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
374 if deltachainspan < mingapsize: |
38635
d083ae26c325
revlog: early return in _slicechunk when span is already small enough
Boris Feld <boris.feld@octobus.net>
parents:
38634
diff
changeset
|
375 yield revs |
d083ae26c325
revlog: early return in _slicechunk when span is already small enough
Boris Feld <boris.feld@octobus.net>
parents:
38634
diff
changeset
|
376 return |
d083ae26c325
revlog: early return in _slicechunk when span is already small enough
Boris Feld <boris.feld@octobus.net>
parents:
38634
diff
changeset
|
377 |
38718
f8762ea73e0d
sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents:
38717
diff
changeset
|
378 readdata = deltachainspan |
40606
bfbfd15d65bd
sparse-revlog: fast-path before computing payload size
Boris Feld <boris.feld@octobus.net>
parents:
40605
diff
changeset
|
379 chainpayload = sum(length(r) for r in revs) |
34824
e2ad93bcc084
revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents:
34823
diff
changeset
|
380 |
34880
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
381 if deltachainspan: |
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
382 density = chainpayload / float(deltachainspan) |
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
383 else: |
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
384 density = 1.0 |
34824
e2ad93bcc084
revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents:
34823
diff
changeset
|
385 |
38641
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
386 if density >= targetdensity: |
38634
f0ea8b847831
revlog: early return in _slicechunk when density is already good
Paul Morelle <paul.morelle@octobus.net>
parents:
38632
diff
changeset
|
387 yield revs |
f0ea8b847831
revlog: early return in _slicechunk when density is already good
Paul Morelle <paul.morelle@octobus.net>
parents:
38632
diff
changeset
|
388 return |
f0ea8b847831
revlog: early return in _slicechunk when density is already good
Paul Morelle <paul.morelle@octobus.net>
parents:
38632
diff
changeset
|
389 |
34880
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
390 # Store the gaps in a heap to have them sorted by decreasing size |
40607
54de23400b2a
sparse-revlog: stop using a heap to track gaps
Boris Feld <boris.feld@octobus.net>
parents:
40606
diff
changeset
|
391 gaps = [] |
34880
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
392 prevend = None |
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
393 for i, rev in enumerate(revs): |
40604
3ac23dad6364
sparse-revlog: drop unused deltainfo parameter from _slicechunktodensity
Boris Feld <boris.feld@octobus.net>
parents:
40603
diff
changeset
|
394 revstart = start(rev) |
3ac23dad6364
sparse-revlog: drop unused deltainfo parameter from _slicechunktodensity
Boris Feld <boris.feld@octobus.net>
parents:
40603
diff
changeset
|
395 revlen = length(rev) |
34824
e2ad93bcc084
revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents:
34823
diff
changeset
|
396 |
34898
1bde8e8e5de0
sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents:
34881
diff
changeset
|
397 # Skip empty revisions to form larger holes |
1bde8e8e5de0
sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents:
34881
diff
changeset
|
398 if revlen == 0: |
1bde8e8e5de0
sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents:
34881
diff
changeset
|
399 continue |
1bde8e8e5de0
sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents:
34881
diff
changeset
|
400 |
34880
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
401 if prevend is not None: |
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
402 gapsize = revstart - prevend |
34881
8c9b08a0c48c
sparse-read: skip gaps too small to be worth splitting
Paul Morelle <paul.morelle@octobus.net>
parents:
34880
diff
changeset
|
403 # only consider holes that are large enough |
38641
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
404 if gapsize > mingapsize: |
40607
54de23400b2a
sparse-revlog: stop using a heap to track gaps
Boris Feld <boris.feld@octobus.net>
parents:
40606
diff
changeset
|
405 gaps.append((gapsize, i)) |
34880
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
406 |
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
407 prevend = revstart + revlen |
40607
54de23400b2a
sparse-revlog: stop using a heap to track gaps
Boris Feld <boris.feld@octobus.net>
parents:
40606
diff
changeset
|
408 # sort the gaps to pop them from largest to small |
54de23400b2a
sparse-revlog: stop using a heap to track gaps
Boris Feld <boris.feld@octobus.net>
parents:
40606
diff
changeset
|
409 gaps.sort() |
34880
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
410 |
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
411 # Collect the indices of the largest holes until the density is acceptable |
40608
526ee887c4d5
sparse-revlog: stop using a heap to track selected gap
Boris Feld <boris.feld@octobus.net>
parents:
40607
diff
changeset
|
412 selected = [] |
40607
54de23400b2a
sparse-revlog: stop using a heap to track gaps
Boris Feld <boris.feld@octobus.net>
parents:
40606
diff
changeset
|
413 while gaps and density < targetdensity: |
54de23400b2a
sparse-revlog: stop using a heap to track gaps
Boris Feld <boris.feld@octobus.net>
parents:
40606
diff
changeset
|
414 gapsize, gapidx = gaps.pop() |
34880
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
415 |
40608
526ee887c4d5
sparse-revlog: stop using a heap to track selected gap
Boris Feld <boris.feld@octobus.net>
parents:
40607
diff
changeset
|
416 selected.append(gapidx) |
34824
e2ad93bcc084
revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents:
34823
diff
changeset
|
417 |
34880
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
418 # the gap sizes are stored as negatives to be sorted decreasingly |
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
419 # by the heap |
40607
54de23400b2a
sparse-revlog: stop using a heap to track gaps
Boris Feld <boris.feld@octobus.net>
parents:
40606
diff
changeset
|
420 readdata -= gapsize |
34880
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
421 if readdata > 0: |
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
422 density = chainpayload / float(readdata) |
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
423 else: |
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
424 density = 1.0 |
40608
526ee887c4d5
sparse-revlog: stop using a heap to track selected gap
Boris Feld <boris.feld@octobus.net>
parents:
40607
diff
changeset
|
425 selected.sort() |
34824
e2ad93bcc084
revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents:
34823
diff
changeset
|
426 |
34880
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
427 # Cut the revs at collected indices |
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
428 previdx = 0 |
40608
526ee887c4d5
sparse-revlog: stop using a heap to track selected gap
Boris Feld <boris.feld@octobus.net>
parents:
40607
diff
changeset
|
429 for idx in selected: |
34898
1bde8e8e5de0
sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents:
34881
diff
changeset
|
430 |
1bde8e8e5de0
sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents:
34881
diff
changeset
|
431 chunk = _trimchunk(revlog, revs, previdx, idx) |
1bde8e8e5de0
sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents:
34881
diff
changeset
|
432 if chunk: |
1bde8e8e5de0
sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents:
34881
diff
changeset
|
433 yield chunk |
1bde8e8e5de0
sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents:
34881
diff
changeset
|
434 |
34880
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
435 previdx = idx |
34898
1bde8e8e5de0
sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents:
34881
diff
changeset
|
436 |
1bde8e8e5de0
sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents:
34881
diff
changeset
|
437 chunk = _trimchunk(revlog, revs, previdx) |
1bde8e8e5de0
sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents:
34881
diff
changeset
|
438 if chunk: |
1bde8e8e5de0
sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents:
34881
diff
changeset
|
439 yield chunk |
34824
e2ad93bcc084
revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents:
34823
diff
changeset
|
440 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
441 |
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
442 def _trimchunk(revlog, revs, startidx, endidx=None): |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
443 """returns revs[startidx:endidx] without empty trailing revs |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
444 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
445 Doctest Setup |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
446 >>> revlog = _testrevlog([ |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
447 ... 5, #0 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
448 ... 10, #1 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
449 ... 12, #2 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
450 ... 12, #3 (empty) |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
451 ... 17, #4 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
452 ... 21, #5 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
453 ... 21, #6 (empty) |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
454 ... ]) |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
455 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
456 Contiguous cases: |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
457 >>> _trimchunk(revlog, [0, 1, 2, 3, 4, 5, 6], 0) |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
458 [0, 1, 2, 3, 4, 5] |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
459 >>> _trimchunk(revlog, [0, 1, 2, 3, 4, 5, 6], 0, 5) |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
460 [0, 1, 2, 3, 4] |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
461 >>> _trimchunk(revlog, [0, 1, 2, 3, 4, 5, 6], 0, 4) |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
462 [0, 1, 2] |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
463 >>> _trimchunk(revlog, [0, 1, 2, 3, 4, 5, 6], 2, 4) |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
464 [2] |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
465 >>> _trimchunk(revlog, [0, 1, 2, 3, 4, 5, 6], 3) |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
466 [3, 4, 5] |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
467 >>> _trimchunk(revlog, [0, 1, 2, 3, 4, 5, 6], 3, 5) |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
468 [3, 4] |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
469 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
470 Discontiguous cases: |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
471 >>> _trimchunk(revlog, [1, 3, 5, 6], 0) |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
472 [1, 3, 5] |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
473 >>> _trimchunk(revlog, [1, 3, 5, 6], 0, 2) |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
474 [1] |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
475 >>> _trimchunk(revlog, [1, 3, 5, 6], 1, 3) |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
476 [3, 5] |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
477 >>> _trimchunk(revlog, [1, 3, 5, 6], 1) |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
478 [3, 5] |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
479 """ |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
480 length = revlog.length |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
481 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
482 if endidx is None: |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
483 endidx = len(revs) |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
484 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
485 # If we have a non-emtpy delta candidate, there are nothing to trim |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
486 if revs[endidx - 1] < len(revlog): |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
487 # Trim empty revs at the end, except the very first revision of a chain |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
488 while ( |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
489 endidx > 1 and endidx > startidx and length(revs[endidx - 1]) == 0 |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
490 ): |
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
491 endidx -= 1 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
492 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
493 return revs[startidx:endidx] |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
494 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
495 |
40605
a32ccd32982b
sparse-revlog: drop unused deltainfo parameter from segmentspan
Boris Feld <boris.feld@octobus.net>
parents:
40604
diff
changeset
|
496 def segmentspan(revlog, revs): |
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
497 """Get the byte span of a segment of revisions |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
498 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
499 revs is a sorted array of revision numbers |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
500 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
501 >>> revlog = _testrevlog([ |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
502 ... 5, #0 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
503 ... 10, #1 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
504 ... 12, #2 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
505 ... 12, #3 (empty) |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
506 ... 17, #4 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
507 ... ]) |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
508 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
509 >>> segmentspan(revlog, [0, 1, 2, 3, 4]) |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
510 17 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
511 >>> segmentspan(revlog, [0, 4]) |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
512 17 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
513 >>> segmentspan(revlog, [3, 4]) |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
514 5 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
515 >>> segmentspan(revlog, [1, 2, 3,]) |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
516 7 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
517 >>> segmentspan(revlog, [1, 3]) |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
518 7 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
519 """ |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
520 if not revs: |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
521 return 0 |
40605
a32ccd32982b
sparse-revlog: drop unused deltainfo parameter from segmentspan
Boris Feld <boris.feld@octobus.net>
parents:
40604
diff
changeset
|
522 end = revlog.end(revs[-1]) |
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
523 return end - revlog.start(revs[0]) |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
524 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
525 |
51020
509f0f7fc89e
delta-computer: stop explicitly taking file handle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50929
diff
changeset
|
526 def _textfromdelta(revlog, baserev, delta, p1, p2, flags, expectednode): |
39331
fd0150a3c2fe
revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents:
39330
diff
changeset
|
527 """build full text from a (base, delta) pair and other metadata""" |
fd0150a3c2fe
revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents:
39330
diff
changeset
|
528 # special case deltas which replace entire base; no need to decode |
fd0150a3c2fe
revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents:
39330
diff
changeset
|
529 # base revision. this neatly avoids censored bases, which throw when |
fd0150a3c2fe
revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents:
39330
diff
changeset
|
530 # they're decoded. |
43077
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
531 hlen = struct.calcsize(b">lll") |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
532 if delta[:hlen] == mdiff.replacediffheader( |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
533 revlog.rawsize(baserev), len(delta) - hlen |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
534 ): |
39331
fd0150a3c2fe
revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents:
39330
diff
changeset
|
535 fulltext = delta[hlen:] |
fd0150a3c2fe
revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents:
39330
diff
changeset
|
536 else: |
fd0150a3c2fe
revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents:
39330
diff
changeset
|
537 # deltabase is rawtext before changed by flag processors, which is |
fd0150a3c2fe
revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents:
39330
diff
changeset
|
538 # equivalent to non-raw text |
51020
509f0f7fc89e
delta-computer: stop explicitly taking file handle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50929
diff
changeset
|
539 basetext = revlog.revision(baserev) |
39331
fd0150a3c2fe
revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents:
39330
diff
changeset
|
540 fulltext = mdiff.patch(basetext, delta) |
fd0150a3c2fe
revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents:
39330
diff
changeset
|
541 |
fd0150a3c2fe
revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents:
39330
diff
changeset
|
542 try: |
42992
dff95420480f
flagprocessors: make `processflagsraw` a module level function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42878
diff
changeset
|
543 validatehash = flagutil.processflagsraw(revlog, fulltext, flags) |
39331
fd0150a3c2fe
revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents:
39330
diff
changeset
|
544 if validatehash: |
fd0150a3c2fe
revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents:
39330
diff
changeset
|
545 revlog.checkhash(fulltext, expectednode, p1=p1, p2=p2) |
fd0150a3c2fe
revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents:
39330
diff
changeset
|
546 if flags & REVIDX_ISCENSORED: |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
547 raise error.StorageError( |
43077
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
548 _(b'node %s is not censored') % expectednode |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
549 ) |
39774
4a2466b2a434
revlog: drop some more error aliases (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39773
diff
changeset
|
550 except error.CensoredNodeError: |
39331
fd0150a3c2fe
revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents:
39330
diff
changeset
|
551 # must pass the censored index flag to add censored revisions |
fd0150a3c2fe
revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents:
39330
diff
changeset
|
552 if not flags & REVIDX_ISCENSORED: |
fd0150a3c2fe
revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents:
39330
diff
changeset
|
553 raise |
fd0150a3c2fe
revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents:
39330
diff
changeset
|
554 return fulltext |
fd0150a3c2fe
revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents:
39330
diff
changeset
|
555 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
556 |
35638
edc9330acac1
revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents:
35637
diff
changeset
|
557 @attr.s(slots=True, frozen=True) |
48946
642e31cb55f0
py3: use class X: instead of class X(object):
Gregory Szorc <gregory.szorc@gmail.com>
parents:
48875
diff
changeset
|
558 class _deltainfo: |
35638
edc9330acac1
revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents:
35637
diff
changeset
|
559 distance = attr.ib() |
edc9330acac1
revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents:
35637
diff
changeset
|
560 deltalen = attr.ib() |
edc9330acac1
revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents:
35637
diff
changeset
|
561 data = attr.ib() |
edc9330acac1
revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents:
35637
diff
changeset
|
562 base = attr.ib() |
edc9330acac1
revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents:
35637
diff
changeset
|
563 chainbase = attr.ib() |
edc9330acac1
revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents:
35637
diff
changeset
|
564 chainlen = attr.ib() |
edc9330acac1
revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents:
35637
diff
changeset
|
565 compresseddeltalen = attr.ib() |
39154
e0da43e2f71f
revlog: compute snapshot depth on delta info
Boris Feld <boris.feld@octobus.net>
parents:
39152
diff
changeset
|
566 snapshotdepth = attr.ib() |
35638
edc9330acac1
revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents:
35637
diff
changeset
|
567 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
568 |
47253
b876f0bf7366
revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46819
diff
changeset
|
569 def drop_u_compression(delta): |
b876f0bf7366
revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46819
diff
changeset
|
570 """turn into a "u" (no-compression) into no-compression without header |
b876f0bf7366
revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46819
diff
changeset
|
571 |
b876f0bf7366
revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46819
diff
changeset
|
572 This is useful for revlog format that has better compression method. |
b876f0bf7366
revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46819
diff
changeset
|
573 """ |
b876f0bf7366
revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46819
diff
changeset
|
574 assert delta.data[0] == b'u', delta.data[0] |
b876f0bf7366
revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46819
diff
changeset
|
575 return _deltainfo( |
b876f0bf7366
revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46819
diff
changeset
|
576 delta.distance, |
b876f0bf7366
revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46819
diff
changeset
|
577 delta.deltalen - 1, |
b876f0bf7366
revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46819
diff
changeset
|
578 (b'', delta.data[1]), |
b876f0bf7366
revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46819
diff
changeset
|
579 delta.base, |
b876f0bf7366
revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46819
diff
changeset
|
580 delta.chainbase, |
b876f0bf7366
revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46819
diff
changeset
|
581 delta.chainlen, |
b876f0bf7366
revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46819
diff
changeset
|
582 delta.compresseddeltalen, |
b876f0bf7366
revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46819
diff
changeset
|
583 delta.snapshotdepth, |
b876f0bf7366
revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46819
diff
changeset
|
584 ) |
b876f0bf7366
revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46819
diff
changeset
|
585 |
b876f0bf7366
revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46819
diff
changeset
|
586 |
49674
5af4a0a73e4c
find-delta: rename _isgooddeltainfo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49657
diff
changeset
|
587 def is_good_delta_info(revlog, deltainfo, revinfo): |
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
588 """Returns True if the given delta is good. Good means that it is within |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
589 the disk span, disk size, and chain length bounds that we know to be |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
590 performant.""" |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
591 if deltainfo is None: |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
592 return False |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
593 |
50698
f1b57672cb94
delta-find: remove dead code intended to deal with forced delta reuse
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50686
diff
changeset
|
594 # the DELTA_BASE_REUSE_FORCE case should have been taken care of sooner so |
f1b57672cb94
delta-find: remove dead code intended to deal with forced delta reuse
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50686
diff
changeset
|
595 # we should never end up asking such question. Adding the assert as a |
f1b57672cb94
delta-find: remove dead code intended to deal with forced delta reuse
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50686
diff
changeset
|
596 # safe-guard to detect anything that would be fishy in this regard. |
f1b57672cb94
delta-find: remove dead code intended to deal with forced delta reuse
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50686
diff
changeset
|
597 assert ( |
f1b57672cb94
delta-find: remove dead code intended to deal with forced delta reuse
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50686
diff
changeset
|
598 revinfo.cachedelta is None |
f1b57672cb94
delta-find: remove dead code intended to deal with forced delta reuse
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50686
diff
changeset
|
599 or revinfo.cachedelta[2] != DELTA_BASE_REUSE_FORCE |
51046
f636103c4d67
revlog: remove legacy usage of `_generaldelta`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51045
diff
changeset
|
600 or not revlog.delta_config.general_delta |
50698
f1b57672cb94
delta-find: remove dead code intended to deal with forced delta reuse
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50686
diff
changeset
|
601 ) |
49768
bcae90c53def
delta-find: add a delta-reuse policy that blindly accepts incoming deltas
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49681
diff
changeset
|
602 |
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
603 # - 'deltainfo.distance' is the distance from the base revision -- |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
604 # bounding it limits the amount of I/O we need to do. |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
605 # - 'deltainfo.compresseddeltalen' is the sum of the total size of |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
606 # deltas we need to apply -- bounding it limits the amount of CPU |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
607 # we consume. |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
608 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
609 textlen = revinfo.textlen |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
610 defaultmax = textlen * 4 |
51058
8bdb2478c4bc
revlog: remove legacy usage of `_maxdeltachainspan`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51055
diff
changeset
|
611 maxdist = revlog.delta_config.max_deltachain_span |
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
612 if not maxdist: |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
613 maxdist = deltainfo.distance # ensure the conditional pass |
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
614 maxdist = max(maxdist, defaultmax) |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
615 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
616 # Bad delta from read span: |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
617 # |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
618 # If the span of data read is larger than the maximum allowed. |
40603
2f7e531ef3e7
sparse-revlog: skip the span check in the sparse-revlog case
Boris Feld <boris.feld@octobus.net>
parents:
40451
diff
changeset
|
619 # |
2f7e531ef3e7
sparse-revlog: skip the span check in the sparse-revlog case
Boris Feld <boris.feld@octobus.net>
parents:
40451
diff
changeset
|
620 # In the sparse-revlog case, we rely on the associated "sparse reading" |
2f7e531ef3e7
sparse-revlog: skip the span check in the sparse-revlog case
Boris Feld <boris.feld@octobus.net>
parents:
40451
diff
changeset
|
621 # to avoid issue related to the span of data. In theory, it would be |
2f7e531ef3e7
sparse-revlog: skip the span check in the sparse-revlog case
Boris Feld <boris.feld@octobus.net>
parents:
40451
diff
changeset
|
622 # possible to build pathological revlog where delta pattern would lead |
2f7e531ef3e7
sparse-revlog: skip the span check in the sparse-revlog case
Boris Feld <boris.feld@octobus.net>
parents:
40451
diff
changeset
|
623 # to too many reads. However, they do not happen in practice at all. So |
2f7e531ef3e7
sparse-revlog: skip the span check in the sparse-revlog case
Boris Feld <boris.feld@octobus.net>
parents:
40451
diff
changeset
|
624 # we skip the span check entirely. |
51060
f71f07a679b4
revlog: remove legacy usage of `_sparserevlog`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51058
diff
changeset
|
625 if not revlog.delta_config.sparse_revlog and maxdist < deltainfo.distance: |
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
626 return False |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
627 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
628 # Bad delta from new delta size: |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
629 # |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
630 # If the delta size is larger than the target text, storing the |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
631 # delta will be inefficient. |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
632 if textlen < deltainfo.deltalen: |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
633 return False |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
634 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
635 # Bad delta from cumulated payload size: |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
636 # |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
637 # If the sum of delta get larger than K * target text length. |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
638 if textlen * LIMIT_DELTA2TEXT < deltainfo.compresseddeltalen: |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
639 return False |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
640 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
641 # Bad delta from chain length: |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
642 # |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
643 # If the number of delta in the chain gets too high. |
51052
e80e2d614c9d
revlog: remove legacy usage of `_maxchainlen`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51046
diff
changeset
|
644 if ( |
e80e2d614c9d
revlog: remove legacy usage of `_maxchainlen`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51046
diff
changeset
|
645 revlog.delta_config.max_chain_len |
e80e2d614c9d
revlog: remove legacy usage of `_maxchainlen`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51046
diff
changeset
|
646 and revlog.delta_config.max_chain_len < deltainfo.chainlen |
e80e2d614c9d
revlog: remove legacy usage of `_maxchainlen`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51046
diff
changeset
|
647 ): |
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
648 return False |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
649 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
650 # bad delta from intermediate snapshot size limit |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
651 # |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
652 # If an intermediate snapshot size is higher than the limit. The |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
653 # limit exist to prevent endless chain of intermediate delta to be |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
654 # created. |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
655 if ( |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
656 deltainfo.snapshotdepth is not None |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
657 and (textlen >> deltainfo.snapshotdepth) < deltainfo.deltalen |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
658 ): |
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
659 return False |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
660 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
661 # bad delta if new intermediate snapshot is larger than the previous |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
662 # snapshot |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
663 if ( |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
664 deltainfo.snapshotdepth |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
665 and revlog.length(deltainfo.base) < deltainfo.deltalen |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
666 ): |
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
667 return False |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
668 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
669 return True |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
670 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
671 |
40978
42f59d3f714d
delta: exclude base candidate much smaller than the target
Boris Feld <boris.feld@octobus.net>
parents:
40957
diff
changeset
|
672 # If a revision's full text is that much bigger than a base candidate full |
42f59d3f714d
delta: exclude base candidate much smaller than the target
Boris Feld <boris.feld@octobus.net>
parents:
40957
diff
changeset
|
673 # text's, it is very unlikely that it will produce a valid delta. We no longer |
42f59d3f714d
delta: exclude base candidate much smaller than the target
Boris Feld <boris.feld@octobus.net>
parents:
40957
diff
changeset
|
674 # consider these candidates. |
41033
b373477948df
revlog: limit base to rev size ratio to 500 instead of 50
Boris Feld <boris.feld@octobus.net>
parents:
40979
diff
changeset
|
675 LIMIT_BASE2TEXT = 500 |
40978
42f59d3f714d
delta: exclude base candidate much smaller than the target
Boris Feld <boris.feld@octobus.net>
parents:
40957
diff
changeset
|
676 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
677 |
49611
2afee2176775
delta-find: expand a function definition and call before extendin it
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49610
diff
changeset
|
678 def _candidategroups( |
2afee2176775
delta-find: expand a function definition and call before extendin it
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49610
diff
changeset
|
679 revlog, |
2afee2176775
delta-find: expand a function definition and call before extendin it
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49610
diff
changeset
|
680 textlen, |
2afee2176775
delta-find: expand a function definition and call before extendin it
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49610
diff
changeset
|
681 p1, |
2afee2176775
delta-find: expand a function definition and call before extendin it
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49610
diff
changeset
|
682 p2, |
2afee2176775
delta-find: expand a function definition and call before extendin it
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49610
diff
changeset
|
683 cachedelta, |
49612
e706bb41fdb3
delta-find: move pre-filtering with other pre-filtering logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49611
diff
changeset
|
684 excluded_bases=None, |
e706bb41fdb3
delta-find: move pre-filtering with other pre-filtering logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49611
diff
changeset
|
685 target_rev=None, |
49681
c261a628e525
delta-find: use a single snapshot cache when applying a group to an object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49680
diff
changeset
|
686 snapshot_cache=None, |
49611
2afee2176775
delta-find: expand a function definition and call before extendin it
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49610
diff
changeset
|
687 ): |
39336
1c6ff52fe9cf
revlogdeltas: split candidate groups selection from the filtering logic
Boris Feld <boris.feld@octobus.net>
parents:
39335
diff
changeset
|
688 """Provides group of revision to be tested as delta base |
1c6ff52fe9cf
revlogdeltas: split candidate groups selection from the filtering logic
Boris Feld <boris.feld@octobus.net>
parents:
39335
diff
changeset
|
689 |
1c6ff52fe9cf
revlogdeltas: split candidate groups selection from the filtering logic
Boris Feld <boris.feld@octobus.net>
parents:
39335
diff
changeset
|
690 This top level function focus on emitting groups with unique and worthwhile |
1c6ff52fe9cf
revlogdeltas: split candidate groups selection from the filtering logic
Boris Feld <boris.feld@octobus.net>
parents:
39335
diff
changeset
|
691 content. See _raw_candidate_groups for details about the group order. |
39334
507f5b1dd7c8
revlogdeltas: extract _getcandidaterevs in a function
Boris Feld <boris.feld@octobus.net>
parents:
39333
diff
changeset
|
692 """ |
39336
1c6ff52fe9cf
revlogdeltas: split candidate groups selection from the filtering logic
Boris Feld <boris.feld@octobus.net>
parents:
39335
diff
changeset
|
693 # should we try to build a delta? |
1c6ff52fe9cf
revlogdeltas: split candidate groups selection from the filtering logic
Boris Feld <boris.feld@octobus.net>
parents:
39335
diff
changeset
|
694 if not (len(revlog) and revlog._storedeltachains): |
39497
5b308a4e6d03
snapshot: use None as a stop value when looking for a good delta
Boris Feld <boris.feld@octobus.net>
parents:
39496
diff
changeset
|
695 yield None |
39336
1c6ff52fe9cf
revlogdeltas: split candidate groups selection from the filtering logic
Boris Feld <boris.feld@octobus.net>
parents:
39335
diff
changeset
|
696 return |
1c6ff52fe9cf
revlogdeltas: split candidate groups selection from the filtering logic
Boris Feld <boris.feld@octobus.net>
parents:
39335
diff
changeset
|
697 |
50354
ca1522fe4ec8
delta-find: assume the target-rev if not specified
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49776
diff
changeset
|
698 if target_rev is None: |
ca1522fe4ec8
delta-find: assume the target-rev if not specified
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49776
diff
changeset
|
699 target_rev = len(revlog) |
ca1522fe4ec8
delta-find: assume the target-rev if not specified
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49776
diff
changeset
|
700 |
51046
f636103c4d67
revlog: remove legacy usage of `_generaldelta`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51045
diff
changeset
|
701 if not revlog.delta_config.general_delta: |
50355
0232571255d3
delta-find: never do anything fancy when general delta is off
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50354
diff
changeset
|
702 # before general delta, there is only one possible delta base |
0232571255d3
delta-find: never do anything fancy when general delta is off
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50354
diff
changeset
|
703 yield (target_rev - 1,) |
0232571255d3
delta-find: never do anything fancy when general delta is off
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50354
diff
changeset
|
704 yield None |
0232571255d3
delta-find: never do anything fancy when general delta is off
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50354
diff
changeset
|
705 return |
0232571255d3
delta-find: never do anything fancy when general delta is off
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50354
diff
changeset
|
706 |
50698
f1b57672cb94
delta-find: remove dead code intended to deal with forced delta reuse
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50686
diff
changeset
|
707 # the DELTA_BASE_REUSE_FORCE case should have been taken care of sooner so |
f1b57672cb94
delta-find: remove dead code intended to deal with forced delta reuse
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50686
diff
changeset
|
708 # we should never end up asking such question. Adding the assert as a |
f1b57672cb94
delta-find: remove dead code intended to deal with forced delta reuse
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50686
diff
changeset
|
709 # safe-guard to detect anything that would be fishy in this regard. |
f1b57672cb94
delta-find: remove dead code intended to deal with forced delta reuse
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50686
diff
changeset
|
710 assert ( |
f1b57672cb94
delta-find: remove dead code intended to deal with forced delta reuse
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50686
diff
changeset
|
711 cachedelta is None |
f1b57672cb94
delta-find: remove dead code intended to deal with forced delta reuse
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50686
diff
changeset
|
712 or cachedelta[2] != DELTA_BASE_REUSE_FORCE |
51046
f636103c4d67
revlog: remove legacy usage of `_generaldelta`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51045
diff
changeset
|
713 or not revlog.delta_config.general_delta |
50698
f1b57672cb94
delta-find: remove dead code intended to deal with forced delta reuse
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50686
diff
changeset
|
714 ) |
49776
acdb9a15137c
bundle: when forcing acceptance of incoming delta also accept snapshot
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49775
diff
changeset
|
715 |
39337
37957e07138c
revlogdeltas: move finddeltainfo filtering inside _candidategroups
Boris Feld <boris.feld@octobus.net>
parents:
39336
diff
changeset
|
716 deltalength = revlog.length |
37957e07138c
revlogdeltas: move finddeltainfo filtering inside _candidategroups
Boris Feld <boris.feld@octobus.net>
parents:
39336
diff
changeset
|
717 deltaparent = revlog.deltaparent |
51060
f71f07a679b4
revlog: remove legacy usage of `_sparserevlog`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51058
diff
changeset
|
718 sparse = revlog.delta_config.sparse_revlog |
39498
04b75f3a3f2a
snapshot: add refining logic at the findeltainfo level
Boris Feld <boris.feld@octobus.net>
parents:
39497
diff
changeset
|
719 good = None |
39337
37957e07138c
revlogdeltas: move finddeltainfo filtering inside _candidategroups
Boris Feld <boris.feld@octobus.net>
parents:
39336
diff
changeset
|
720 |
37957e07138c
revlogdeltas: move finddeltainfo filtering inside _candidategroups
Boris Feld <boris.feld@octobus.net>
parents:
39336
diff
changeset
|
721 deltas_limit = textlen * LIMIT_DELTA2TEXT |
51055
fa7d307e2150
revlog: remove legacy usage of `_candidate_group_chunk_size`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51053
diff
changeset
|
722 group_chunk_size = revlog.delta_config.candidate_group_chunk_size |
39337
37957e07138c
revlogdeltas: move finddeltainfo filtering inside _candidategroups
Boris Feld <boris.feld@octobus.net>
parents:
39336
diff
changeset
|
723 |
42057
566daffc607d
cleanup: use set literals where possible
Martin von Zweigbergk <martinvonz@google.com>
parents:
41819
diff
changeset
|
724 tested = {nullrev} |
49610
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
725 candidates = _refinedgroups( |
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
726 revlog, |
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
727 p1, |
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
728 p2, |
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
729 cachedelta, |
49681
c261a628e525
delta-find: use a single snapshot cache when applying a group to an object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49680
diff
changeset
|
730 snapshot_cache=snapshot_cache, |
49610
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
731 ) |
39499
51cec7fb672e
snapshot: also use None as a stop value for `_refinegroup`
Boris Feld <boris.feld@octobus.net>
parents:
39498
diff
changeset
|
732 while True: |
39500
cc85ebb68ff9
snapshot: turn _refinedgroups into a coroutine
Boris Feld <boris.feld@octobus.net>
parents:
39499
diff
changeset
|
733 temptative = candidates.send(good) |
39499
51cec7fb672e
snapshot: also use None as a stop value for `_refinegroup`
Boris Feld <boris.feld@octobus.net>
parents:
39498
diff
changeset
|
734 if temptative is None: |
51cec7fb672e
snapshot: also use None as a stop value for `_refinegroup`
Boris Feld <boris.feld@octobus.net>
parents:
39498
diff
changeset
|
735 break |
39337
37957e07138c
revlogdeltas: move finddeltainfo filtering inside _candidategroups
Boris Feld <boris.feld@octobus.net>
parents:
39336
diff
changeset
|
736 group = [] |
37957e07138c
revlogdeltas: move finddeltainfo filtering inside _candidategroups
Boris Feld <boris.feld@octobus.net>
parents:
39336
diff
changeset
|
737 for rev in temptative: |
37957e07138c
revlogdeltas: move finddeltainfo filtering inside _candidategroups
Boris Feld <boris.feld@octobus.net>
parents:
39336
diff
changeset
|
738 # skip over empty delta (no need to include them in a chain) |
50357
8038493eb31a
delta-find: simply code that is now never invoqued without general delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50356
diff
changeset
|
739 while not (rev == nullrev or rev in tested or deltalength(rev)): |
39594
bdb41eaa8b59
snapshot: fix line order when skipping over empty deltas
Boris Feld <boris.feld@octobus.net>
parents:
39505
diff
changeset
|
740 tested.add(rev) |
39337
37957e07138c
revlogdeltas: move finddeltainfo filtering inside _candidategroups
Boris Feld <boris.feld@octobus.net>
parents:
39336
diff
changeset
|
741 rev = deltaparent(rev) |
40957
f960c51eebf3
delta: filter nullrev out first
Boris Feld <boris.feld@octobus.net>
parents:
40709
diff
changeset
|
742 # no need to try a delta against nullrev, this will be done as a |
f960c51eebf3
delta: filter nullrev out first
Boris Feld <boris.feld@octobus.net>
parents:
40709
diff
changeset
|
743 # last resort. |
f960c51eebf3
delta: filter nullrev out first
Boris Feld <boris.feld@octobus.net>
parents:
40709
diff
changeset
|
744 if rev == nullrev: |
f960c51eebf3
delta: filter nullrev out first
Boris Feld <boris.feld@octobus.net>
parents:
40709
diff
changeset
|
745 continue |
39337
37957e07138c
revlogdeltas: move finddeltainfo filtering inside _candidategroups
Boris Feld <boris.feld@octobus.net>
parents:
39336
diff
changeset
|
746 # filter out revision we tested already |
37957e07138c
revlogdeltas: move finddeltainfo filtering inside _candidategroups
Boris Feld <boris.feld@octobus.net>
parents:
39336
diff
changeset
|
747 if rev in tested: |
37957e07138c
revlogdeltas: move finddeltainfo filtering inside _candidategroups
Boris Feld <boris.feld@octobus.net>
parents:
39336
diff
changeset
|
748 continue |
49768
bcae90c53def
delta-find: add a delta-reuse policy that blindly accepts incoming deltas
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49681
diff
changeset
|
749 |
49612
e706bb41fdb3
delta-find: move pre-filtering with other pre-filtering logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49611
diff
changeset
|
750 # an higher authority deamed the base unworthy (e.g. censored) |
e706bb41fdb3
delta-find: move pre-filtering with other pre-filtering logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49611
diff
changeset
|
751 if excluded_bases is not None and rev in excluded_bases: |
49615
4956942c0416
delta-find: adjust the moment when we mark something as "tested"
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49614
diff
changeset
|
752 tested.add(rev) |
49612
e706bb41fdb3
delta-find: move pre-filtering with other pre-filtering logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49611
diff
changeset
|
753 continue |
e706bb41fdb3
delta-find: move pre-filtering with other pre-filtering logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49611
diff
changeset
|
754 # We are in some recomputation cases and that rev is too high in |
e706bb41fdb3
delta-find: move pre-filtering with other pre-filtering logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49611
diff
changeset
|
755 # the revlog |
e706bb41fdb3
delta-find: move pre-filtering with other pre-filtering logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49611
diff
changeset
|
756 if target_rev is not None and rev >= target_rev: |
49615
4956942c0416
delta-find: adjust the moment when we mark something as "tested"
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49614
diff
changeset
|
757 tested.add(rev) |
49612
e706bb41fdb3
delta-find: move pre-filtering with other pre-filtering logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49611
diff
changeset
|
758 continue |
39337
37957e07138c
revlogdeltas: move finddeltainfo filtering inside _candidategroups
Boris Feld <boris.feld@octobus.net>
parents:
39336
diff
changeset
|
759 # filter out delta base that will never produce good delta |
37957e07138c
revlogdeltas: move finddeltainfo filtering inside _candidategroups
Boris Feld <boris.feld@octobus.net>
parents:
39336
diff
changeset
|
760 if deltas_limit < revlog.length(rev): |
49615
4956942c0416
delta-find: adjust the moment when we mark something as "tested"
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49614
diff
changeset
|
761 tested.add(rev) |
39337
37957e07138c
revlogdeltas: move finddeltainfo filtering inside _candidategroups
Boris Feld <boris.feld@octobus.net>
parents:
39336
diff
changeset
|
762 continue |
40978
42f59d3f714d
delta: exclude base candidate much smaller than the target
Boris Feld <boris.feld@octobus.net>
parents:
40957
diff
changeset
|
763 if sparse and revlog.rawsize(rev) < (textlen // LIMIT_BASE2TEXT): |
49615
4956942c0416
delta-find: adjust the moment when we mark something as "tested"
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49614
diff
changeset
|
764 tested.add(rev) |
40978
42f59d3f714d
delta: exclude base candidate much smaller than the target
Boris Feld <boris.feld@octobus.net>
parents:
40957
diff
changeset
|
765 continue |
39337
37957e07138c
revlogdeltas: move finddeltainfo filtering inside _candidategroups
Boris Feld <boris.feld@octobus.net>
parents:
39336
diff
changeset
|
766 # no delta for rawtext-changing revs (see "candelta" for why) |
37957e07138c
revlogdeltas: move finddeltainfo filtering inside _candidategroups
Boris Feld <boris.feld@octobus.net>
parents:
39336
diff
changeset
|
767 if revlog.flags(rev) & REVIDX_RAWTEXT_CHANGING_FLAGS: |
49615
4956942c0416
delta-find: adjust the moment when we mark something as "tested"
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49614
diff
changeset
|
768 tested.add(rev) |
39337
37957e07138c
revlogdeltas: move finddeltainfo filtering inside _candidategroups
Boris Feld <boris.feld@octobus.net>
parents:
39336
diff
changeset
|
769 continue |
49610
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
770 |
40979
ba09db267cb6
delta: ignore base whose chains already don't match expectations
Boris Feld <boris.feld@octobus.net>
parents:
40978
diff
changeset
|
771 # If we reach here, we are about to build and test a delta. |
ba09db267cb6
delta: ignore base whose chains already don't match expectations
Boris Feld <boris.feld@octobus.net>
parents:
40978
diff
changeset
|
772 # The delta building process will compute the chaininfo in all |
ba09db267cb6
delta: ignore base whose chains already don't match expectations
Boris Feld <boris.feld@octobus.net>
parents:
40978
diff
changeset
|
773 # case, since that computation is cached, it is fine to access it |
ba09db267cb6
delta: ignore base whose chains already don't match expectations
Boris Feld <boris.feld@octobus.net>
parents:
40978
diff
changeset
|
774 # here too. |
ba09db267cb6
delta: ignore base whose chains already don't match expectations
Boris Feld <boris.feld@octobus.net>
parents:
40978
diff
changeset
|
775 chainlen, chainsize = revlog._chaininfo(rev) |
ba09db267cb6
delta: ignore base whose chains already don't match expectations
Boris Feld <boris.feld@octobus.net>
parents:
40978
diff
changeset
|
776 # if chain will be too long, skip base |
51052
e80e2d614c9d
revlog: remove legacy usage of `_maxchainlen`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51046
diff
changeset
|
777 if ( |
e80e2d614c9d
revlog: remove legacy usage of `_maxchainlen`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51046
diff
changeset
|
778 revlog.delta_config.max_chain_len |
e80e2d614c9d
revlog: remove legacy usage of `_maxchainlen`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51046
diff
changeset
|
779 and chainlen >= revlog.delta_config.max_chain_len |
e80e2d614c9d
revlog: remove legacy usage of `_maxchainlen`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51046
diff
changeset
|
780 ): |
49615
4956942c0416
delta-find: adjust the moment when we mark something as "tested"
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49614
diff
changeset
|
781 tested.add(rev) |
40979
ba09db267cb6
delta: ignore base whose chains already don't match expectations
Boris Feld <boris.feld@octobus.net>
parents:
40978
diff
changeset
|
782 continue |
ba09db267cb6
delta: ignore base whose chains already don't match expectations
Boris Feld <boris.feld@octobus.net>
parents:
40978
diff
changeset
|
783 # if chain already have too much data, skip base |
ba09db267cb6
delta: ignore base whose chains already don't match expectations
Boris Feld <boris.feld@octobus.net>
parents:
40978
diff
changeset
|
784 if deltas_limit < chainsize: |
49615
4956942c0416
delta-find: adjust the moment when we mark something as "tested"
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49614
diff
changeset
|
785 tested.add(rev) |
40979
ba09db267cb6
delta: ignore base whose chains already don't match expectations
Boris Feld <boris.feld@octobus.net>
parents:
40978
diff
changeset
|
786 continue |
51083
5645524c6b62
revlog: also migrates `revlog.upperboundcomp` to ConfigClass
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51067
diff
changeset
|
787 if sparse and revlog.delta_config.upper_bound_comp is not None: |
5645524c6b62
revlog: also migrates `revlog.upperboundcomp` to ConfigClass
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51067
diff
changeset
|
788 maxcomp = revlog.delta_config.upper_bound_comp |
42463
a0b26fc8fbba
deltas: skip if projected delta size does not match text size constraint
Valentin Gatien-Baron <vgatien-baron@janestreet.com>, Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42057
diff
changeset
|
789 basenotsnap = (p1, p2, nullrev) |
a0b26fc8fbba
deltas: skip if projected delta size does not match text size constraint
Valentin Gatien-Baron <vgatien-baron@janestreet.com>, Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42057
diff
changeset
|
790 if rev not in basenotsnap and revlog.issnapshot(rev): |
a0b26fc8fbba
deltas: skip if projected delta size does not match text size constraint
Valentin Gatien-Baron <vgatien-baron@janestreet.com>, Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42057
diff
changeset
|
791 snapshotdepth = revlog.snapshotdepth(rev) |
a0b26fc8fbba
deltas: skip if projected delta size does not match text size constraint
Valentin Gatien-Baron <vgatien-baron@janestreet.com>, Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42057
diff
changeset
|
792 # If text is significantly larger than the base, we can |
a0b26fc8fbba
deltas: skip if projected delta size does not match text size constraint
Valentin Gatien-Baron <vgatien-baron@janestreet.com>, Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42057
diff
changeset
|
793 # expect the resulting delta to be proportional to the size |
a0b26fc8fbba
deltas: skip if projected delta size does not match text size constraint
Valentin Gatien-Baron <vgatien-baron@janestreet.com>, Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42057
diff
changeset
|
794 # difference |
a0b26fc8fbba
deltas: skip if projected delta size does not match text size constraint
Valentin Gatien-Baron <vgatien-baron@janestreet.com>, Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42057
diff
changeset
|
795 revsize = revlog.rawsize(rev) |
a0b26fc8fbba
deltas: skip if projected delta size does not match text size constraint
Valentin Gatien-Baron <vgatien-baron@janestreet.com>, Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42057
diff
changeset
|
796 rawsizedistance = max(textlen - revsize, 0) |
a0b26fc8fbba
deltas: skip if projected delta size does not match text size constraint
Valentin Gatien-Baron <vgatien-baron@janestreet.com>, Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42057
diff
changeset
|
797 # use an estimate of the compression upper bound. |
a0b26fc8fbba
deltas: skip if projected delta size does not match text size constraint
Valentin Gatien-Baron <vgatien-baron@janestreet.com>, Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42057
diff
changeset
|
798 lowestrealisticdeltalen = rawsizedistance // maxcomp |
a0b26fc8fbba
deltas: skip if projected delta size does not match text size constraint
Valentin Gatien-Baron <vgatien-baron@janestreet.com>, Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42057
diff
changeset
|
799 |
a0b26fc8fbba
deltas: skip if projected delta size does not match text size constraint
Valentin Gatien-Baron <vgatien-baron@janestreet.com>, Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42057
diff
changeset
|
800 # check the absolute constraint on the delta size |
a0b26fc8fbba
deltas: skip if projected delta size does not match text size constraint
Valentin Gatien-Baron <vgatien-baron@janestreet.com>, Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42057
diff
changeset
|
801 snapshotlimit = textlen >> snapshotdepth |
a0b26fc8fbba
deltas: skip if projected delta size does not match text size constraint
Valentin Gatien-Baron <vgatien-baron@janestreet.com>, Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42057
diff
changeset
|
802 if snapshotlimit < lowestrealisticdeltalen: |
a0b26fc8fbba
deltas: skip if projected delta size does not match text size constraint
Valentin Gatien-Baron <vgatien-baron@janestreet.com>, Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42057
diff
changeset
|
803 # delta lower bound is larger than accepted upper bound |
49615
4956942c0416
delta-find: adjust the moment when we mark something as "tested"
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49614
diff
changeset
|
804 tested.add(rev) |
42463
a0b26fc8fbba
deltas: skip if projected delta size does not match text size constraint
Valentin Gatien-Baron <vgatien-baron@janestreet.com>, Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42057
diff
changeset
|
805 continue |
a0b26fc8fbba
deltas: skip if projected delta size does not match text size constraint
Valentin Gatien-Baron <vgatien-baron@janestreet.com>, Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42057
diff
changeset
|
806 |
42464
66c27df1be84
deltas: skip if projected delta size is bigger than previous snapshot
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42463
diff
changeset
|
807 # check the relative constraint on the delta size |
66c27df1be84
deltas: skip if projected delta size is bigger than previous snapshot
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42463
diff
changeset
|
808 revlength = revlog.length(rev) |
66c27df1be84
deltas: skip if projected delta size is bigger than previous snapshot
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42463
diff
changeset
|
809 if revlength < lowestrealisticdeltalen: |
66c27df1be84
deltas: skip if projected delta size is bigger than previous snapshot
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42463
diff
changeset
|
810 # delta probable lower bound is larger than target base |
49615
4956942c0416
delta-find: adjust the moment when we mark something as "tested"
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49614
diff
changeset
|
811 tested.add(rev) |
42464
66c27df1be84
deltas: skip if projected delta size is bigger than previous snapshot
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42463
diff
changeset
|
812 continue |
66c27df1be84
deltas: skip if projected delta size is bigger than previous snapshot
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42463
diff
changeset
|
813 |
39337
37957e07138c
revlogdeltas: move finddeltainfo filtering inside _candidategroups
Boris Feld <boris.feld@octobus.net>
parents:
39336
diff
changeset
|
814 group.append(rev) |
39336
1c6ff52fe9cf
revlogdeltas: split candidate groups selection from the filtering logic
Boris Feld <boris.feld@octobus.net>
parents:
39335
diff
changeset
|
815 if group: |
49657
f5f113f1b011
delta-find: add a way to control the number of bases tested at the same time
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49615
diff
changeset
|
816 # When the size of the candidate group is big, it can result in a |
f5f113f1b011
delta-find: add a way to control the number of bases tested at the same time
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49615
diff
changeset
|
817 # quite significant performance impact. To reduce this, we can send |
f5f113f1b011
delta-find: add a way to control the number of bases tested at the same time
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49615
diff
changeset
|
818 # them in smaller batches until the new batch does not provide any |
f5f113f1b011
delta-find: add a way to control the number of bases tested at the same time
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49615
diff
changeset
|
819 # improvements. |
f5f113f1b011
delta-find: add a way to control the number of bases tested at the same time
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49615
diff
changeset
|
820 # |
f5f113f1b011
delta-find: add a way to control the number of bases tested at the same time
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49615
diff
changeset
|
821 # This might reduce the overall efficiency of the compression in |
f5f113f1b011
delta-find: add a way to control the number of bases tested at the same time
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49615
diff
changeset
|
822 # some corner cases, but that should also prevent very pathological |
f5f113f1b011
delta-find: add a way to control the number of bases tested at the same time
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49615
diff
changeset
|
823 # cases from being an issue. (eg. 20 000 candidates). |
f5f113f1b011
delta-find: add a way to control the number of bases tested at the same time
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49615
diff
changeset
|
824 # |
f5f113f1b011
delta-find: add a way to control the number of bases tested at the same time
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49615
diff
changeset
|
825 # XXX note that the ordering of the group becomes important as it |
f5f113f1b011
delta-find: add a way to control the number of bases tested at the same time
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49615
diff
changeset
|
826 # now impacts the final result. The current order is unprocessed |
f5f113f1b011
delta-find: add a way to control the number of bases tested at the same time
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49615
diff
changeset
|
827 # and can be improved. |
f5f113f1b011
delta-find: add a way to control the number of bases tested at the same time
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49615
diff
changeset
|
828 if group_chunk_size == 0: |
f5f113f1b011
delta-find: add a way to control the number of bases tested at the same time
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49615
diff
changeset
|
829 tested.update(group) |
f5f113f1b011
delta-find: add a way to control the number of bases tested at the same time
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49615
diff
changeset
|
830 good = yield tuple(group) |
f5f113f1b011
delta-find: add a way to control the number of bases tested at the same time
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49615
diff
changeset
|
831 else: |
f5f113f1b011
delta-find: add a way to control the number of bases tested at the same time
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49615
diff
changeset
|
832 prev_good = good |
f5f113f1b011
delta-find: add a way to control the number of bases tested at the same time
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49615
diff
changeset
|
833 for start in range(0, len(group), group_chunk_size): |
f5f113f1b011
delta-find: add a way to control the number of bases tested at the same time
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49615
diff
changeset
|
834 sub_group = group[start : start + group_chunk_size] |
f5f113f1b011
delta-find: add a way to control the number of bases tested at the same time
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49615
diff
changeset
|
835 tested.update(sub_group) |
f5f113f1b011
delta-find: add a way to control the number of bases tested at the same time
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49615
diff
changeset
|
836 good = yield tuple(sub_group) |
f5f113f1b011
delta-find: add a way to control the number of bases tested at the same time
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49615
diff
changeset
|
837 if prev_good == good: |
f5f113f1b011
delta-find: add a way to control the number of bases tested at the same time
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49615
diff
changeset
|
838 break |
f5f113f1b011
delta-find: add a way to control the number of bases tested at the same time
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49615
diff
changeset
|
839 |
39497
5b308a4e6d03
snapshot: use None as a stop value when looking for a good delta
Boris Feld <boris.feld@octobus.net>
parents:
39496
diff
changeset
|
840 yield None |
39336
1c6ff52fe9cf
revlogdeltas: split candidate groups selection from the filtering logic
Boris Feld <boris.feld@octobus.net>
parents:
39335
diff
changeset
|
841 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
842 |
49681
c261a628e525
delta-find: use a single snapshot cache when applying a group to an object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49680
diff
changeset
|
843 def _refinedgroups(revlog, p1, p2, cachedelta, snapshot_cache=None): |
39496
2f9f7889549b
snapshot: introduce an intermediate `_refinedgroups` generator
Boris Feld <boris.feld@octobus.net>
parents:
39495
diff
changeset
|
844 good = None |
39501
993d7e2c8b79
snapshot: make sure we'll never refine delta base from a reused source
Boris Feld <boris.feld@octobus.net>
parents:
39500
diff
changeset
|
845 # First we try to reuse a the delta contained in the bundle. |
993d7e2c8b79
snapshot: make sure we'll never refine delta base from a reused source
Boris Feld <boris.feld@octobus.net>
parents:
39500
diff
changeset
|
846 # (or from the source revlog) |
993d7e2c8b79
snapshot: make sure we'll never refine delta base from a reused source
Boris Feld <boris.feld@octobus.net>
parents:
39500
diff
changeset
|
847 # |
993d7e2c8b79
snapshot: make sure we'll never refine delta base from a reused source
Boris Feld <boris.feld@octobus.net>
parents:
39500
diff
changeset
|
848 # This logic only applies to general delta repositories and can be disabled |
993d7e2c8b79
snapshot: make sure we'll never refine delta base from a reused source
Boris Feld <boris.feld@octobus.net>
parents:
39500
diff
changeset
|
849 # through configuration. Disabling reuse source delta is useful when |
993d7e2c8b79
snapshot: make sure we'll never refine delta base from a reused source
Boris Feld <boris.feld@octobus.net>
parents:
39500
diff
changeset
|
850 # we want to make sure we recomputed "optimal" deltas. |
49610
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
851 debug_info = None |
49677
05db41701ece
find-delta: pass the cache-delta usage policy alongside the cache-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49675
diff
changeset
|
852 if cachedelta is not None and cachedelta[2] > DELTA_BASE_REUSE_NO: |
39501
993d7e2c8b79
snapshot: make sure we'll never refine delta base from a reused source
Boris Feld <boris.feld@octobus.net>
parents:
39500
diff
changeset
|
853 # Assume what we received from the server is a good choice |
993d7e2c8b79
snapshot: make sure we'll never refine delta base from a reused source
Boris Feld <boris.feld@octobus.net>
parents:
39500
diff
changeset
|
854 # build delta will reuse the cache |
49610
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
855 if debug_info is not None: |
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
856 debug_info['cached-delta.tested'] += 1 |
39501
993d7e2c8b79
snapshot: make sure we'll never refine delta base from a reused source
Boris Feld <boris.feld@octobus.net>
parents:
39500
diff
changeset
|
857 good = yield (cachedelta[0],) |
993d7e2c8b79
snapshot: make sure we'll never refine delta base from a reused source
Boris Feld <boris.feld@octobus.net>
parents:
39500
diff
changeset
|
858 if good is not None: |
49610
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
859 if debug_info is not None: |
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
860 debug_info['cached-delta.accepted'] += 1 |
39501
993d7e2c8b79
snapshot: make sure we'll never refine delta base from a reused source
Boris Feld <boris.feld@octobus.net>
parents:
39500
diff
changeset
|
861 yield None |
993d7e2c8b79
snapshot: make sure we'll never refine delta base from a reused source
Boris Feld <boris.feld@octobus.net>
parents:
39500
diff
changeset
|
862 return |
49681
c261a628e525
delta-find: use a single snapshot cache when applying a group to an object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49680
diff
changeset
|
863 if snapshot_cache is None: |
c261a628e525
delta-find: use a single snapshot cache when applying a group to an object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49680
diff
changeset
|
864 snapshot_cache = SnapshotCache() |
49610
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
865 groups = _rawgroups( |
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
866 revlog, |
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
867 p1, |
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
868 p2, |
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
869 cachedelta, |
49678
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
870 snapshot_cache, |
49610
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
871 ) |
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
872 for candidates in groups: |
39496
2f9f7889549b
snapshot: introduce an intermediate `_refinedgroups` generator
Boris Feld <boris.feld@octobus.net>
parents:
39495
diff
changeset
|
873 good = yield candidates |
2f9f7889549b
snapshot: introduce an intermediate `_refinedgroups` generator
Boris Feld <boris.feld@octobus.net>
parents:
39495
diff
changeset
|
874 if good is not None: |
2f9f7889549b
snapshot: introduce an intermediate `_refinedgroups` generator
Boris Feld <boris.feld@octobus.net>
parents:
39495
diff
changeset
|
875 break |
39502
e4d4361d0bcd
snapshot: try to refine new snapshot base down the chain
Boris Feld <boris.feld@octobus.net>
parents:
39501
diff
changeset
|
876 |
40428
bafa1c4bb7a8
sparse-revlog: only refine delta candidates in the sparse case (issue6006)
Boris Feld <boris.feld@octobus.net>
parents:
39777
diff
changeset
|
877 # If sparse revlog is enabled, we can try to refine the available deltas |
51060
f71f07a679b4
revlog: remove legacy usage of `_sparserevlog`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51058
diff
changeset
|
878 if not revlog.delta_config.sparse_revlog: |
40428
bafa1c4bb7a8
sparse-revlog: only refine delta candidates in the sparse case (issue6006)
Boris Feld <boris.feld@octobus.net>
parents:
39777
diff
changeset
|
879 yield None |
bafa1c4bb7a8
sparse-revlog: only refine delta candidates in the sparse case (issue6006)
Boris Feld <boris.feld@octobus.net>
parents:
39777
diff
changeset
|
880 return |
bafa1c4bb7a8
sparse-revlog: only refine delta candidates in the sparse case (issue6006)
Boris Feld <boris.feld@octobus.net>
parents:
39777
diff
changeset
|
881 |
39502
e4d4361d0bcd
snapshot: try to refine new snapshot base down the chain
Boris Feld <boris.feld@octobus.net>
parents:
39501
diff
changeset
|
882 # if we have a refinable value, try to refine it |
e4d4361d0bcd
snapshot: try to refine new snapshot base down the chain
Boris Feld <boris.feld@octobus.net>
parents:
39501
diff
changeset
|
883 if good is not None and good not in (p1, p2) and revlog.issnapshot(good): |
e4d4361d0bcd
snapshot: try to refine new snapshot base down the chain
Boris Feld <boris.feld@octobus.net>
parents:
39501
diff
changeset
|
884 # refine snapshot down |
e4d4361d0bcd
snapshot: try to refine new snapshot base down the chain
Boris Feld <boris.feld@octobus.net>
parents:
39501
diff
changeset
|
885 previous = None |
e4d4361d0bcd
snapshot: try to refine new snapshot base down the chain
Boris Feld <boris.feld@octobus.net>
parents:
39501
diff
changeset
|
886 while previous != good: |
e4d4361d0bcd
snapshot: try to refine new snapshot base down the chain
Boris Feld <boris.feld@octobus.net>
parents:
39501
diff
changeset
|
887 previous = good |
e4d4361d0bcd
snapshot: try to refine new snapshot base down the chain
Boris Feld <boris.feld@octobus.net>
parents:
39501
diff
changeset
|
888 base = revlog.deltaparent(good) |
e4d4361d0bcd
snapshot: try to refine new snapshot base down the chain
Boris Feld <boris.feld@octobus.net>
parents:
39501
diff
changeset
|
889 if base == nullrev: |
e4d4361d0bcd
snapshot: try to refine new snapshot base down the chain
Boris Feld <boris.feld@octobus.net>
parents:
39501
diff
changeset
|
890 break |
e4d4361d0bcd
snapshot: try to refine new snapshot base down the chain
Boris Feld <boris.feld@octobus.net>
parents:
39501
diff
changeset
|
891 good = yield (base,) |
39503
5aef5afa8654
snapshot: refine candidate snapshot base upward
Boris Feld <boris.feld@octobus.net>
parents:
39502
diff
changeset
|
892 # refine snapshot up |
49678
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
893 if not snapshot_cache.snapshots: |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
894 snapshot_cache.update(revlog, good + 1) |
39503
5aef5afa8654
snapshot: refine candidate snapshot base upward
Boris Feld <boris.feld@octobus.net>
parents:
39502
diff
changeset
|
895 previous = None |
5aef5afa8654
snapshot: refine candidate snapshot base upward
Boris Feld <boris.feld@octobus.net>
parents:
39502
diff
changeset
|
896 while good != previous: |
5aef5afa8654
snapshot: refine candidate snapshot base upward
Boris Feld <boris.feld@octobus.net>
parents:
39502
diff
changeset
|
897 previous = good |
49678
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
898 children = tuple(sorted(c for c in snapshot_cache.snapshots[good])) |
39503
5aef5afa8654
snapshot: refine candidate snapshot base upward
Boris Feld <boris.feld@octobus.net>
parents:
39502
diff
changeset
|
899 good = yield children |
5aef5afa8654
snapshot: refine candidate snapshot base upward
Boris Feld <boris.feld@octobus.net>
parents:
39502
diff
changeset
|
900 |
49610
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
901 if debug_info is not None: |
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
902 if good is None: |
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
903 debug_info['no-solution'] += 1 |
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
904 |
39499
51cec7fb672e
snapshot: also use None as a stop value for `_refinegroup`
Boris Feld <boris.feld@octobus.net>
parents:
39498
diff
changeset
|
905 yield None |
39496
2f9f7889549b
snapshot: introduce an intermediate `_refinedgroups` generator
Boris Feld <boris.feld@octobus.net>
parents:
39495
diff
changeset
|
906 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
907 |
49678
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
908 def _rawgroups(revlog, p1, p2, cachedelta, snapshot_cache=None): |
39336
1c6ff52fe9cf
revlogdeltas: split candidate groups selection from the filtering logic
Boris Feld <boris.feld@octobus.net>
parents:
39335
diff
changeset
|
909 """Provides group of revision to be tested as delta base |
1c6ff52fe9cf
revlogdeltas: split candidate groups selection from the filtering logic
Boris Feld <boris.feld@octobus.net>
parents:
39335
diff
changeset
|
910 |
1c6ff52fe9cf
revlogdeltas: split candidate groups selection from the filtering logic
Boris Feld <boris.feld@octobus.net>
parents:
39335
diff
changeset
|
911 This lower level function focus on emitting delta theorically interresting |
1c6ff52fe9cf
revlogdeltas: split candidate groups selection from the filtering logic
Boris Feld <boris.feld@octobus.net>
parents:
39335
diff
changeset
|
912 without looking it any practical details. |
1c6ff52fe9cf
revlogdeltas: split candidate groups selection from the filtering logic
Boris Feld <boris.feld@octobus.net>
parents:
39335
diff
changeset
|
913 |
1c6ff52fe9cf
revlogdeltas: split candidate groups selection from the filtering logic
Boris Feld <boris.feld@octobus.net>
parents:
39335
diff
changeset
|
914 The group order aims at providing fast or small candidates first. |
39334
507f5b1dd7c8
revlogdeltas: extract _getcandidaterevs in a function
Boris Feld <boris.feld@octobus.net>
parents:
39333
diff
changeset
|
915 """ |
50357
8038493eb31a
delta-find: simply code that is now never invoqued without general delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50356
diff
changeset
|
916 # Why search for delta base if we cannot use a delta base ? |
51046
f636103c4d67
revlog: remove legacy usage of `_generaldelta`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51045
diff
changeset
|
917 assert revlog.delta_config.general_delta |
50357
8038493eb31a
delta-find: simply code that is now never invoqued without general delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50356
diff
changeset
|
918 # also see issue6056 |
51060
f71f07a679b4
revlog: remove legacy usage of `_sparserevlog`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51058
diff
changeset
|
919 sparse = revlog.delta_config.sparse_revlog |
39334
507f5b1dd7c8
revlogdeltas: extract _getcandidaterevs in a function
Boris Feld <boris.feld@octobus.net>
parents:
39333
diff
changeset
|
920 curr = len(revlog) |
507f5b1dd7c8
revlogdeltas: extract _getcandidaterevs in a function
Boris Feld <boris.feld@octobus.net>
parents:
39333
diff
changeset
|
921 prev = curr - 1 |
39492
a33f394b2bfd
snapshot: try intermediate snapshot against parents' base
Boris Feld <boris.feld@octobus.net>
parents:
39488
diff
changeset
|
922 deltachain = lambda rev: revlog._deltachain(rev)[0] |
39334
507f5b1dd7c8
revlogdeltas: extract _getcandidaterevs in a function
Boris Feld <boris.feld@octobus.net>
parents:
39333
diff
changeset
|
923 |
50357
8038493eb31a
delta-find: simply code that is now never invoqued without general delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50356
diff
changeset
|
924 # exclude already lazy tested base if any |
8038493eb31a
delta-find: simply code that is now never invoqued without general delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50356
diff
changeset
|
925 parents = [p for p in (p1, p2) if p != nullrev] |
39334
507f5b1dd7c8
revlogdeltas: extract _getcandidaterevs in a function
Boris Feld <boris.feld@octobus.net>
parents:
39333
diff
changeset
|
926 |
51053
96f521df2508
revlog: remove legacy usage of `_deltabothparents`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51052
diff
changeset
|
927 if not revlog.delta_config.delta_both_parents and len(parents) == 2: |
50357
8038493eb31a
delta-find: simply code that is now never invoqued without general delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50356
diff
changeset
|
928 parents.sort() |
8038493eb31a
delta-find: simply code that is now never invoqued without general delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50356
diff
changeset
|
929 # To minimize the chance of having to build a fulltext, |
8038493eb31a
delta-find: simply code that is now never invoqued without general delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50356
diff
changeset
|
930 # pick first whichever parent is closest to us (max rev) |
8038493eb31a
delta-find: simply code that is now never invoqued without general delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50356
diff
changeset
|
931 yield (parents[1],) |
8038493eb31a
delta-find: simply code that is now never invoqued without general delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50356
diff
changeset
|
932 # then the other one (min rev) if the first did not fit |
8038493eb31a
delta-find: simply code that is now never invoqued without general delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50356
diff
changeset
|
933 yield (parents[0],) |
8038493eb31a
delta-find: simply code that is now never invoqued without general delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50356
diff
changeset
|
934 elif len(parents) > 0: |
8038493eb31a
delta-find: simply code that is now never invoqued without general delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50356
diff
changeset
|
935 # Test all parents (1 or 2), and keep the best candidate |
8038493eb31a
delta-find: simply code that is now never invoqued without general delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50356
diff
changeset
|
936 yield parents |
39334
507f5b1dd7c8
revlogdeltas: extract _getcandidaterevs in a function
Boris Feld <boris.feld@octobus.net>
parents:
39333
diff
changeset
|
937 |
39492
a33f394b2bfd
snapshot: try intermediate snapshot against parents' base
Boris Feld <boris.feld@octobus.net>
parents:
39488
diff
changeset
|
938 if sparse and parents: |
49678
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
939 if snapshot_cache is None: |
49613
5447c1507c86
delta-find: small documentation update
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49612
diff
changeset
|
940 # map: base-rev: [snapshot-revs] |
49678
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
941 snapshot_cache = SnapshotCache() |
39492
a33f394b2bfd
snapshot: try intermediate snapshot against parents' base
Boris Feld <boris.feld@octobus.net>
parents:
39488
diff
changeset
|
942 # See if we can use an existing snapshot in the parent chains to use as |
a33f394b2bfd
snapshot: try intermediate snapshot against parents' base
Boris Feld <boris.feld@octobus.net>
parents:
39488
diff
changeset
|
943 # a base for a new intermediate-snapshot |
39494
e72130f58f5d
snapshot: consider all snapshots in the parents' chains
Boris Feld <boris.feld@octobus.net>
parents:
39493
diff
changeset
|
944 # |
e72130f58f5d
snapshot: consider all snapshots in the parents' chains
Boris Feld <boris.feld@octobus.net>
parents:
39493
diff
changeset
|
945 # search for snapshot in parents delta chain |
e72130f58f5d
snapshot: consider all snapshots in the parents' chains
Boris Feld <boris.feld@octobus.net>
parents:
39493
diff
changeset
|
946 # map: snapshot-level: snapshot-rev |
e72130f58f5d
snapshot: consider all snapshots in the parents' chains
Boris Feld <boris.feld@octobus.net>
parents:
39493
diff
changeset
|
947 parents_snaps = collections.defaultdict(set) |
39504
05a165dc4f55
snapshot: extract parent chain computation
Boris Feld <boris.feld@octobus.net>
parents:
39503
diff
changeset
|
948 candidate_chains = [deltachain(p) for p in parents] |
05a165dc4f55
snapshot: extract parent chain computation
Boris Feld <boris.feld@octobus.net>
parents:
39503
diff
changeset
|
949 for chain in candidate_chains: |
05a165dc4f55
snapshot: extract parent chain computation
Boris Feld <boris.feld@octobus.net>
parents:
39503
diff
changeset
|
950 for idx, s in enumerate(chain): |
39494
e72130f58f5d
snapshot: consider all snapshots in the parents' chains
Boris Feld <boris.feld@octobus.net>
parents:
39493
diff
changeset
|
951 if not revlog.issnapshot(s): |
e72130f58f5d
snapshot: consider all snapshots in the parents' chains
Boris Feld <boris.feld@octobus.net>
parents:
39493
diff
changeset
|
952 break |
e72130f58f5d
snapshot: consider all snapshots in the parents' chains
Boris Feld <boris.feld@octobus.net>
parents:
39493
diff
changeset
|
953 parents_snaps[idx].add(s) |
39495
6a53842727c1
snapshot: consider unrelated snapshots at a similar level first
Boris Feld <boris.feld@octobus.net>
parents:
39494
diff
changeset
|
954 snapfloor = min(parents_snaps[0]) + 1 |
49678
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
955 snapshot_cache.update(revlog, snapfloor) |
39505
c6b8eab5db19
snapshot: also consider the snapshot chain of one unrelated revision
Boris Feld <boris.feld@octobus.net>
parents:
39504
diff
changeset
|
956 # search for the highest "unrelated" revision |
c6b8eab5db19
snapshot: also consider the snapshot chain of one unrelated revision
Boris Feld <boris.feld@octobus.net>
parents:
39504
diff
changeset
|
957 # |
c6b8eab5db19
snapshot: also consider the snapshot chain of one unrelated revision
Boris Feld <boris.feld@octobus.net>
parents:
39504
diff
changeset
|
958 # Adding snapshots used by "unrelated" revision increase the odd we |
c6b8eab5db19
snapshot: also consider the snapshot chain of one unrelated revision
Boris Feld <boris.feld@octobus.net>
parents:
39504
diff
changeset
|
959 # reuse an independant, yet better snapshot chain. |
c6b8eab5db19
snapshot: also consider the snapshot chain of one unrelated revision
Boris Feld <boris.feld@octobus.net>
parents:
39504
diff
changeset
|
960 # |
c6b8eab5db19
snapshot: also consider the snapshot chain of one unrelated revision
Boris Feld <boris.feld@octobus.net>
parents:
39504
diff
changeset
|
961 # XXX instead of building a set of revisions, we could lazily enumerate |
c6b8eab5db19
snapshot: also consider the snapshot chain of one unrelated revision
Boris Feld <boris.feld@octobus.net>
parents:
39504
diff
changeset
|
962 # over the chains. That would be more efficient, however we stick to |
c6b8eab5db19
snapshot: also consider the snapshot chain of one unrelated revision
Boris Feld <boris.feld@octobus.net>
parents:
39504
diff
changeset
|
963 # simple code for now. |
c6b8eab5db19
snapshot: also consider the snapshot chain of one unrelated revision
Boris Feld <boris.feld@octobus.net>
parents:
39504
diff
changeset
|
964 all_revs = set() |
c6b8eab5db19
snapshot: also consider the snapshot chain of one unrelated revision
Boris Feld <boris.feld@octobus.net>
parents:
39504
diff
changeset
|
965 for chain in candidate_chains: |
c6b8eab5db19
snapshot: also consider the snapshot chain of one unrelated revision
Boris Feld <boris.feld@octobus.net>
parents:
39504
diff
changeset
|
966 all_revs.update(chain) |
c6b8eab5db19
snapshot: also consider the snapshot chain of one unrelated revision
Boris Feld <boris.feld@octobus.net>
parents:
39504
diff
changeset
|
967 other = None |
c6b8eab5db19
snapshot: also consider the snapshot chain of one unrelated revision
Boris Feld <boris.feld@octobus.net>
parents:
39504
diff
changeset
|
968 for r in revlog.revs(prev, snapfloor): |
c6b8eab5db19
snapshot: also consider the snapshot chain of one unrelated revision
Boris Feld <boris.feld@octobus.net>
parents:
39504
diff
changeset
|
969 if r not in all_revs: |
c6b8eab5db19
snapshot: also consider the snapshot chain of one unrelated revision
Boris Feld <boris.feld@octobus.net>
parents:
39504
diff
changeset
|
970 other = r |
c6b8eab5db19
snapshot: also consider the snapshot chain of one unrelated revision
Boris Feld <boris.feld@octobus.net>
parents:
39504
diff
changeset
|
971 break |
c6b8eab5db19
snapshot: also consider the snapshot chain of one unrelated revision
Boris Feld <boris.feld@octobus.net>
parents:
39504
diff
changeset
|
972 if other is not None: |
c6b8eab5db19
snapshot: also consider the snapshot chain of one unrelated revision
Boris Feld <boris.feld@octobus.net>
parents:
39504
diff
changeset
|
973 # To avoid unfair competition, we won't use unrelated intermediate |
c6b8eab5db19
snapshot: also consider the snapshot chain of one unrelated revision
Boris Feld <boris.feld@octobus.net>
parents:
39504
diff
changeset
|
974 # snapshot that are deeper than the ones from the parent delta |
c6b8eab5db19
snapshot: also consider the snapshot chain of one unrelated revision
Boris Feld <boris.feld@octobus.net>
parents:
39504
diff
changeset
|
975 # chain. |
c6b8eab5db19
snapshot: also consider the snapshot chain of one unrelated revision
Boris Feld <boris.feld@octobus.net>
parents:
39504
diff
changeset
|
976 max_depth = max(parents_snaps.keys()) |
c6b8eab5db19
snapshot: also consider the snapshot chain of one unrelated revision
Boris Feld <boris.feld@octobus.net>
parents:
39504
diff
changeset
|
977 chain = deltachain(other) |
49614
01ccb45b7393
delta-find: rename a variable for clarity
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49613
diff
changeset
|
978 for depth, s in enumerate(chain): |
39505
c6b8eab5db19
snapshot: also consider the snapshot chain of one unrelated revision
Boris Feld <boris.feld@octobus.net>
parents:
39504
diff
changeset
|
979 if s < snapfloor: |
c6b8eab5db19
snapshot: also consider the snapshot chain of one unrelated revision
Boris Feld <boris.feld@octobus.net>
parents:
39504
diff
changeset
|
980 continue |
49614
01ccb45b7393
delta-find: rename a variable for clarity
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49613
diff
changeset
|
981 if max_depth < depth: |
39505
c6b8eab5db19
snapshot: also consider the snapshot chain of one unrelated revision
Boris Feld <boris.feld@octobus.net>
parents:
39504
diff
changeset
|
982 break |
c6b8eab5db19
snapshot: also consider the snapshot chain of one unrelated revision
Boris Feld <boris.feld@octobus.net>
parents:
39504
diff
changeset
|
983 if not revlog.issnapshot(s): |
c6b8eab5db19
snapshot: also consider the snapshot chain of one unrelated revision
Boris Feld <boris.feld@octobus.net>
parents:
39504
diff
changeset
|
984 break |
49614
01ccb45b7393
delta-find: rename a variable for clarity
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49613
diff
changeset
|
985 parents_snaps[depth].add(s) |
39494
e72130f58f5d
snapshot: consider all snapshots in the parents' chains
Boris Feld <boris.feld@octobus.net>
parents:
39493
diff
changeset
|
986 # Test them as possible intermediate snapshot base |
e72130f58f5d
snapshot: consider all snapshots in the parents' chains
Boris Feld <boris.feld@octobus.net>
parents:
39493
diff
changeset
|
987 # We test them from highest to lowest level. High level one are more |
e72130f58f5d
snapshot: consider all snapshots in the parents' chains
Boris Feld <boris.feld@octobus.net>
parents:
39493
diff
changeset
|
988 # likely to result in small delta |
39495
6a53842727c1
snapshot: consider unrelated snapshots at a similar level first
Boris Feld <boris.feld@octobus.net>
parents:
39494
diff
changeset
|
989 floor = None |
39494
e72130f58f5d
snapshot: consider all snapshots in the parents' chains
Boris Feld <boris.feld@octobus.net>
parents:
39493
diff
changeset
|
990 for idx, snaps in sorted(parents_snaps.items(), reverse=True): |
39495
6a53842727c1
snapshot: consider unrelated snapshots at a similar level first
Boris Feld <boris.feld@octobus.net>
parents:
39494
diff
changeset
|
991 siblings = set() |
6a53842727c1
snapshot: consider unrelated snapshots at a similar level first
Boris Feld <boris.feld@octobus.net>
parents:
39494
diff
changeset
|
992 for s in snaps: |
49678
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
993 siblings.update(snapshot_cache.snapshots[s]) |
39495
6a53842727c1
snapshot: consider unrelated snapshots at a similar level first
Boris Feld <boris.feld@octobus.net>
parents:
39494
diff
changeset
|
994 # Before considering making a new intermediate snapshot, we check |
6a53842727c1
snapshot: consider unrelated snapshots at a similar level first
Boris Feld <boris.feld@octobus.net>
parents:
39494
diff
changeset
|
995 # if an existing snapshot, children of base we consider, would be |
6a53842727c1
snapshot: consider unrelated snapshots at a similar level first
Boris Feld <boris.feld@octobus.net>
parents:
39494
diff
changeset
|
996 # suitable. |
6a53842727c1
snapshot: consider unrelated snapshots at a similar level first
Boris Feld <boris.feld@octobus.net>
parents:
39494
diff
changeset
|
997 # |
6a53842727c1
snapshot: consider unrelated snapshots at a similar level first
Boris Feld <boris.feld@octobus.net>
parents:
39494
diff
changeset
|
998 # It give a change to reuse a delta chain "unrelated" to the |
6a53842727c1
snapshot: consider unrelated snapshots at a similar level first
Boris Feld <boris.feld@octobus.net>
parents:
39494
diff
changeset
|
999 # current revision instead of starting our own. Without such |
6a53842727c1
snapshot: consider unrelated snapshots at a similar level first
Boris Feld <boris.feld@octobus.net>
parents:
39494
diff
changeset
|
1000 # re-use, topological branches would keep reopening new chains. |
6a53842727c1
snapshot: consider unrelated snapshots at a similar level first
Boris Feld <boris.feld@octobus.net>
parents:
39494
diff
changeset
|
1001 # Creating more and more snapshot as the repository grow. |
6a53842727c1
snapshot: consider unrelated snapshots at a similar level first
Boris Feld <boris.feld@octobus.net>
parents:
39494
diff
changeset
|
1002 |
6a53842727c1
snapshot: consider unrelated snapshots at a similar level first
Boris Feld <boris.feld@octobus.net>
parents:
39494
diff
changeset
|
1003 if floor is not None: |
6a53842727c1
snapshot: consider unrelated snapshots at a similar level first
Boris Feld <boris.feld@octobus.net>
parents:
39494
diff
changeset
|
1004 # We only do this for siblings created after the one in our |
6a53842727c1
snapshot: consider unrelated snapshots at a similar level first
Boris Feld <boris.feld@octobus.net>
parents:
39494
diff
changeset
|
1005 # parent's delta chain. Those created before has less chances |
6a53842727c1
snapshot: consider unrelated snapshots at a similar level first
Boris Feld <boris.feld@octobus.net>
parents:
39494
diff
changeset
|
1006 # to be valid base since our ancestors had to create a new |
6a53842727c1
snapshot: consider unrelated snapshots at a similar level first
Boris Feld <boris.feld@octobus.net>
parents:
39494
diff
changeset
|
1007 # snapshot. |
6a53842727c1
snapshot: consider unrelated snapshots at a similar level first
Boris Feld <boris.feld@octobus.net>
parents:
39494
diff
changeset
|
1008 siblings = [r for r in siblings if floor < r] |
6a53842727c1
snapshot: consider unrelated snapshots at a similar level first
Boris Feld <boris.feld@octobus.net>
parents:
39494
diff
changeset
|
1009 yield tuple(sorted(siblings)) |
6a53842727c1
snapshot: consider unrelated snapshots at a similar level first
Boris Feld <boris.feld@octobus.net>
parents:
39494
diff
changeset
|
1010 # then test the base from our parent's delta chain. |
39494
e72130f58f5d
snapshot: consider all snapshots in the parents' chains
Boris Feld <boris.feld@octobus.net>
parents:
39493
diff
changeset
|
1011 yield tuple(sorted(snaps)) |
39495
6a53842727c1
snapshot: consider unrelated snapshots at a similar level first
Boris Feld <boris.feld@octobus.net>
parents:
39494
diff
changeset
|
1012 floor = min(snaps) |
39493
3ca144f1c8dd
snapshot: search for unrelated but reusable full-snapshot
Boris Feld <boris.feld@octobus.net>
parents:
39492
diff
changeset
|
1013 # No suitable base found in the parent chain, search if any full |
3ca144f1c8dd
snapshot: search for unrelated but reusable full-snapshot
Boris Feld <boris.feld@octobus.net>
parents:
39492
diff
changeset
|
1014 # snapshots emitted since parent's base would be a suitable base for an |
3ca144f1c8dd
snapshot: search for unrelated but reusable full-snapshot
Boris Feld <boris.feld@octobus.net>
parents:
39492
diff
changeset
|
1015 # intermediate snapshot. |
3ca144f1c8dd
snapshot: search for unrelated but reusable full-snapshot
Boris Feld <boris.feld@octobus.net>
parents:
39492
diff
changeset
|
1016 # |
3ca144f1c8dd
snapshot: search for unrelated but reusable full-snapshot
Boris Feld <boris.feld@octobus.net>
parents:
39492
diff
changeset
|
1017 # It give a chance to reuse a delta chain unrelated to the current |
3ca144f1c8dd
snapshot: search for unrelated but reusable full-snapshot
Boris Feld <boris.feld@octobus.net>
parents:
39492
diff
changeset
|
1018 # revisions instead of starting our own. Without such re-use, |
3ca144f1c8dd
snapshot: search for unrelated but reusable full-snapshot
Boris Feld <boris.feld@octobus.net>
parents:
39492
diff
changeset
|
1019 # topological branches would keep reopening new full chains. Creating |
3ca144f1c8dd
snapshot: search for unrelated but reusable full-snapshot
Boris Feld <boris.feld@octobus.net>
parents:
39492
diff
changeset
|
1020 # more and more snapshot as the repository grow. |
49680
40e24d82b513
delta-find: make sure we only use newer full snapshot as candidate
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49679
diff
changeset
|
1021 full = [r for r in snapshot_cache.snapshots[nullrev] if snapfloor <= r] |
40e24d82b513
delta-find: make sure we only use newer full snapshot as candidate
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49679
diff
changeset
|
1022 yield tuple(sorted(full)) |
39492
a33f394b2bfd
snapshot: try intermediate snapshot against parents' base
Boris Feld <boris.feld@octobus.net>
parents:
39488
diff
changeset
|
1023 |
39505
c6b8eab5db19
snapshot: also consider the snapshot chain of one unrelated revision
Boris Feld <boris.feld@octobus.net>
parents:
39504
diff
changeset
|
1024 if not sparse: |
c6b8eab5db19
snapshot: also consider the snapshot chain of one unrelated revision
Boris Feld <boris.feld@octobus.net>
parents:
39504
diff
changeset
|
1025 # other approach failed try against prev to hopefully save us a |
c6b8eab5db19
snapshot: also consider the snapshot chain of one unrelated revision
Boris Feld <boris.feld@octobus.net>
parents:
39504
diff
changeset
|
1026 # fulltext. |
c6b8eab5db19
snapshot: also consider the snapshot chain of one unrelated revision
Boris Feld <boris.feld@octobus.net>
parents:
39504
diff
changeset
|
1027 yield (prev,) |
39334
507f5b1dd7c8
revlogdeltas: extract _getcandidaterevs in a function
Boris Feld <boris.feld@octobus.net>
parents:
39333
diff
changeset
|
1028 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1029 |
49678
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1030 class SnapshotCache: |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1031 __slots__ = ('snapshots', '_start_rev', '_end_rev') |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1032 |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1033 def __init__(self): |
49679
b670eb3dd6c9
delta-find: use sets instead of list in the snapshot cache
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49678
diff
changeset
|
1034 self.snapshots = collections.defaultdict(set) |
49678
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1035 self._start_rev = None |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1036 self._end_rev = None |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1037 |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1038 def update(self, revlog, start_rev=0): |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1039 """find snapshots from start_rev to tip""" |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1040 nb_revs = len(revlog) |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1041 end_rev = nb_revs - 1 |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1042 if start_rev > end_rev: |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1043 return # range is empty |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1044 |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1045 if self._start_rev is None: |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1046 assert self._end_rev is None |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1047 self._update(revlog, start_rev, end_rev) |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1048 elif not (self._start_rev <= start_rev and end_rev <= self._end_rev): |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1049 if start_rev < self._start_rev: |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1050 self._update(revlog, start_rev, self._start_rev - 1) |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1051 if self._end_rev < end_rev: |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1052 self._update(revlog, self._end_rev + 1, end_rev) |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1053 |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1054 if self._start_rev is None: |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1055 assert self._end_rev is None |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1056 self._end_rev = end_rev |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1057 self._start_rev = start_rev |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1058 else: |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1059 self._start_rev = min(self._start_rev, start_rev) |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1060 self._end_rev = max(self._end_rev, end_rev) |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1061 assert self._start_rev <= self._end_rev, ( |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1062 self._start_rev, |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1063 self._end_rev, |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1064 ) |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1065 |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1066 def _update(self, revlog, start_rev, end_rev): |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1067 """internal method that actually do update content""" |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1068 assert self._start_rev is None or ( |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1069 start_rev < self._start_rev or start_rev > self._end_rev |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1070 ), (self._start_rev, self._end_rev, start_rev, end_rev) |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1071 assert self._start_rev is None or ( |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1072 end_rev < self._start_rev or end_rev > self._end_rev |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1073 ), (self._start_rev, self._end_rev, start_rev, end_rev) |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1074 cache = self.snapshots |
50928
d718eddf01d9
safehasattr: drop usage in favor of hasattr
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50698
diff
changeset
|
1075 if hasattr(revlog.index, 'findsnapshots'): |
49678
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1076 revlog.index.findsnapshots(cache, start_rev, end_rev) |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1077 else: |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1078 deltaparent = revlog.deltaparent |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1079 issnapshot = revlog.issnapshot |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1080 for rev in revlog.revs(start_rev, end_rev): |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1081 if issnapshot(rev): |
49679
b670eb3dd6c9
delta-find: use sets instead of list in the snapshot cache
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49678
diff
changeset
|
1082 cache[deltaparent(rev)].add(rev) |
49678
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1083 |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1084 |
48946
642e31cb55f0
py3: use class X: instead of class X(object):
Gregory Szorc <gregory.szorc@gmail.com>
parents:
48875
diff
changeset
|
1085 class deltacomputer: |
49610
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
1086 def __init__( |
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
1087 self, |
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
1088 revlog, |
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
1089 write_debug=None, |
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
1090 debug_search=False, |
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
1091 debug_info=None, |
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
1092 ): |
35738
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1093 self.revlog = revlog |
49226
e6b7c6fbeb48
deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
1094 self._write_debug = write_debug |
50649
c84cc0ac77e4
delta-fine: use the `_debug_search` attribute directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50648
diff
changeset
|
1095 if write_debug is None: |
c84cc0ac77e4
delta-fine: use the `_debug_search` attribute directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50648
diff
changeset
|
1096 self._debug_search = False |
c84cc0ac77e4
delta-fine: use the `_debug_search` attribute directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50648
diff
changeset
|
1097 else: |
c84cc0ac77e4
delta-fine: use the `_debug_search` attribute directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50648
diff
changeset
|
1098 self._debug_search = debug_search |
49610
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
1099 self._debug_info = debug_info |
49681
c261a628e525
delta-find: use a single snapshot cache when applying a group to an object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49680
diff
changeset
|
1100 self._snapshot_cache = SnapshotCache() |
35738
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1101 |
50648
5d210ff4b657
delta-find: move the `gather_debug` logic in a property
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50357
diff
changeset
|
1102 @property |
5d210ff4b657
delta-find: move the `gather_debug` logic in a property
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50357
diff
changeset
|
1103 def _gather_debug(self): |
5d210ff4b657
delta-find: move the `gather_debug` logic in a property
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50357
diff
changeset
|
1104 return self._write_debug is not None or self._debug_info is not None |
5d210ff4b657
delta-find: move the `gather_debug` logic in a property
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50357
diff
changeset
|
1105 |
51020
509f0f7fc89e
delta-computer: stop explicitly taking file handle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50929
diff
changeset
|
1106 def buildtext(self, revinfo): |
35738
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1107 """Builds a fulltext version of a revision |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1108 |
47399
34cc102c73f5
revlog: move `revisioninfo` in `revlogutils`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47253
diff
changeset
|
1109 revinfo: revisioninfo instance that contains all needed info |
35738
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1110 """ |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1111 btext = revinfo.btext |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1112 if btext[0] is not None: |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1113 return btext[0] |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1114 |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1115 revlog = self.revlog |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1116 cachedelta = revinfo.cachedelta |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1117 baserev = cachedelta[0] |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1118 delta = cachedelta[1] |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1119 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1120 fulltext = btext[0] = _textfromdelta( |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1121 revlog, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1122 baserev, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1123 delta, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1124 revinfo.p1, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1125 revinfo.p2, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1126 revinfo.flags, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1127 revinfo.node, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1128 ) |
39331
fd0150a3c2fe
revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents:
39330
diff
changeset
|
1129 return fulltext |
35738
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1130 |
51020
509f0f7fc89e
delta-computer: stop explicitly taking file handle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50929
diff
changeset
|
1131 def _builddeltadiff(self, base, revinfo): |
35738
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1132 revlog = self.revlog |
51020
509f0f7fc89e
delta-computer: stop explicitly taking file handle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50929
diff
changeset
|
1133 t = self.buildtext(revinfo) |
35738
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1134 if revlog.iscensored(base): |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1135 # deltas based on a censored revision must replace the |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1136 # full content in one patch, so delta works everywhere |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1137 header = mdiff.replacediffheader(revlog.rawsize(base), len(t)) |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1138 delta = header + t |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1139 else: |
51020
509f0f7fc89e
delta-computer: stop explicitly taking file handle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50929
diff
changeset
|
1140 ptext = revlog.rawdata(base) |
35738
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1141 delta = mdiff.textdiff(ptext, t) |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1142 |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1143 return delta |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1144 |
51020
509f0f7fc89e
delta-computer: stop explicitly taking file handle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50929
diff
changeset
|
1145 def _builddeltainfo(self, revinfo, base, target_rev=None): |
35738
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1146 # can we use the cached delta? |
42465
6e9ba867a946
delta: move some delta chain related computation earlier in deltainfo
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42464
diff
changeset
|
1147 revlog = self.revlog |
6e9ba867a946
delta: move some delta chain related computation earlier in deltainfo
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42464
diff
changeset
|
1148 chainbase = revlog.chainbase(base) |
51046
f636103c4d67
revlog: remove legacy usage of `_generaldelta`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51045
diff
changeset
|
1149 if revlog.delta_config.general_delta: |
42465
6e9ba867a946
delta: move some delta chain related computation earlier in deltainfo
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42464
diff
changeset
|
1150 deltabase = base |
6e9ba867a946
delta: move some delta chain related computation earlier in deltainfo
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42464
diff
changeset
|
1151 else: |
50356
2a6949ab9d23
delta-find: add a simple safeguard to prevent bad non-general-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50355
diff
changeset
|
1152 if target_rev is not None and base != target_rev - 1: |
2a6949ab9d23
delta-find: add a simple safeguard to prevent bad non-general-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50355
diff
changeset
|
1153 msg = ( |
2a6949ab9d23
delta-find: add a simple safeguard to prevent bad non-general-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50355
diff
changeset
|
1154 b'general delta cannot use delta for something else ' |
2a6949ab9d23
delta-find: add a simple safeguard to prevent bad non-general-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50355
diff
changeset
|
1155 b'than `prev`: %d<-%d' |
2a6949ab9d23
delta-find: add a simple safeguard to prevent bad non-general-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50355
diff
changeset
|
1156 ) |
2a6949ab9d23
delta-find: add a simple safeguard to prevent bad non-general-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50355
diff
changeset
|
1157 msg %= (base, target_rev) |
2a6949ab9d23
delta-find: add a simple safeguard to prevent bad non-general-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50355
diff
changeset
|
1158 raise error.ProgrammingError(msg) |
42465
6e9ba867a946
delta: move some delta chain related computation earlier in deltainfo
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42464
diff
changeset
|
1159 deltabase = chainbase |
6e9ba867a946
delta: move some delta chain related computation earlier in deltainfo
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42464
diff
changeset
|
1160 snapshotdepth = None |
51060
f71f07a679b4
revlog: remove legacy usage of `_sparserevlog`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51058
diff
changeset
|
1161 if revlog.delta_config.sparse_revlog and deltabase == nullrev: |
42465
6e9ba867a946
delta: move some delta chain related computation earlier in deltainfo
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42464
diff
changeset
|
1162 snapshotdepth = 0 |
51060
f71f07a679b4
revlog: remove legacy usage of `_sparserevlog`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51058
diff
changeset
|
1163 elif revlog.delta_config.sparse_revlog and revlog.issnapshot(deltabase): |
42465
6e9ba867a946
delta: move some delta chain related computation earlier in deltainfo
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42464
diff
changeset
|
1164 # A delta chain should always be one full snapshot, |
6e9ba867a946
delta: move some delta chain related computation earlier in deltainfo
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42464
diff
changeset
|
1165 # zero or more semi-snapshots, and zero or more deltas |
6e9ba867a946
delta: move some delta chain related computation earlier in deltainfo
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42464
diff
changeset
|
1166 p1, p2 = revlog.rev(revinfo.p1), revlog.rev(revinfo.p2) |
6e9ba867a946
delta: move some delta chain related computation earlier in deltainfo
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42464
diff
changeset
|
1167 if deltabase not in (p1, p2) and revlog.issnapshot(deltabase): |
6e9ba867a946
delta: move some delta chain related computation earlier in deltainfo
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42464
diff
changeset
|
1168 snapshotdepth = len(revlog._deltachain(deltabase)[0]) |
39595
a911932d5003
revlog: reuse cached delta for identical base revision (issue5975)
Boris Feld <boris.feld@octobus.net>
parents:
39594
diff
changeset
|
1169 delta = None |
a911932d5003
revlog: reuse cached delta for identical base revision (issue5975)
Boris Feld <boris.feld@octobus.net>
parents:
39594
diff
changeset
|
1170 if revinfo.cachedelta: |
49675
0fca63953810
find-delta: minor preparatory change
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49674
diff
changeset
|
1171 cachebase = revinfo.cachedelta[0] |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1172 # check if the diff still apply |
39595
a911932d5003
revlog: reuse cached delta for identical base revision (issue5975)
Boris Feld <boris.feld@octobus.net>
parents:
39594
diff
changeset
|
1173 currentbase = cachebase |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1174 while ( |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1175 currentbase != nullrev |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1176 and currentbase != base |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1177 and self.revlog.length(currentbase) == 0 |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1178 ): |
39595
a911932d5003
revlog: reuse cached delta for identical base revision (issue5975)
Boris Feld <boris.feld@octobus.net>
parents:
39594
diff
changeset
|
1179 currentbase = self.revlog.deltaparent(currentbase) |
51067
26dcdbe15024
revlog: remove legacy usage of `_lazydeltabase`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51063
diff
changeset
|
1180 if self.revlog.delta_config.lazy_delta and currentbase == base: |
39595
a911932d5003
revlog: reuse cached delta for identical base revision (issue5975)
Boris Feld <boris.feld@octobus.net>
parents:
39594
diff
changeset
|
1181 delta = revinfo.cachedelta[1] |
a911932d5003
revlog: reuse cached delta for identical base revision (issue5975)
Boris Feld <boris.feld@octobus.net>
parents:
39594
diff
changeset
|
1182 if delta is None: |
51020
509f0f7fc89e
delta-computer: stop explicitly taking file handle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50929
diff
changeset
|
1183 delta = self._builddeltadiff(base, revinfo) |
50649
c84cc0ac77e4
delta-fine: use the `_debug_search` attribute directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50648
diff
changeset
|
1184 if self._debug_search: |
49228
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1185 msg = b"DBG-DELTAS-SEARCH: uncompressed-delta-size=%d\n" |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1186 msg %= len(delta) |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1187 self._write_debug(msg) |
42467
c1c1872d25d1
deltas: skip if projected compressed size does not match text size constraint
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42466
diff
changeset
|
1188 # snapshotdept need to be neither None nor 0 level snapshot |
51083
5645524c6b62
revlog: also migrates `revlog.upperboundcomp` to ConfigClass
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51067
diff
changeset
|
1189 if revlog.delta_config.upper_bound_comp is not None and snapshotdepth: |
5645524c6b62
revlog: also migrates `revlog.upperboundcomp` to ConfigClass
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51067
diff
changeset
|
1190 lowestrealisticdeltalen = ( |
5645524c6b62
revlog: also migrates `revlog.upperboundcomp` to ConfigClass
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51067
diff
changeset
|
1191 len(delta) // revlog.delta_config.upper_bound_comp |
5645524c6b62
revlog: also migrates `revlog.upperboundcomp` to ConfigClass
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51067
diff
changeset
|
1192 ) |
42467
c1c1872d25d1
deltas: skip if projected compressed size does not match text size constraint
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42466
diff
changeset
|
1193 snapshotlimit = revinfo.textlen >> snapshotdepth |
50649
c84cc0ac77e4
delta-fine: use the `_debug_search` attribute directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50648
diff
changeset
|
1194 if self._debug_search: |
49228
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1195 msg = b"DBG-DELTAS-SEARCH: projected-lower-size=%d\n" |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1196 msg %= lowestrealisticdeltalen |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1197 self._write_debug(msg) |
42467
c1c1872d25d1
deltas: skip if projected compressed size does not match text size constraint
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42466
diff
changeset
|
1198 if snapshotlimit < lowestrealisticdeltalen: |
50649
c84cc0ac77e4
delta-fine: use the `_debug_search` attribute directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50648
diff
changeset
|
1199 if self._debug_search: |
49228
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1200 msg = b"DBG-DELTAS-SEARCH: DISCARDED (snapshot limit)\n" |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1201 self._write_debug(msg) |
42467
c1c1872d25d1
deltas: skip if projected compressed size does not match text size constraint
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42466
diff
changeset
|
1202 return None |
42468
9b5fbe5ead89
deltas: skip if projected compressed size is bigger than previous snapshot
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42467
diff
changeset
|
1203 if revlog.length(base) < lowestrealisticdeltalen: |
50649
c84cc0ac77e4
delta-fine: use the `_debug_search` attribute directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50648
diff
changeset
|
1204 if self._debug_search: |
49228
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1205 msg = b"DBG-DELTAS-SEARCH: DISCARDED (prev size)\n" |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1206 self._write_debug(msg) |
42468
9b5fbe5ead89
deltas: skip if projected compressed size is bigger than previous snapshot
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42467
diff
changeset
|
1207 return None |
51091
a82704902db8
revlog: move the compression/decompression logic on the inner object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51083
diff
changeset
|
1208 header, data = revlog._inner.compress(delta) |
35738
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1209 deltalen = len(header) + len(data) |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1210 offset = revlog.end(len(revlog) - 1) |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1211 dist = deltalen + offset - revlog.start(chainbase) |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1212 chainlen, compresseddeltalen = revlog._chaininfo(base) |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1213 chainlen += 1 |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1214 compresseddeltalen += deltalen |
39154
e0da43e2f71f
revlog: compute snapshot depth on delta info
Boris Feld <boris.feld@octobus.net>
parents:
39152
diff
changeset
|
1215 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1216 return _deltainfo( |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1217 dist, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1218 deltalen, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1219 (header, data), |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1220 deltabase, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1221 chainbase, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1222 chainlen, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1223 compresseddeltalen, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1224 snapshotdepth, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1225 ) |
35738
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1226 |
51020
509f0f7fc89e
delta-computer: stop explicitly taking file handle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50929
diff
changeset
|
1227 def _fullsnapshotinfo(self, revinfo, curr): |
509f0f7fc89e
delta-computer: stop explicitly taking file handle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50929
diff
changeset
|
1228 rawtext = self.buildtext(revinfo) |
51091
a82704902db8
revlog: move the compression/decompression logic on the inner object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51083
diff
changeset
|
1229 data = self.revlog._inner.compress(rawtext) |
39333
5d343a24bff5
revlogdeltas: always return a delta info object in finddeltainfo
Boris Feld <boris.feld@octobus.net>
parents:
39332
diff
changeset
|
1230 compresseddeltalen = deltalen = dist = len(data[1]) + len(data[0]) |
5d343a24bff5
revlogdeltas: always return a delta info object in finddeltainfo
Boris Feld <boris.feld@octobus.net>
parents:
39332
diff
changeset
|
1231 deltabase = chainbase = curr |
5d343a24bff5
revlogdeltas: always return a delta info object in finddeltainfo
Boris Feld <boris.feld@octobus.net>
parents:
39332
diff
changeset
|
1232 snapshotdepth = 0 |
5d343a24bff5
revlogdeltas: always return a delta info object in finddeltainfo
Boris Feld <boris.feld@octobus.net>
parents:
39332
diff
changeset
|
1233 chainlen = 1 |
5d343a24bff5
revlogdeltas: always return a delta info object in finddeltainfo
Boris Feld <boris.feld@octobus.net>
parents:
39332
diff
changeset
|
1234 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1235 return _deltainfo( |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1236 dist, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1237 deltalen, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1238 data, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1239 deltabase, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1240 chainbase, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1241 chainlen, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1242 compresseddeltalen, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1243 snapshotdepth, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1244 ) |
39333
5d343a24bff5
revlogdeltas: always return a delta info object in finddeltainfo
Boris Feld <boris.feld@octobus.net>
parents:
39332
diff
changeset
|
1245 |
51020
509f0f7fc89e
delta-computer: stop explicitly taking file handle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50929
diff
changeset
|
1246 def finddeltainfo(self, revinfo, excluded_bases=None, target_rev=None): |
35738
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1247 """Find an acceptable delta against a candidate revision |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1248 |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1249 revinfo: information about the revision (instance of _revisioninfo) |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1250 |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1251 Returns the first acceptable candidate revision, as ordered by |
39334
507f5b1dd7c8
revlogdeltas: extract _getcandidaterevs in a function
Boris Feld <boris.feld@octobus.net>
parents:
39333
diff
changeset
|
1252 _candidategroups |
39333
5d343a24bff5
revlogdeltas: always return a delta info object in finddeltainfo
Boris Feld <boris.feld@octobus.net>
parents:
39332
diff
changeset
|
1253 |
5d343a24bff5
revlogdeltas: always return a delta info object in finddeltainfo
Boris Feld <boris.feld@octobus.net>
parents:
39332
diff
changeset
|
1254 If no suitable deltabase is found, we return delta info for a full |
5d343a24bff5
revlogdeltas: always return a delta info object in finddeltainfo
Boris Feld <boris.feld@octobus.net>
parents:
39332
diff
changeset
|
1255 snapshot. |
47401
1efe3cdef53a
revlog: add a ways to blacklist some revision when searching for a delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47399
diff
changeset
|
1256 |
1efe3cdef53a
revlog: add a ways to blacklist some revision when searching for a delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47399
diff
changeset
|
1257 `excluded_bases` is an optional set of revision that cannot be used as |
1efe3cdef53a
revlog: add a ways to blacklist some revision when searching for a delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47399
diff
changeset
|
1258 a delta base. Use this to recompute delta suitable in censor or strip |
1efe3cdef53a
revlog: add a ways to blacklist some revision when searching for a delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47399
diff
changeset
|
1259 context. |
35738
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1260 """ |
47456
93f4e183b3f5
deltas: at a `target_rev` parameter to finddeltainfo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47452
diff
changeset
|
1261 if target_rev is None: |
47457
f8330a3fc39f
censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47456
diff
changeset
|
1262 target_rev = len(self.revlog) |
47456
93f4e183b3f5
deltas: at a `target_rev` parameter to finddeltainfo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47452
diff
changeset
|
1263 |
50653
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1264 gather_debug = self._gather_debug |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1265 cachedelta = revinfo.cachedelta |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1266 revlog = self.revlog |
50657
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1267 p1r = p2r = None |
50653
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1268 |
50657
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1269 if excluded_bases is None: |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1270 excluded_bases = set() |
50653
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1271 |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1272 if gather_debug: |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1273 start = util.timer() |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1274 dbg = self._one_dbg_data() |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1275 dbg['revision'] = target_rev |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1276 target_revlog = b"UNKNOWN" |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1277 target_type = self.revlog.target[0] |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1278 target_key = self.revlog.target[1] |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1279 if target_type == KIND_CHANGELOG: |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1280 target_revlog = b'CHANGELOG:' |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1281 elif target_type == KIND_MANIFESTLOG: |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1282 target_revlog = b'MANIFESTLOG:' |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1283 if target_key: |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1284 target_revlog += b'%s:' % target_key |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1285 elif target_type == KIND_FILELOG: |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1286 target_revlog = b'FILELOG:' |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1287 if target_key: |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1288 target_revlog += b'%s:' % target_key |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1289 dbg['target-revlog'] = target_revlog |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1290 p1r = revlog.rev(revinfo.p1) |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1291 p2r = revlog.rev(revinfo.p2) |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1292 if p1r != nullrev: |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1293 p1_chain_len = revlog._chaininfo(p1r)[0] |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1294 else: |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1295 p1_chain_len = -1 |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1296 if p2r != nullrev: |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1297 p2_chain_len = revlog._chaininfo(p2r)[0] |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1298 else: |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1299 p2_chain_len = -1 |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1300 dbg['p1-chain-len'] = p1_chain_len |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1301 dbg['p2-chain-len'] = p2_chain_len |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1302 |
50654
bfb6404089a5
delta-find: gather the condition to blindly use a full snapshot together
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50653
diff
changeset
|
1303 # 1) if the revision is empty, no amount of delta can beat it |
bfb6404089a5
delta-find: gather the condition to blindly use a full snapshot together
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50653
diff
changeset
|
1304 # |
bfb6404089a5
delta-find: gather the condition to blindly use a full snapshot together
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50653
diff
changeset
|
1305 # 2) no delta for flag processor revision (see "candelta" for why) |
bfb6404089a5
delta-find: gather the condition to blindly use a full snapshot together
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50653
diff
changeset
|
1306 # not calling candelta since only one revision needs test, also to |
bfb6404089a5
delta-find: gather the condition to blindly use a full snapshot together
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50653
diff
changeset
|
1307 # avoid overhead fetching flags again. |
bfb6404089a5
delta-find: gather the condition to blindly use a full snapshot together
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50653
diff
changeset
|
1308 if not revinfo.textlen or revinfo.flags & REVIDX_RAWTEXT_CHANGING_FLAGS: |
51020
509f0f7fc89e
delta-computer: stop explicitly taking file handle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50929
diff
changeset
|
1309 deltainfo = self._fullsnapshotinfo(revinfo, target_rev) |
50655
485c9410b75a
deltafind: issue debug information when we fast-path rivial case too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50654
diff
changeset
|
1310 if gather_debug: |
485c9410b75a
deltafind: issue debug information when we fast-path rivial case too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50654
diff
changeset
|
1311 end = util.timer() |
485c9410b75a
deltafind: issue debug information when we fast-path rivial case too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50654
diff
changeset
|
1312 dbg['duration'] = end - start |
485c9410b75a
deltafind: issue debug information when we fast-path rivial case too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50654
diff
changeset
|
1313 dbg[ |
485c9410b75a
deltafind: issue debug information when we fast-path rivial case too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50654
diff
changeset
|
1314 'delta-base' |
485c9410b75a
deltafind: issue debug information when we fast-path rivial case too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50654
diff
changeset
|
1315 ] = deltainfo.base # pytype: disable=attribute-error |
485c9410b75a
deltafind: issue debug information when we fast-path rivial case too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50654
diff
changeset
|
1316 dbg['search_round_count'] = 0 |
50657
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1317 dbg['using-cached-base'] = False |
50655
485c9410b75a
deltafind: issue debug information when we fast-path rivial case too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50654
diff
changeset
|
1318 dbg['delta_try_count'] = 0 |
485c9410b75a
deltafind: issue debug information when we fast-path rivial case too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50654
diff
changeset
|
1319 dbg['type'] = b"full" |
485c9410b75a
deltafind: issue debug information when we fast-path rivial case too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50654
diff
changeset
|
1320 dbg['snapshot-depth'] = 0 |
485c9410b75a
deltafind: issue debug information when we fast-path rivial case too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50654
diff
changeset
|
1321 self._dbg_process_data(dbg) |
485c9410b75a
deltafind: issue debug information when we fast-path rivial case too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50654
diff
changeset
|
1322 return deltainfo |
39085
dbb3e9e44fce
revlog: do not search for delta for empty content
Boris Feld <boris.feld@octobus.net>
parents:
39084
diff
changeset
|
1323 |
50657
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1324 deltainfo = None |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1325 |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1326 # If this source delta are to be forcibly reuse, let us comply early. |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1327 if ( |
51046
f636103c4d67
revlog: remove legacy usage of `_generaldelta`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51045
diff
changeset
|
1328 revlog.delta_config.general_delta |
50657
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1329 and revinfo.cachedelta is not None |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1330 and revinfo.cachedelta[2] == DELTA_BASE_REUSE_FORCE |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1331 ): |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1332 base = revinfo.cachedelta[0] |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1333 if base == nullrev: |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1334 dbg_type = b"full" |
51020
509f0f7fc89e
delta-computer: stop explicitly taking file handle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50929
diff
changeset
|
1335 deltainfo = self._fullsnapshotinfo(revinfo, target_rev) |
50657
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1336 if gather_debug: |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1337 snapshotdepth = 0 |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1338 elif base not in excluded_bases: |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1339 delta = revinfo.cachedelta[1] |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1340 header, data = revlog.compress(delta) |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1341 deltalen = len(header) + len(data) |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1342 if gather_debug: |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1343 offset = revlog.end(len(revlog) - 1) |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1344 chainbase = revlog.chainbase(base) |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1345 distance = deltalen + offset - revlog.start(chainbase) |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1346 chainlen, compresseddeltalen = revlog._chaininfo(base) |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1347 chainlen += 1 |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1348 compresseddeltalen += deltalen |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1349 if base == p1r or base == p2r: |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1350 dbg_type = b"delta" |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1351 snapshotdepth = None |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1352 elif not revlog.issnapshot(base): |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1353 snapshotdepth = None |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1354 else: |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1355 dbg_type = b"snapshot" |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1356 snapshotdepth = revlog.snapshotdepth(base) + 1 |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1357 else: |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1358 distance = None |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1359 chainbase = None |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1360 chainlen = None |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1361 compresseddeltalen = None |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1362 snapshotdepth = None |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1363 deltainfo = _deltainfo( |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1364 distance=distance, |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1365 deltalen=deltalen, |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1366 data=(header, data), |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1367 base=base, |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1368 chainbase=chainbase, |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1369 chainlen=chainlen, |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1370 compresseddeltalen=compresseddeltalen, |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1371 snapshotdepth=snapshotdepth, |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1372 ) |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1373 |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1374 if deltainfo is not None: |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1375 if gather_debug: |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1376 end = util.timer() |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1377 dbg['duration'] = end - start |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1378 dbg[ |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1379 'delta-base' |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1380 ] = deltainfo.base # pytype: disable=attribute-error |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1381 dbg['search_round_count'] = 0 |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1382 dbg['using-cached-base'] = True |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1383 dbg['delta_try_count'] = 0 |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1384 dbg['type'] = b"full" |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1385 if snapshotdepth is None: |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1386 dbg['snapshot-depth'] = 0 |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1387 else: |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1388 dbg['snapshot-depth'] = snapshotdepth |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1389 self._dbg_process_data(dbg) |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1390 return deltainfo |
47401
1efe3cdef53a
revlog: add a ways to blacklist some revision when searching for a delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47399
diff
changeset
|
1391 |
49226
e6b7c6fbeb48
deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
1392 # count the number of different delta we tried (for debug purpose) |
e6b7c6fbeb48
deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
1393 dbg_try_count = 0 |
e6b7c6fbeb48
deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
1394 # count the number of "search round" we did. (for debug purpose) |
e6b7c6fbeb48
deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
1395 dbg_try_rounds = 0 |
e6b7c6fbeb48
deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
1396 dbg_type = b'unknown' |
e6b7c6fbeb48
deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
1397 |
50653
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1398 if p1r is None: |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1399 p1r = revlog.rev(revinfo.p1) |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1400 p2r = revlog.rev(revinfo.p2) |
49226
e6b7c6fbeb48
deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
1401 |
50649
c84cc0ac77e4
delta-fine: use the `_debug_search` attribute directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50648
diff
changeset
|
1402 if self._debug_search: |
49228
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1403 msg = b"DBG-DELTAS-SEARCH: SEARCH rev=%d\n" |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1404 msg %= target_rev |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1405 self._write_debug(msg) |
49226
e6b7c6fbeb48
deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
1406 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1407 groups = _candidategroups( |
49611
2afee2176775
delta-find: expand a function definition and call before extendin it
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49610
diff
changeset
|
1408 self.revlog, |
2afee2176775
delta-find: expand a function definition and call before extendin it
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49610
diff
changeset
|
1409 revinfo.textlen, |
2afee2176775
delta-find: expand a function definition and call before extendin it
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49610
diff
changeset
|
1410 p1r, |
2afee2176775
delta-find: expand a function definition and call before extendin it
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49610
diff
changeset
|
1411 p2r, |
2afee2176775
delta-find: expand a function definition and call before extendin it
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49610
diff
changeset
|
1412 cachedelta, |
49612
e706bb41fdb3
delta-find: move pre-filtering with other pre-filtering logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49611
diff
changeset
|
1413 excluded_bases, |
e706bb41fdb3
delta-find: move pre-filtering with other pre-filtering logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49611
diff
changeset
|
1414 target_rev, |
49681
c261a628e525
delta-find: use a single snapshot cache when applying a group to an object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49680
diff
changeset
|
1415 snapshot_cache=self._snapshot_cache, |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1416 ) |
39497
5b308a4e6d03
snapshot: use None as a stop value when looking for a good delta
Boris Feld <boris.feld@octobus.net>
parents:
39496
diff
changeset
|
1417 candidaterevs = next(groups) |
5b308a4e6d03
snapshot: use None as a stop value when looking for a good delta
Boris Feld <boris.feld@octobus.net>
parents:
39496
diff
changeset
|
1418 while candidaterevs is not None: |
49226
e6b7c6fbeb48
deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
1419 dbg_try_rounds += 1 |
50649
c84cc0ac77e4
delta-fine: use the `_debug_search` attribute directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50648
diff
changeset
|
1420 if self._debug_search: |
49228
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1421 prev = None |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1422 if deltainfo is not None: |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1423 prev = deltainfo.base |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1424 |
49608
78ba41878f2e
delta-find: add debug information about reuse of cached data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49228
diff
changeset
|
1425 if ( |
78ba41878f2e
delta-find: add debug information about reuse of cached data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49228
diff
changeset
|
1426 cachedelta is not None |
78ba41878f2e
delta-find: add debug information about reuse of cached data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49228
diff
changeset
|
1427 and len(candidaterevs) == 1 |
78ba41878f2e
delta-find: add debug information about reuse of cached data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49228
diff
changeset
|
1428 and cachedelta[0] in candidaterevs |
78ba41878f2e
delta-find: add debug information about reuse of cached data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49228
diff
changeset
|
1429 ): |
78ba41878f2e
delta-find: add debug information about reuse of cached data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49228
diff
changeset
|
1430 round_type = b"cached-delta" |
50652
ebb292ffdf4d
delta-find: fix `parents` round detection
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50651
diff
changeset
|
1431 elif p1r in candidaterevs or p2r in candidaterevs: |
49228
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1432 round_type = b"parents" |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1433 elif prev is not None and all(c < prev for c in candidaterevs): |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1434 round_type = b"refine-down" |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1435 elif prev is not None and all(c > prev for c in candidaterevs): |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1436 round_type = b"refine-up" |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1437 else: |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1438 round_type = b"search-down" |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1439 msg = b"DBG-DELTAS-SEARCH: ROUND #%d - %d candidates - %s\n" |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1440 msg %= (dbg_try_rounds, len(candidaterevs), round_type) |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1441 self._write_debug(msg) |
35738
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1442 nominateddeltas = [] |
39498
04b75f3a3f2a
snapshot: add refining logic at the findeltainfo level
Boris Feld <boris.feld@octobus.net>
parents:
39497
diff
changeset
|
1443 if deltainfo is not None: |
50649
c84cc0ac77e4
delta-fine: use the `_debug_search` attribute directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50648
diff
changeset
|
1444 if self._debug_search: |
49228
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1445 msg = ( |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1446 b"DBG-DELTAS-SEARCH: CONTENDER: rev=%d - length=%d\n" |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1447 ) |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1448 msg %= (deltainfo.base, deltainfo.deltalen) |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1449 self._write_debug(msg) |
39498
04b75f3a3f2a
snapshot: add refining logic at the findeltainfo level
Boris Feld <boris.feld@octobus.net>
parents:
39497
diff
changeset
|
1450 # if we already found a good delta, |
04b75f3a3f2a
snapshot: add refining logic at the findeltainfo level
Boris Feld <boris.feld@octobus.net>
parents:
39497
diff
changeset
|
1451 # challenge it against refined candidates |
04b75f3a3f2a
snapshot: add refining logic at the findeltainfo level
Boris Feld <boris.feld@octobus.net>
parents:
39497
diff
changeset
|
1452 nominateddeltas.append(deltainfo) |
35738
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1453 for candidaterev in candidaterevs: |
50649
c84cc0ac77e4
delta-fine: use the `_debug_search` attribute directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50648
diff
changeset
|
1454 if self._debug_search: |
49228
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1455 msg = b"DBG-DELTAS-SEARCH: CANDIDATE: rev=%d\n" |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1456 msg %= candidaterev |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1457 self._write_debug(msg) |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1458 candidate_type = None |
50652
ebb292ffdf4d
delta-find: fix `parents` round detection
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50651
diff
changeset
|
1459 if candidaterev == p1r: |
49228
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1460 candidate_type = b"p1" |
50652
ebb292ffdf4d
delta-find: fix `parents` round detection
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50651
diff
changeset
|
1461 elif candidaterev == p2r: |
49228
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1462 candidate_type = b"p2" |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1463 elif self.revlog.issnapshot(candidaterev): |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1464 candidate_type = b"snapshot-%d" |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1465 candidate_type %= self.revlog.snapshotdepth( |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1466 candidaterev |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1467 ) |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1468 |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1469 if candidate_type is not None: |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1470 msg = b"DBG-DELTAS-SEARCH: type=%s\n" |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1471 msg %= candidate_type |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1472 self._write_debug(msg) |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1473 msg = b"DBG-DELTAS-SEARCH: size=%d\n" |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1474 msg %= self.revlog.length(candidaterev) |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1475 self._write_debug(msg) |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1476 msg = b"DBG-DELTAS-SEARCH: base=%d\n" |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1477 msg %= self.revlog.deltaparent(candidaterev) |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1478 self._write_debug(msg) |
49612
e706bb41fdb3
delta-find: move pre-filtering with other pre-filtering logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49611
diff
changeset
|
1479 |
49226
e6b7c6fbeb48
deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
1480 dbg_try_count += 1 |
49228
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1481 |
50649
c84cc0ac77e4
delta-fine: use the `_debug_search` attribute directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50648
diff
changeset
|
1482 if self._debug_search: |
49228
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1483 delta_start = util.timer() |
50356
2a6949ab9d23
delta-find: add a simple safeguard to prevent bad non-general-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50355
diff
changeset
|
1484 candidatedelta = self._builddeltainfo( |
2a6949ab9d23
delta-find: add a simple safeguard to prevent bad non-general-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50355
diff
changeset
|
1485 revinfo, |
2a6949ab9d23
delta-find: add a simple safeguard to prevent bad non-general-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50355
diff
changeset
|
1486 candidaterev, |
2a6949ab9d23
delta-find: add a simple safeguard to prevent bad non-general-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50355
diff
changeset
|
1487 target_rev=target_rev, |
2a6949ab9d23
delta-find: add a simple safeguard to prevent bad non-general-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50355
diff
changeset
|
1488 ) |
50649
c84cc0ac77e4
delta-fine: use the `_debug_search` attribute directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50648
diff
changeset
|
1489 if self._debug_search: |
49228
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1490 delta_end = util.timer() |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1491 msg = b"DBG-DELTAS-SEARCH: delta-search-time=%f\n" |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1492 msg %= delta_end - delta_start |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1493 self._write_debug(msg) |
42466
465f2d0df9ae
deltas: accept and skip None return for delta info
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42465
diff
changeset
|
1494 if candidatedelta is not None: |
49674
5af4a0a73e4c
find-delta: rename _isgooddeltainfo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49657
diff
changeset
|
1495 if is_good_delta_info(self.revlog, candidatedelta, revinfo): |
50649
c84cc0ac77e4
delta-fine: use the `_debug_search` attribute directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50648
diff
changeset
|
1496 if self._debug_search: |
49228
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1497 msg = b"DBG-DELTAS-SEARCH: DELTA: length=%d (GOOD)\n" |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1498 msg %= candidatedelta.deltalen |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1499 self._write_debug(msg) |
42466
465f2d0df9ae
deltas: accept and skip None return for delta info
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42465
diff
changeset
|
1500 nominateddeltas.append(candidatedelta) |
50649
c84cc0ac77e4
delta-fine: use the `_debug_search` attribute directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50648
diff
changeset
|
1501 elif self._debug_search: |
49228
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1502 msg = b"DBG-DELTAS-SEARCH: DELTA: length=%d (BAD)\n" |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1503 msg %= candidatedelta.deltalen |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1504 self._write_debug(msg) |
50649
c84cc0ac77e4
delta-fine: use the `_debug_search` attribute directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50648
diff
changeset
|
1505 elif self._debug_search: |
49228
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1506 msg = b"DBG-DELTAS-SEARCH: NO-DELTA\n" |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1507 self._write_debug(msg) |
35738
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1508 if nominateddeltas: |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1509 deltainfo = min(nominateddeltas, key=lambda x: x.deltalen) |
39498
04b75f3a3f2a
snapshot: add refining logic at the findeltainfo level
Boris Feld <boris.feld@octobus.net>
parents:
39497
diff
changeset
|
1510 if deltainfo is not None: |
04b75f3a3f2a
snapshot: add refining logic at the findeltainfo level
Boris Feld <boris.feld@octobus.net>
parents:
39497
diff
changeset
|
1511 candidaterevs = groups.send(deltainfo.base) |
04b75f3a3f2a
snapshot: add refining logic at the findeltainfo level
Boris Feld <boris.feld@octobus.net>
parents:
39497
diff
changeset
|
1512 else: |
04b75f3a3f2a
snapshot: add refining logic at the findeltainfo level
Boris Feld <boris.feld@octobus.net>
parents:
39497
diff
changeset
|
1513 candidaterevs = next(groups) |
35738
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1514 |
39333
5d343a24bff5
revlogdeltas: always return a delta info object in finddeltainfo
Boris Feld <boris.feld@octobus.net>
parents:
39332
diff
changeset
|
1515 if deltainfo is None: |
49226
e6b7c6fbeb48
deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
1516 dbg_type = b"full" |
51020
509f0f7fc89e
delta-computer: stop explicitly taking file handle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50929
diff
changeset
|
1517 deltainfo = self._fullsnapshotinfo(revinfo, target_rev) |
49228
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1518 elif deltainfo.snapshotdepth: # pytype: disable=attribute-error |
49226
e6b7c6fbeb48
deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
1519 dbg_type = b"snapshot" |
e6b7c6fbeb48
deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
1520 else: |
e6b7c6fbeb48
deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
1521 dbg_type = b"delta" |
e6b7c6fbeb48
deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
1522 |
49610
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
1523 if gather_debug: |
49226
e6b7c6fbeb48
deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
1524 end = util.timer() |
49775
d57b966cdeb1
delta-find: properly report full snapshot used from cache as such
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49768
diff
changeset
|
1525 if dbg_type == b'full': |
d57b966cdeb1
delta-find: properly report full snapshot used from cache as such
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49768
diff
changeset
|
1526 used_cached = ( |
d57b966cdeb1
delta-find: properly report full snapshot used from cache as such
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49768
diff
changeset
|
1527 cachedelta is not None |
d57b966cdeb1
delta-find: properly report full snapshot used from cache as such
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49768
diff
changeset
|
1528 and dbg_try_rounds == 0 |
d57b966cdeb1
delta-find: properly report full snapshot used from cache as such
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49768
diff
changeset
|
1529 and dbg_try_count == 0 |
d57b966cdeb1
delta-find: properly report full snapshot used from cache as such
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49768
diff
changeset
|
1530 and cachedelta[0] == nullrev |
d57b966cdeb1
delta-find: properly report full snapshot used from cache as such
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49768
diff
changeset
|
1531 ) |
d57b966cdeb1
delta-find: properly report full snapshot used from cache as such
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49768
diff
changeset
|
1532 else: |
d57b966cdeb1
delta-find: properly report full snapshot used from cache as such
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49768
diff
changeset
|
1533 used_cached = ( |
d57b966cdeb1
delta-find: properly report full snapshot used from cache as such
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49768
diff
changeset
|
1534 cachedelta is not None |
d57b966cdeb1
delta-find: properly report full snapshot used from cache as such
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49768
diff
changeset
|
1535 and dbg_try_rounds == 1 |
d57b966cdeb1
delta-find: properly report full snapshot used from cache as such
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49768
diff
changeset
|
1536 and dbg_try_count == 1 |
d57b966cdeb1
delta-find: properly report full snapshot used from cache as such
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49768
diff
changeset
|
1537 and deltainfo.base == cachedelta[0] |
d57b966cdeb1
delta-find: properly report full snapshot used from cache as such
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49768
diff
changeset
|
1538 ) |
50651
d1dc4a03125e
delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50650
diff
changeset
|
1539 dbg['duration'] = end - start |
d1dc4a03125e
delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50650
diff
changeset
|
1540 dbg[ |
d1dc4a03125e
delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50650
diff
changeset
|
1541 'delta-base' |
d1dc4a03125e
delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50650
diff
changeset
|
1542 ] = deltainfo.base # pytype: disable=attribute-error |
d1dc4a03125e
delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50650
diff
changeset
|
1543 dbg['search_round_count'] = dbg_try_rounds |
d1dc4a03125e
delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50650
diff
changeset
|
1544 dbg['using-cached-base'] = used_cached |
d1dc4a03125e
delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50650
diff
changeset
|
1545 dbg['delta_try_count'] = dbg_try_count |
d1dc4a03125e
delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50650
diff
changeset
|
1546 dbg['type'] = dbg_type |
49228
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1547 if ( |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1548 deltainfo.snapshotdepth # pytype: disable=attribute-error |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1549 is not None |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1550 ): |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1551 dbg[ |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1552 'snapshot-depth' |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1553 ] = deltainfo.snapshotdepth # pytype: disable=attribute-error |
49226
e6b7c6fbeb48
deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
1554 else: |
e6b7c6fbeb48
deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
1555 dbg['snapshot-depth'] = 0 |
50650
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1556 self._dbg_process_data(dbg) |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1557 return deltainfo |
49226
e6b7c6fbeb48
deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
1558 |
50651
d1dc4a03125e
delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50650
diff
changeset
|
1559 def _one_dbg_data(self): |
d1dc4a03125e
delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50650
diff
changeset
|
1560 return { |
d1dc4a03125e
delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50650
diff
changeset
|
1561 'duration': None, |
d1dc4a03125e
delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50650
diff
changeset
|
1562 'revision': None, |
d1dc4a03125e
delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50650
diff
changeset
|
1563 'delta-base': None, |
d1dc4a03125e
delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50650
diff
changeset
|
1564 'search_round_count': None, |
d1dc4a03125e
delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50650
diff
changeset
|
1565 'using-cached-base': None, |
d1dc4a03125e
delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50650
diff
changeset
|
1566 'delta_try_count': None, |
d1dc4a03125e
delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50650
diff
changeset
|
1567 'type': None, |
d1dc4a03125e
delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50650
diff
changeset
|
1568 'p1-chain-len': None, |
d1dc4a03125e
delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50650
diff
changeset
|
1569 'p2-chain-len': None, |
d1dc4a03125e
delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50650
diff
changeset
|
1570 'snapshot-depth': None, |
d1dc4a03125e
delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50650
diff
changeset
|
1571 'target-revlog': None, |
d1dc4a03125e
delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50650
diff
changeset
|
1572 } |
d1dc4a03125e
delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50650
diff
changeset
|
1573 |
50650
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1574 def _dbg_process_data(self, dbg): |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1575 if self._debug_info is not None: |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1576 self._debug_info.append(dbg) |
49610
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
1577 |
50650
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1578 if self._write_debug is not None: |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1579 msg = ( |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1580 b"DBG-DELTAS:" |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1581 b" %-12s" |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1582 b" rev=%d:" |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1583 b" delta-base=%d" |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1584 b" is-cached=%d" |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1585 b" - search-rounds=%d" |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1586 b" try-count=%d" |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1587 b" - delta-type=%-6s" |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1588 b" snap-depth=%d" |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1589 b" - p1-chain-length=%d" |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1590 b" p2-chain-length=%d" |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1591 b" - duration=%f" |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1592 b"\n" |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1593 ) |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1594 msg %= ( |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1595 dbg["target-revlog"], |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1596 dbg["revision"], |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1597 dbg["delta-base"], |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1598 dbg["using-cached-base"], |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1599 dbg["search_round_count"], |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1600 dbg["delta_try_count"], |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1601 dbg["type"], |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1602 dbg["snapshot-depth"], |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1603 dbg["p1-chain-len"], |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1604 dbg["p2-chain-len"], |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1605 dbg["duration"], |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1606 ) |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1607 self._write_debug(msg) |
47452
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1608 |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1609 |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1610 def delta_compression(default_compression_header, deltainfo): |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1611 """return (COMPRESSION_MODE, deltainfo) |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1612 |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1613 used by revlog v2+ format to dispatch between PLAIN and DEFAULT |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1614 compression. |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1615 """ |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1616 h, d = deltainfo.data |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1617 compression_mode = COMP_MODE_INLINE |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1618 if not h and not d: |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1619 # not data to store at all... declare them uncompressed |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1620 compression_mode = COMP_MODE_PLAIN |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1621 elif not h: |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1622 t = d[0:1] |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1623 if t == b'\0': |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1624 compression_mode = COMP_MODE_PLAIN |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1625 elif t == default_compression_header: |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1626 compression_mode = COMP_MODE_DEFAULT |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1627 elif h == b'u': |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1628 # we have a more efficient way to declare uncompressed |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1629 h = b'' |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1630 compression_mode = COMP_MODE_PLAIN |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1631 deltainfo = drop_u_compression(deltainfo) |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1632 return compression_mode, deltainfo |