Mercurial > hg
annotate mercurial/revlogutils/deltas.py @ 50698:f1b57672cb94
delta-find: remove dead code intended to deal with forced delta reuse
Since the case was dealt with sooner (see XXX), we no longer need to deal with
it in this part of the code.
author | Pierre-Yves David <pierre-yves.david@octobus.net> |
---|---|
date | Sun, 18 Jun 2023 00:04:53 +0200 |
parents | a41eeb877d07 |
children | d718eddf01d9 |
rev | line source |
---|---|
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
1 # revlogdeltas.py - Logic around delta computation for revlog |
8226
8b2cd04a6e97
put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents:
8225
diff
changeset
|
2 # |
46819
d4ba4d51f85f
contributor: change mentions of mpm to olivia
Raphaël Gomès <rgomes@octobus.net>
parents:
43089
diff
changeset
|
3 # Copyright 2005-2007 Olivia Mackall <olivia@selenic.com> |
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
4 # Copyright 2018 Octobus <contact@octobus.net> |
8226
8b2cd04a6e97
put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents:
8225
diff
changeset
|
5 # |
8b2cd04a6e97
put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents:
8225
diff
changeset
|
6 # This software may be used and distributed according to the terms of the |
10263 | 7 # GNU General Public License version 2 or any later version. |
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
8 """Helper class to compute deltas stored inside revlogs""" |
8227
0a9542703300
turn some comments back into module docstrings
Martin Geisler <mg@lazybytes.net>
parents:
8226
diff
changeset
|
9 |
27361
29f50344fa83
revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27251
diff
changeset
|
10 |
39493
3ca144f1c8dd
snapshot: search for unrelated but reusable full-snapshot
Boris Feld <boris.feld@octobus.net>
parents:
39492
diff
changeset
|
11 import collections |
27361
29f50344fa83
revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27251
diff
changeset
|
12 import struct |
29f50344fa83
revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27251
diff
changeset
|
13 |
29f50344fa83
revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27251
diff
changeset
|
14 # import stuff from node for others to import from revlog |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
15 from ..node import nullrev |
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
16 from ..i18n import _ |
43089
c59eb1560c44
py3: manually import getattr where it is needed
Gregory Szorc <gregory.szorc@gmail.com>
parents:
43077
diff
changeset
|
17 from ..pycompat import getattr |
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
18 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
19 from .constants import ( |
47452
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
20 COMP_MODE_DEFAULT, |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
21 COMP_MODE_INLINE, |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
22 COMP_MODE_PLAIN, |
49768
bcae90c53def
delta-find: add a delta-reuse policy that blindly accepts incoming deltas
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49681
diff
changeset
|
23 DELTA_BASE_REUSE_FORCE, |
49677
05db41701ece
find-delta: pass the cache-delta usage policy alongside the cache-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49675
diff
changeset
|
24 DELTA_BASE_REUSE_NO, |
49226
e6b7c6fbeb48
deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
25 KIND_CHANGELOG, |
e6b7c6fbeb48
deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
26 KIND_FILELOG, |
e6b7c6fbeb48
deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
27 KIND_MANIFESTLOG, |
39329
729082bb9938
revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents:
39232
diff
changeset
|
28 REVIDX_ISCENSORED, |
729082bb9938
revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents:
39232
diff
changeset
|
29 REVIDX_RAWTEXT_CHANGING_FLAGS, |
729082bb9938
revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents:
39232
diff
changeset
|
30 ) |
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
31 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
32 from ..thirdparty import attr |
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
33 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
34 from .. import ( |
27361
29f50344fa83
revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27251
diff
changeset
|
35 error, |
29f50344fa83
revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27251
diff
changeset
|
36 mdiff, |
41108
38e88450138c
delta: have a native implementation of _findsnapshot
Boris Feld <boris.feld@octobus.net>
parents:
41079
diff
changeset
|
37 util, |
27361
29f50344fa83
revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27251
diff
changeset
|
38 ) |
10913
f2ecc5733c89
revlog: factor out _maxinline global.
Greg Ward <greg-hg@gerg.ca>
parents:
10404
diff
changeset
|
39 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
40 from . import flagutil |
42992
dff95420480f
flagprocessors: make `processflagsraw` a module level function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42878
diff
changeset
|
41 |
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
42 # maximum <delta-chain-data>/<revision-text-length> ratio |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
43 LIMIT_DELTA2TEXT = 2 |
1091
d62130f99a73
Move hash function back to revlog from node
mpm@selenic.com
parents:
1089
diff
changeset
|
44 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
45 |
48946
642e31cb55f0
py3: use class X: instead of class X(object):
Gregory Szorc <gregory.szorc@gmail.com>
parents:
48875
diff
changeset
|
46 class _testrevlog: |
38637
e33f784f2a44
revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents:
38636
diff
changeset
|
47 """minimalist fake revlog to use in doctests""" |
e33f784f2a44
revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents:
38636
diff
changeset
|
48 |
40641
85b14f0dc334
doctest: add a `issnapshot` method to _testrevlog
Boris Feld <boris.feld@octobus.net>
parents:
40608
diff
changeset
|
49 def __init__(self, data, density=0.5, mingap=0, snapshot=()): |
38637
e33f784f2a44
revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents:
38636
diff
changeset
|
50 """data is an list of revision payload boundaries""" |
e33f784f2a44
revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents:
38636
diff
changeset
|
51 self._data = data |
e33f784f2a44
revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents:
38636
diff
changeset
|
52 self._srdensitythreshold = density |
e33f784f2a44
revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents:
38636
diff
changeset
|
53 self._srmingapsize = mingap |
40641
85b14f0dc334
doctest: add a `issnapshot` method to _testrevlog
Boris Feld <boris.feld@octobus.net>
parents:
40608
diff
changeset
|
54 self._snapshot = set(snapshot) |
40709
39d29542fe40
sparse-revlog: put the native implementation of slicechunktodensity to use
Boris Feld <boris.feld@octobus.net>
parents:
40657
diff
changeset
|
55 self.index = None |
38637
e33f784f2a44
revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents:
38636
diff
changeset
|
56 |
e33f784f2a44
revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents:
38636
diff
changeset
|
57 def start(self, rev): |
41079
88d813cd9acd
revlog: fix pure python slicing test when chain contains nullrev
Boris Feld <boris.feld@octobus.net>
parents:
41033
diff
changeset
|
58 if rev == nullrev: |
88d813cd9acd
revlog: fix pure python slicing test when chain contains nullrev
Boris Feld <boris.feld@octobus.net>
parents:
41033
diff
changeset
|
59 return 0 |
38637
e33f784f2a44
revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents:
38636
diff
changeset
|
60 if rev == 0: |
e33f784f2a44
revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents:
38636
diff
changeset
|
61 return 0 |
e33f784f2a44
revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents:
38636
diff
changeset
|
62 return self._data[rev - 1] |
e33f784f2a44
revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents:
38636
diff
changeset
|
63 |
e33f784f2a44
revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents:
38636
diff
changeset
|
64 def end(self, rev): |
41079
88d813cd9acd
revlog: fix pure python slicing test when chain contains nullrev
Boris Feld <boris.feld@octobus.net>
parents:
41033
diff
changeset
|
65 if rev == nullrev: |
88d813cd9acd
revlog: fix pure python slicing test when chain contains nullrev
Boris Feld <boris.feld@octobus.net>
parents:
41033
diff
changeset
|
66 return 0 |
38637
e33f784f2a44
revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents:
38636
diff
changeset
|
67 return self._data[rev] |
e33f784f2a44
revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents:
38636
diff
changeset
|
68 |
e33f784f2a44
revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents:
38636
diff
changeset
|
69 def length(self, rev): |
e33f784f2a44
revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents:
38636
diff
changeset
|
70 return self.end(rev) - self.start(rev) |
e33f784f2a44
revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents:
38636
diff
changeset
|
71 |
38718
f8762ea73e0d
sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents:
38717
diff
changeset
|
72 def __len__(self): |
f8762ea73e0d
sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents:
38717
diff
changeset
|
73 return len(self._data) |
f8762ea73e0d
sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents:
38717
diff
changeset
|
74 |
40641
85b14f0dc334
doctest: add a `issnapshot` method to _testrevlog
Boris Feld <boris.feld@octobus.net>
parents:
40608
diff
changeset
|
75 def issnapshot(self, rev): |
41079
88d813cd9acd
revlog: fix pure python slicing test when chain contains nullrev
Boris Feld <boris.feld@octobus.net>
parents:
41033
diff
changeset
|
76 if rev == nullrev: |
88d813cd9acd
revlog: fix pure python slicing test when chain contains nullrev
Boris Feld <boris.feld@octobus.net>
parents:
41033
diff
changeset
|
77 return True |
40641
85b14f0dc334
doctest: add a `issnapshot` method to _testrevlog
Boris Feld <boris.feld@octobus.net>
parents:
40608
diff
changeset
|
78 return rev in self._snapshot |
85b14f0dc334
doctest: add a `issnapshot` method to _testrevlog
Boris Feld <boris.feld@octobus.net>
parents:
40608
diff
changeset
|
79 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
80 |
40604
3ac23dad6364
sparse-revlog: drop unused deltainfo parameter from _slicechunktodensity
Boris Feld <boris.feld@octobus.net>
parents:
40603
diff
changeset
|
81 def slicechunk(revlog, revs, targetsize=None): |
34824
e2ad93bcc084
revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents:
34823
diff
changeset
|
82 """slice revs to reduce the amount of unrelated data to be read from disk. |
e2ad93bcc084
revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents:
34823
diff
changeset
|
83 |
e2ad93bcc084
revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents:
34823
diff
changeset
|
84 ``revs`` is sliced into groups that should be read in one time. |
e2ad93bcc084
revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents:
34823
diff
changeset
|
85 Assume that revs are sorted. |
38640
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
86 |
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
87 The initial chunk is sliced until the overall density (payload/chunks-span |
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
88 ratio) is above `revlog._srdensitythreshold`. No gap smaller than |
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
89 `revlog._srmingapsize` is skipped. |
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
90 |
38643
967fee55e8d9
revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents:
38642
diff
changeset
|
91 If `targetsize` is set, no chunk larger than `targetsize` will be yield. |
967fee55e8d9
revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents:
38642
diff
changeset
|
92 For consistency with other slicing choice, this limit won't go lower than |
967fee55e8d9
revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents:
38642
diff
changeset
|
93 `revlog._srmingapsize`. |
967fee55e8d9
revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents:
38642
diff
changeset
|
94 |
967fee55e8d9
revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents:
38642
diff
changeset
|
95 If individual revisions chunk are larger than this limit, they will still |
967fee55e8d9
revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents:
38642
diff
changeset
|
96 be raised individually. |
967fee55e8d9
revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents:
38642
diff
changeset
|
97 |
40642
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
98 >>> data = [ |
38640
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
99 ... 5, #00 (5) |
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
100 ... 10, #01 (5) |
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
101 ... 12, #02 (2) |
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
102 ... 12, #03 (empty) |
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
103 ... 27, #04 (15) |
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
104 ... 31, #05 (4) |
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
105 ... 31, #06 (empty) |
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
106 ... 42, #07 (11) |
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
107 ... 47, #08 (5) |
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
108 ... 47, #09 (empty) |
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
109 ... 48, #10 (1) |
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
110 ... 51, #11 (3) |
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
111 ... 74, #12 (23) |
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
112 ... 85, #13 (11) |
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
113 ... 86, #14 (1) |
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
114 ... 91, #15 (5) |
40642
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
115 ... ] |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
116 >>> revlog = _testrevlog(data, snapshot=range(16)) |
38640
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
117 |
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
118 >>> list(slicechunk(revlog, list(range(16)))) |
38640
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
119 [[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]] |
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
120 >>> list(slicechunk(revlog, [0, 15])) |
38640
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
121 [[0], [15]] |
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
122 >>> list(slicechunk(revlog, [0, 11, 15])) |
38640
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
123 [[0], [11], [15]] |
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
124 >>> list(slicechunk(revlog, [0, 11, 13, 15])) |
38640
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
125 [[0], [11, 13, 15]] |
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
126 >>> list(slicechunk(revlog, [1, 2, 3, 5, 8, 10, 11, 14])) |
38640
f62b8fb0a484
revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents:
38639
diff
changeset
|
127 [[1, 2], [5, 8, 10, 11], [14]] |
38643
967fee55e8d9
revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents:
38642
diff
changeset
|
128 |
967fee55e8d9
revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents:
38642
diff
changeset
|
129 Slicing with a maximum chunk size |
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
130 >>> list(slicechunk(revlog, [0, 11, 13, 15], targetsize=15)) |
38643
967fee55e8d9
revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents:
38642
diff
changeset
|
131 [[0], [11], [13], [15]] |
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
132 >>> list(slicechunk(revlog, [0, 11, 13, 15], targetsize=20)) |
38643
967fee55e8d9
revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents:
38642
diff
changeset
|
133 [[0], [11], [13, 15]] |
41079
88d813cd9acd
revlog: fix pure python slicing test when chain contains nullrev
Boris Feld <boris.feld@octobus.net>
parents:
41033
diff
changeset
|
134 |
88d813cd9acd
revlog: fix pure python slicing test when chain contains nullrev
Boris Feld <boris.feld@octobus.net>
parents:
41033
diff
changeset
|
135 Slicing involving nullrev |
88d813cd9acd
revlog: fix pure python slicing test when chain contains nullrev
Boris Feld <boris.feld@octobus.net>
parents:
41033
diff
changeset
|
136 >>> list(slicechunk(revlog, [-1, 0, 11, 13, 15], targetsize=20)) |
88d813cd9acd
revlog: fix pure python slicing test when chain contains nullrev
Boris Feld <boris.feld@octobus.net>
parents:
41033
diff
changeset
|
137 [[-1, 0], [11], [13, 15]] |
88d813cd9acd
revlog: fix pure python slicing test when chain contains nullrev
Boris Feld <boris.feld@octobus.net>
parents:
41033
diff
changeset
|
138 >>> list(slicechunk(revlog, [-1, 13, 15], targetsize=5)) |
88d813cd9acd
revlog: fix pure python slicing test when chain contains nullrev
Boris Feld <boris.feld@octobus.net>
parents:
41033
diff
changeset
|
139 [[-1], [13], [15]] |
34824
e2ad93bcc084
revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents:
34823
diff
changeset
|
140 """ |
38643
967fee55e8d9
revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents:
38642
diff
changeset
|
141 if targetsize is not None: |
967fee55e8d9
revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents:
38642
diff
changeset
|
142 targetsize = max(targetsize, revlog._srmingapsize) |
38718
f8762ea73e0d
sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents:
38717
diff
changeset
|
143 # targetsize should not be specified when evaluating delta candidates: |
f8762ea73e0d
sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents:
38717
diff
changeset
|
144 # * targetsize is used to ensure we stay within specification when reading, |
40709
39d29542fe40
sparse-revlog: put the native implementation of slicechunktodensity to use
Boris Feld <boris.feld@octobus.net>
parents:
40657
diff
changeset
|
145 densityslicing = getattr(revlog.index, 'slicechunktodensity', None) |
39d29542fe40
sparse-revlog: put the native implementation of slicechunktodensity to use
Boris Feld <boris.feld@octobus.net>
parents:
40657
diff
changeset
|
146 if densityslicing is None: |
39d29542fe40
sparse-revlog: put the native implementation of slicechunktodensity to use
Boris Feld <boris.feld@octobus.net>
parents:
40657
diff
changeset
|
147 densityslicing = lambda x, y, z: _slicechunktodensity(revlog, x, y, z) |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
148 for chunk in densityslicing( |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
149 revs, revlog._srdensitythreshold, revlog._srmingapsize |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
150 ): |
38643
967fee55e8d9
revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents:
38642
diff
changeset
|
151 for subchunk in _slicechunktosize(revlog, chunk, targetsize): |
967fee55e8d9
revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents:
38642
diff
changeset
|
152 yield subchunk |
38641
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
153 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
154 |
38718
f8762ea73e0d
sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents:
38717
diff
changeset
|
155 def _slicechunktosize(revlog, revs, targetsize=None): |
38642
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
156 """slice revs to match the target size |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
157 |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
158 This is intended to be used on chunk that density slicing selected by that |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
159 are still too large compared to the read garantee of revlog. This might |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
160 happens when "minimal gap size" interrupted the slicing or when chain are |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
161 built in a way that create large blocks next to each other. |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
162 |
40642
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
163 >>> data = [ |
38642
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
164 ... 3, #0 (3) |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
165 ... 5, #1 (2) |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
166 ... 6, #2 (1) |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
167 ... 8, #3 (2) |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
168 ... 8, #4 (empty) |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
169 ... 11, #5 (3) |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
170 ... 12, #6 (1) |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
171 ... 13, #7 (1) |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
172 ... 14, #8 (1) |
40642
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
173 ... ] |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
174 |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
175 == All snapshots cases == |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
176 >>> revlog = _testrevlog(data, snapshot=range(9)) |
38642
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
177 |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
178 Cases where chunk is already small enough |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
179 >>> list(_slicechunktosize(revlog, [0], 3)) |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
180 [[0]] |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
181 >>> list(_slicechunktosize(revlog, [6, 7], 3)) |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
182 [[6, 7]] |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
183 >>> list(_slicechunktosize(revlog, [0], None)) |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
184 [[0]] |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
185 >>> list(_slicechunktosize(revlog, [6, 7], None)) |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
186 [[6, 7]] |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
187 |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
188 cases where we need actual slicing |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
189 >>> list(_slicechunktosize(revlog, [0, 1], 3)) |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
190 [[0], [1]] |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
191 >>> list(_slicechunktosize(revlog, [1, 3], 3)) |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
192 [[1], [3]] |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
193 >>> list(_slicechunktosize(revlog, [1, 2, 3], 3)) |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
194 [[1, 2], [3]] |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
195 >>> list(_slicechunktosize(revlog, [3, 5], 3)) |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
196 [[3], [5]] |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
197 >>> list(_slicechunktosize(revlog, [3, 4, 5], 3)) |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
198 [[3], [5]] |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
199 >>> list(_slicechunktosize(revlog, [5, 6, 7, 8], 3)) |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
200 [[5], [6, 7, 8]] |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
201 >>> list(_slicechunktosize(revlog, [0, 1, 2, 3, 4, 5, 6, 7, 8], 3)) |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
202 [[0], [1, 2], [3], [5], [6, 7, 8]] |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
203 |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
204 Case with too large individual chunk (must return valid chunk) |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
205 >>> list(_slicechunktosize(revlog, [0, 1], 2)) |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
206 [[0], [1]] |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
207 >>> list(_slicechunktosize(revlog, [1, 3], 1)) |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
208 [[1], [3]] |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
209 >>> list(_slicechunktosize(revlog, [3, 4, 5], 2)) |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
210 [[3], [5]] |
40642
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
211 |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
212 == No Snapshot cases == |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
213 >>> revlog = _testrevlog(data) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
214 |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
215 Cases where chunk is already small enough |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
216 >>> list(_slicechunktosize(revlog, [0], 3)) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
217 [[0]] |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
218 >>> list(_slicechunktosize(revlog, [6, 7], 3)) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
219 [[6, 7]] |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
220 >>> list(_slicechunktosize(revlog, [0], None)) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
221 [[0]] |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
222 >>> list(_slicechunktosize(revlog, [6, 7], None)) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
223 [[6, 7]] |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
224 |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
225 cases where we need actual slicing |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
226 >>> list(_slicechunktosize(revlog, [0, 1], 3)) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
227 [[0], [1]] |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
228 >>> list(_slicechunktosize(revlog, [1, 3], 3)) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
229 [[1], [3]] |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
230 >>> list(_slicechunktosize(revlog, [1, 2, 3], 3)) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
231 [[1], [2, 3]] |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
232 >>> list(_slicechunktosize(revlog, [3, 5], 3)) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
233 [[3], [5]] |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
234 >>> list(_slicechunktosize(revlog, [3, 4, 5], 3)) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
235 [[3], [4, 5]] |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
236 >>> list(_slicechunktosize(revlog, [5, 6, 7, 8], 3)) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
237 [[5], [6, 7, 8]] |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
238 >>> list(_slicechunktosize(revlog, [0, 1, 2, 3, 4, 5, 6, 7, 8], 3)) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
239 [[0], [1, 2], [3], [5], [6, 7, 8]] |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
240 |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
241 Case with too large individual chunk (must return valid chunk) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
242 >>> list(_slicechunktosize(revlog, [0, 1], 2)) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
243 [[0], [1]] |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
244 >>> list(_slicechunktosize(revlog, [1, 3], 1)) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
245 [[1], [3]] |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
246 >>> list(_slicechunktosize(revlog, [3, 4, 5], 2)) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
247 [[3], [5]] |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
248 |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
249 == mixed case == |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
250 >>> revlog = _testrevlog(data, snapshot=[0, 1, 2]) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
251 >>> list(_slicechunktosize(revlog, list(range(9)), 5)) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
252 [[0, 1], [2], [3, 4, 5], [6, 7, 8]] |
38642
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
253 """ |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
254 assert targetsize is None or 0 <= targetsize |
40642
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
255 startdata = revlog.start(revs[0]) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
256 enddata = revlog.end(revs[-1]) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
257 fullspan = enddata - startdata |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
258 if targetsize is None or fullspan <= targetsize: |
38642
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
259 yield revs |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
260 return |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
261 |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
262 startrevidx = 0 |
40657
2eb48aa0acce
sparse-revlog: align endrevidx usages in the _slicechunktosize
Boris Feld <boris.feld@octobus.net>
parents:
40654
diff
changeset
|
263 endrevidx = 1 |
38642
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
264 iterrevs = enumerate(revs) |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
265 next(iterrevs) # skip first rev. |
40642
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
266 # first step: get snapshots out of the way |
38642
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
267 for idx, r in iterrevs: |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
268 span = revlog.end(r) - startdata |
40642
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
269 snapshot = revlog.issnapshot(r) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
270 if span <= targetsize and snapshot: |
40657
2eb48aa0acce
sparse-revlog: align endrevidx usages in the _slicechunktosize
Boris Feld <boris.feld@octobus.net>
parents:
40654
diff
changeset
|
271 endrevidx = idx + 1 |
38642
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
272 else: |
40657
2eb48aa0acce
sparse-revlog: align endrevidx usages in the _slicechunktosize
Boris Feld <boris.feld@octobus.net>
parents:
40654
diff
changeset
|
273 chunk = _trimchunk(revlog, revs, startrevidx, endrevidx) |
38642
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
274 if chunk: |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
275 yield chunk |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
276 startrevidx = idx |
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
277 startdata = revlog.start(r) |
40657
2eb48aa0acce
sparse-revlog: align endrevidx usages in the _slicechunktosize
Boris Feld <boris.feld@octobus.net>
parents:
40654
diff
changeset
|
278 endrevidx = idx + 1 |
40642
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
279 if not snapshot: |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
280 break |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
281 |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
282 # for the others, we use binary slicing to quickly converge toward valid |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
283 # chunks (otherwise, we might end up looking for start/end of many |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
284 # revisions). This logic is not looking for the perfect slicing point, it |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
285 # focuses on quickly converging toward valid chunks. |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
286 nbitem = len(revs) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
287 while (enddata - startdata) > targetsize: |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
288 endrevidx = nbitem |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
289 if nbitem - startrevidx <= 1: |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
290 break # protect against individual chunk larger than limit |
40642
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
291 localenddata = revlog.end(revs[endrevidx - 1]) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
292 span = localenddata - startdata |
40654
fd1d41ccbe38
sparse-revlog: use `span` variable as intended
Boris Feld <boris.feld@octobus.net>
parents:
40642
diff
changeset
|
293 while span > targetsize: |
40642
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
294 if endrevidx - startrevidx <= 1: |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
295 break # protect against individual chunk larger than limit |
40642
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
296 endrevidx -= (endrevidx - startrevidx) // 2 |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
297 localenddata = revlog.end(revs[endrevidx - 1]) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
298 span = localenddata - startdata |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
299 chunk = _trimchunk(revlog, revs, startrevidx, endrevidx) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
300 if chunk: |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
301 yield chunk |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
302 startrevidx = endrevidx |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
303 startdata = revlog.start(revs[startrevidx]) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
304 |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
305 chunk = _trimchunk(revlog, revs, startrevidx) |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
306 if chunk: |
9c3c697267db
sparse-revlog: rework the way we enforce chunk size limit
Boris Feld <boris.feld@octobus.net>
parents:
40641
diff
changeset
|
307 yield chunk |
38642
e59e27e52297
revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents:
38641
diff
changeset
|
308 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
309 |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
310 def _slicechunktodensity(revlog, revs, targetdensity=0.5, mingapsize=0): |
38641
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
311 """slice revs to reduce the amount of unrelated data to be read from disk. |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
312 |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
313 ``revs`` is sliced into groups that should be read in one time. |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
314 Assume that revs are sorted. |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
315 |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
316 The initial chunk is sliced until the overall density (payload/chunks-span |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
317 ratio) is above `targetdensity`. No gap smaller than `mingapsize` is |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
318 skipped. |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
319 |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
320 >>> revlog = _testrevlog([ |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
321 ... 5, #00 (5) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
322 ... 10, #01 (5) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
323 ... 12, #02 (2) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
324 ... 12, #03 (empty) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
325 ... 27, #04 (15) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
326 ... 31, #05 (4) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
327 ... 31, #06 (empty) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
328 ... 42, #07 (11) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
329 ... 47, #08 (5) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
330 ... 47, #09 (empty) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
331 ... 48, #10 (1) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
332 ... 51, #11 (3) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
333 ... 74, #12 (23) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
334 ... 85, #13 (11) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
335 ... 86, #14 (1) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
336 ... 91, #15 (5) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
337 ... ]) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
338 |
38655
cd1c484e31e8
revlog: adjust doctest examples to be portable to Python 3
Augie Fackler <augie@google.com>
parents:
38644
diff
changeset
|
339 >>> list(_slicechunktodensity(revlog, list(range(16)))) |
38641
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
340 [[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]] |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
341 >>> list(_slicechunktodensity(revlog, [0, 15])) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
342 [[0], [15]] |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
343 >>> list(_slicechunktodensity(revlog, [0, 11, 15])) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
344 [[0], [11], [15]] |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
345 >>> list(_slicechunktodensity(revlog, [0, 11, 13, 15])) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
346 [[0], [11, 13, 15]] |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
347 >>> list(_slicechunktodensity(revlog, [1, 2, 3, 5, 8, 10, 11, 14])) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
348 [[1, 2], [5, 8, 10, 11], [14]] |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
349 >>> list(_slicechunktodensity(revlog, [1, 2, 3, 5, 8, 10, 11, 14], |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
350 ... mingapsize=20)) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
351 [[1, 2, 3, 5, 8, 10, 11], [14]] |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
352 >>> list(_slicechunktodensity(revlog, [1, 2, 3, 5, 8, 10, 11, 14], |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
353 ... targetdensity=0.95)) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
354 [[1, 2], [5], [8, 10, 11], [14]] |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
355 >>> list(_slicechunktodensity(revlog, [1, 2, 3, 5, 8, 10, 11, 14], |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
356 ... targetdensity=0.95, mingapsize=12)) |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
357 [[1, 2], [5, 8, 10, 11], [14]] |
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
358 """ |
34824
e2ad93bcc084
revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents:
34823
diff
changeset
|
359 start = revlog.start |
e2ad93bcc084
revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents:
34823
diff
changeset
|
360 length = revlog.length |
e2ad93bcc084
revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents:
34823
diff
changeset
|
361 |
34880
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
362 if len(revs) <= 1: |
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
363 yield revs |
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
364 return |
34824
e2ad93bcc084
revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents:
34823
diff
changeset
|
365 |
40604
3ac23dad6364
sparse-revlog: drop unused deltainfo parameter from _slicechunktodensity
Boris Feld <boris.feld@octobus.net>
parents:
40603
diff
changeset
|
366 deltachainspan = segmentspan(revlog, revs) |
34880
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
367 |
38641
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
368 if deltachainspan < mingapsize: |
38635
d083ae26c325
revlog: early return in _slicechunk when span is already small enough
Boris Feld <boris.feld@octobus.net>
parents:
38634
diff
changeset
|
369 yield revs |
d083ae26c325
revlog: early return in _slicechunk when span is already small enough
Boris Feld <boris.feld@octobus.net>
parents:
38634
diff
changeset
|
370 return |
d083ae26c325
revlog: early return in _slicechunk when span is already small enough
Boris Feld <boris.feld@octobus.net>
parents:
38634
diff
changeset
|
371 |
38718
f8762ea73e0d
sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents:
38717
diff
changeset
|
372 readdata = deltachainspan |
40606
bfbfd15d65bd
sparse-revlog: fast-path before computing payload size
Boris Feld <boris.feld@octobus.net>
parents:
40605
diff
changeset
|
373 chainpayload = sum(length(r) for r in revs) |
34824
e2ad93bcc084
revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents:
34823
diff
changeset
|
374 |
34880
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
375 if deltachainspan: |
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
376 density = chainpayload / float(deltachainspan) |
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
377 else: |
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
378 density = 1.0 |
34824
e2ad93bcc084
revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents:
34823
diff
changeset
|
379 |
38641
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
380 if density >= targetdensity: |
38634
f0ea8b847831
revlog: early return in _slicechunk when density is already good
Paul Morelle <paul.morelle@octobus.net>
parents:
38632
diff
changeset
|
381 yield revs |
f0ea8b847831
revlog: early return in _slicechunk when density is already good
Paul Morelle <paul.morelle@octobus.net>
parents:
38632
diff
changeset
|
382 return |
f0ea8b847831
revlog: early return in _slicechunk when density is already good
Paul Morelle <paul.morelle@octobus.net>
parents:
38632
diff
changeset
|
383 |
34880
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
384 # Store the gaps in a heap to have them sorted by decreasing size |
40607
54de23400b2a
sparse-revlog: stop using a heap to track gaps
Boris Feld <boris.feld@octobus.net>
parents:
40606
diff
changeset
|
385 gaps = [] |
34880
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
386 prevend = None |
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
387 for i, rev in enumerate(revs): |
40604
3ac23dad6364
sparse-revlog: drop unused deltainfo parameter from _slicechunktodensity
Boris Feld <boris.feld@octobus.net>
parents:
40603
diff
changeset
|
388 revstart = start(rev) |
3ac23dad6364
sparse-revlog: drop unused deltainfo parameter from _slicechunktodensity
Boris Feld <boris.feld@octobus.net>
parents:
40603
diff
changeset
|
389 revlen = length(rev) |
34824
e2ad93bcc084
revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents:
34823
diff
changeset
|
390 |
34898
1bde8e8e5de0
sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents:
34881
diff
changeset
|
391 # Skip empty revisions to form larger holes |
1bde8e8e5de0
sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents:
34881
diff
changeset
|
392 if revlen == 0: |
1bde8e8e5de0
sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents:
34881
diff
changeset
|
393 continue |
1bde8e8e5de0
sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents:
34881
diff
changeset
|
394 |
34880
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
395 if prevend is not None: |
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
396 gapsize = revstart - prevend |
34881
8c9b08a0c48c
sparse-read: skip gaps too small to be worth splitting
Paul Morelle <paul.morelle@octobus.net>
parents:
34880
diff
changeset
|
397 # only consider holes that are large enough |
38641
feba6be0941b
revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents:
38640
diff
changeset
|
398 if gapsize > mingapsize: |
40607
54de23400b2a
sparse-revlog: stop using a heap to track gaps
Boris Feld <boris.feld@octobus.net>
parents:
40606
diff
changeset
|
399 gaps.append((gapsize, i)) |
34880
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
400 |
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
401 prevend = revstart + revlen |
40607
54de23400b2a
sparse-revlog: stop using a heap to track gaps
Boris Feld <boris.feld@octobus.net>
parents:
40606
diff
changeset
|
402 # sort the gaps to pop them from largest to small |
54de23400b2a
sparse-revlog: stop using a heap to track gaps
Boris Feld <boris.feld@octobus.net>
parents:
40606
diff
changeset
|
403 gaps.sort() |
34880
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
404 |
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
405 # Collect the indices of the largest holes until the density is acceptable |
40608
526ee887c4d5
sparse-revlog: stop using a heap to track selected gap
Boris Feld <boris.feld@octobus.net>
parents:
40607
diff
changeset
|
406 selected = [] |
40607
54de23400b2a
sparse-revlog: stop using a heap to track gaps
Boris Feld <boris.feld@octobus.net>
parents:
40606
diff
changeset
|
407 while gaps and density < targetdensity: |
54de23400b2a
sparse-revlog: stop using a heap to track gaps
Boris Feld <boris.feld@octobus.net>
parents:
40606
diff
changeset
|
408 gapsize, gapidx = gaps.pop() |
34880
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
409 |
40608
526ee887c4d5
sparse-revlog: stop using a heap to track selected gap
Boris Feld <boris.feld@octobus.net>
parents:
40607
diff
changeset
|
410 selected.append(gapidx) |
34824
e2ad93bcc084
revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents:
34823
diff
changeset
|
411 |
34880
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
412 # the gap sizes are stored as negatives to be sorted decreasingly |
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
413 # by the heap |
40607
54de23400b2a
sparse-revlog: stop using a heap to track gaps
Boris Feld <boris.feld@octobus.net>
parents:
40606
diff
changeset
|
414 readdata -= gapsize |
34880
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
415 if readdata > 0: |
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
416 density = chainpayload / float(readdata) |
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
417 else: |
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
418 density = 1.0 |
40608
526ee887c4d5
sparse-revlog: stop using a heap to track selected gap
Boris Feld <boris.feld@octobus.net>
parents:
40607
diff
changeset
|
419 selected.sort() |
34824
e2ad93bcc084
revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents:
34823
diff
changeset
|
420 |
34880
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
421 # Cut the revs at collected indices |
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
422 previdx = 0 |
40608
526ee887c4d5
sparse-revlog: stop using a heap to track selected gap
Boris Feld <boris.feld@octobus.net>
parents:
40607
diff
changeset
|
423 for idx in selected: |
34898
1bde8e8e5de0
sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents:
34881
diff
changeset
|
424 |
1bde8e8e5de0
sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents:
34881
diff
changeset
|
425 chunk = _trimchunk(revlog, revs, previdx, idx) |
1bde8e8e5de0
sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents:
34881
diff
changeset
|
426 if chunk: |
1bde8e8e5de0
sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents:
34881
diff
changeset
|
427 yield chunk |
1bde8e8e5de0
sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents:
34881
diff
changeset
|
428 |
34880
9e18ab7f7240
sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents:
34825
diff
changeset
|
429 previdx = idx |
34898
1bde8e8e5de0
sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents:
34881
diff
changeset
|
430 |
1bde8e8e5de0
sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents:
34881
diff
changeset
|
431 chunk = _trimchunk(revlog, revs, previdx) |
1bde8e8e5de0
sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents:
34881
diff
changeset
|
432 if chunk: |
1bde8e8e5de0
sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents:
34881
diff
changeset
|
433 yield chunk |
34824
e2ad93bcc084
revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents:
34823
diff
changeset
|
434 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
435 |
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
436 def _trimchunk(revlog, revs, startidx, endidx=None): |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
437 """returns revs[startidx:endidx] without empty trailing revs |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
438 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
439 Doctest Setup |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
440 >>> revlog = _testrevlog([ |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
441 ... 5, #0 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
442 ... 10, #1 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
443 ... 12, #2 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
444 ... 12, #3 (empty) |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
445 ... 17, #4 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
446 ... 21, #5 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
447 ... 21, #6 (empty) |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
448 ... ]) |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
449 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
450 Contiguous cases: |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
451 >>> _trimchunk(revlog, [0, 1, 2, 3, 4, 5, 6], 0) |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
452 [0, 1, 2, 3, 4, 5] |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
453 >>> _trimchunk(revlog, [0, 1, 2, 3, 4, 5, 6], 0, 5) |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
454 [0, 1, 2, 3, 4] |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
455 >>> _trimchunk(revlog, [0, 1, 2, 3, 4, 5, 6], 0, 4) |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
456 [0, 1, 2] |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
457 >>> _trimchunk(revlog, [0, 1, 2, 3, 4, 5, 6], 2, 4) |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
458 [2] |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
459 >>> _trimchunk(revlog, [0, 1, 2, 3, 4, 5, 6], 3) |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
460 [3, 4, 5] |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
461 >>> _trimchunk(revlog, [0, 1, 2, 3, 4, 5, 6], 3, 5) |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
462 [3, 4] |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
463 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
464 Discontiguous cases: |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
465 >>> _trimchunk(revlog, [1, 3, 5, 6], 0) |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
466 [1, 3, 5] |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
467 >>> _trimchunk(revlog, [1, 3, 5, 6], 0, 2) |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
468 [1] |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
469 >>> _trimchunk(revlog, [1, 3, 5, 6], 1, 3) |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
470 [3, 5] |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
471 >>> _trimchunk(revlog, [1, 3, 5, 6], 1) |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
472 [3, 5] |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
473 """ |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
474 length = revlog.length |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
475 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
476 if endidx is None: |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
477 endidx = len(revs) |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
478 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
479 # If we have a non-emtpy delta candidate, there are nothing to trim |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
480 if revs[endidx - 1] < len(revlog): |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
481 # Trim empty revs at the end, except the very first revision of a chain |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
482 while ( |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
483 endidx > 1 and endidx > startidx and length(revs[endidx - 1]) == 0 |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
484 ): |
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
485 endidx -= 1 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
486 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
487 return revs[startidx:endidx] |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
488 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
489 |
40605
a32ccd32982b
sparse-revlog: drop unused deltainfo parameter from segmentspan
Boris Feld <boris.feld@octobus.net>
parents:
40604
diff
changeset
|
490 def segmentspan(revlog, revs): |
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
491 """Get the byte span of a segment of revisions |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
492 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
493 revs is a sorted array of revision numbers |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
494 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
495 >>> revlog = _testrevlog([ |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
496 ... 5, #0 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
497 ... 10, #1 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
498 ... 12, #2 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
499 ... 12, #3 (empty) |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
500 ... 17, #4 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
501 ... ]) |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
502 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
503 >>> segmentspan(revlog, [0, 1, 2, 3, 4]) |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
504 17 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
505 >>> segmentspan(revlog, [0, 4]) |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
506 17 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
507 >>> segmentspan(revlog, [3, 4]) |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
508 5 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
509 >>> segmentspan(revlog, [1, 2, 3,]) |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
510 7 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
511 >>> segmentspan(revlog, [1, 3]) |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
512 7 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
513 """ |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
514 if not revs: |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
515 return 0 |
40605
a32ccd32982b
sparse-revlog: drop unused deltainfo parameter from segmentspan
Boris Feld <boris.feld@octobus.net>
parents:
40604
diff
changeset
|
516 end = revlog.end(revs[-1]) |
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
517 return end - revlog.start(revs[0]) |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
518 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
519 |
39331
fd0150a3c2fe
revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents:
39330
diff
changeset
|
520 def _textfromdelta(fh, revlog, baserev, delta, p1, p2, flags, expectednode): |
fd0150a3c2fe
revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents:
39330
diff
changeset
|
521 """build full text from a (base, delta) pair and other metadata""" |
fd0150a3c2fe
revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents:
39330
diff
changeset
|
522 # special case deltas which replace entire base; no need to decode |
fd0150a3c2fe
revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents:
39330
diff
changeset
|
523 # base revision. this neatly avoids censored bases, which throw when |
fd0150a3c2fe
revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents:
39330
diff
changeset
|
524 # they're decoded. |
43077
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
525 hlen = struct.calcsize(b">lll") |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
526 if delta[:hlen] == mdiff.replacediffheader( |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
527 revlog.rawsize(baserev), len(delta) - hlen |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
528 ): |
39331
fd0150a3c2fe
revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents:
39330
diff
changeset
|
529 fulltext = delta[hlen:] |
fd0150a3c2fe
revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents:
39330
diff
changeset
|
530 else: |
fd0150a3c2fe
revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents:
39330
diff
changeset
|
531 # deltabase is rawtext before changed by flag processors, which is |
fd0150a3c2fe
revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents:
39330
diff
changeset
|
532 # equivalent to non-raw text |
48529
c514936d92b4
revlog: remove deprecated APIs
Raphaël Gomès <rgomes@octobus.net>
parents:
47457
diff
changeset
|
533 basetext = revlog.revision(baserev, _df=fh) |
39331
fd0150a3c2fe
revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents:
39330
diff
changeset
|
534 fulltext = mdiff.patch(basetext, delta) |
fd0150a3c2fe
revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents:
39330
diff
changeset
|
535 |
fd0150a3c2fe
revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents:
39330
diff
changeset
|
536 try: |
42992
dff95420480f
flagprocessors: make `processflagsraw` a module level function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42878
diff
changeset
|
537 validatehash = flagutil.processflagsraw(revlog, fulltext, flags) |
39331
fd0150a3c2fe
revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents:
39330
diff
changeset
|
538 if validatehash: |
fd0150a3c2fe
revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents:
39330
diff
changeset
|
539 revlog.checkhash(fulltext, expectednode, p1=p1, p2=p2) |
fd0150a3c2fe
revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents:
39330
diff
changeset
|
540 if flags & REVIDX_ISCENSORED: |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
541 raise error.StorageError( |
43077
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
542 _(b'node %s is not censored') % expectednode |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
543 ) |
39774
4a2466b2a434
revlog: drop some more error aliases (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39773
diff
changeset
|
544 except error.CensoredNodeError: |
39331
fd0150a3c2fe
revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents:
39330
diff
changeset
|
545 # must pass the censored index flag to add censored revisions |
fd0150a3c2fe
revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents:
39330
diff
changeset
|
546 if not flags & REVIDX_ISCENSORED: |
fd0150a3c2fe
revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents:
39330
diff
changeset
|
547 raise |
fd0150a3c2fe
revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents:
39330
diff
changeset
|
548 return fulltext |
fd0150a3c2fe
revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents:
39330
diff
changeset
|
549 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
550 |
35638
edc9330acac1
revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents:
35637
diff
changeset
|
551 @attr.s(slots=True, frozen=True) |
48946
642e31cb55f0
py3: use class X: instead of class X(object):
Gregory Szorc <gregory.szorc@gmail.com>
parents:
48875
diff
changeset
|
552 class _deltainfo: |
35638
edc9330acac1
revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents:
35637
diff
changeset
|
553 distance = attr.ib() |
edc9330acac1
revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents:
35637
diff
changeset
|
554 deltalen = attr.ib() |
edc9330acac1
revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents:
35637
diff
changeset
|
555 data = attr.ib() |
edc9330acac1
revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents:
35637
diff
changeset
|
556 base = attr.ib() |
edc9330acac1
revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents:
35637
diff
changeset
|
557 chainbase = attr.ib() |
edc9330acac1
revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents:
35637
diff
changeset
|
558 chainlen = attr.ib() |
edc9330acac1
revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents:
35637
diff
changeset
|
559 compresseddeltalen = attr.ib() |
39154
e0da43e2f71f
revlog: compute snapshot depth on delta info
Boris Feld <boris.feld@octobus.net>
parents:
39152
diff
changeset
|
560 snapshotdepth = attr.ib() |
35638
edc9330acac1
revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents:
35637
diff
changeset
|
561 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
562 |
47253
b876f0bf7366
revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46819
diff
changeset
|
563 def drop_u_compression(delta): |
b876f0bf7366
revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46819
diff
changeset
|
564 """turn into a "u" (no-compression) into no-compression without header |
b876f0bf7366
revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46819
diff
changeset
|
565 |
b876f0bf7366
revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46819
diff
changeset
|
566 This is useful for revlog format that has better compression method. |
b876f0bf7366
revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46819
diff
changeset
|
567 """ |
b876f0bf7366
revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46819
diff
changeset
|
568 assert delta.data[0] == b'u', delta.data[0] |
b876f0bf7366
revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46819
diff
changeset
|
569 return _deltainfo( |
b876f0bf7366
revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46819
diff
changeset
|
570 delta.distance, |
b876f0bf7366
revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46819
diff
changeset
|
571 delta.deltalen - 1, |
b876f0bf7366
revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46819
diff
changeset
|
572 (b'', delta.data[1]), |
b876f0bf7366
revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46819
diff
changeset
|
573 delta.base, |
b876f0bf7366
revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46819
diff
changeset
|
574 delta.chainbase, |
b876f0bf7366
revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46819
diff
changeset
|
575 delta.chainlen, |
b876f0bf7366
revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46819
diff
changeset
|
576 delta.compresseddeltalen, |
b876f0bf7366
revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46819
diff
changeset
|
577 delta.snapshotdepth, |
b876f0bf7366
revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46819
diff
changeset
|
578 ) |
b876f0bf7366
revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46819
diff
changeset
|
579 |
b876f0bf7366
revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46819
diff
changeset
|
580 |
49674
5af4a0a73e4c
find-delta: rename _isgooddeltainfo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49657
diff
changeset
|
581 def is_good_delta_info(revlog, deltainfo, revinfo): |
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
582 """Returns True if the given delta is good. Good means that it is within |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
583 the disk span, disk size, and chain length bounds that we know to be |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
584 performant.""" |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
585 if deltainfo is None: |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
586 return False |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
587 |
50698
f1b57672cb94
delta-find: remove dead code intended to deal with forced delta reuse
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50686
diff
changeset
|
588 # the DELTA_BASE_REUSE_FORCE case should have been taken care of sooner so |
f1b57672cb94
delta-find: remove dead code intended to deal with forced delta reuse
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50686
diff
changeset
|
589 # we should never end up asking such question. Adding the assert as a |
f1b57672cb94
delta-find: remove dead code intended to deal with forced delta reuse
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50686
diff
changeset
|
590 # safe-guard to detect anything that would be fishy in this regard. |
f1b57672cb94
delta-find: remove dead code intended to deal with forced delta reuse
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50686
diff
changeset
|
591 assert ( |
f1b57672cb94
delta-find: remove dead code intended to deal with forced delta reuse
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50686
diff
changeset
|
592 revinfo.cachedelta is None |
f1b57672cb94
delta-find: remove dead code intended to deal with forced delta reuse
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50686
diff
changeset
|
593 or revinfo.cachedelta[2] != DELTA_BASE_REUSE_FORCE |
f1b57672cb94
delta-find: remove dead code intended to deal with forced delta reuse
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50686
diff
changeset
|
594 or not revlog._generaldelta |
f1b57672cb94
delta-find: remove dead code intended to deal with forced delta reuse
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50686
diff
changeset
|
595 ) |
49768
bcae90c53def
delta-find: add a delta-reuse policy that blindly accepts incoming deltas
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49681
diff
changeset
|
596 |
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
597 # - 'deltainfo.distance' is the distance from the base revision -- |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
598 # bounding it limits the amount of I/O we need to do. |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
599 # - 'deltainfo.compresseddeltalen' is the sum of the total size of |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
600 # deltas we need to apply -- bounding it limits the amount of CPU |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
601 # we consume. |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
602 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
603 textlen = revinfo.textlen |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
604 defaultmax = textlen * 4 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
605 maxdist = revlog._maxdeltachainspan |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
606 if not maxdist: |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
607 maxdist = deltainfo.distance # ensure the conditional pass |
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
608 maxdist = max(maxdist, defaultmax) |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
609 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
610 # Bad delta from read span: |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
611 # |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
612 # If the span of data read is larger than the maximum allowed. |
40603
2f7e531ef3e7
sparse-revlog: skip the span check in the sparse-revlog case
Boris Feld <boris.feld@octobus.net>
parents:
40451
diff
changeset
|
613 # |
2f7e531ef3e7
sparse-revlog: skip the span check in the sparse-revlog case
Boris Feld <boris.feld@octobus.net>
parents:
40451
diff
changeset
|
614 # In the sparse-revlog case, we rely on the associated "sparse reading" |
2f7e531ef3e7
sparse-revlog: skip the span check in the sparse-revlog case
Boris Feld <boris.feld@octobus.net>
parents:
40451
diff
changeset
|
615 # to avoid issue related to the span of data. In theory, it would be |
2f7e531ef3e7
sparse-revlog: skip the span check in the sparse-revlog case
Boris Feld <boris.feld@octobus.net>
parents:
40451
diff
changeset
|
616 # possible to build pathological revlog where delta pattern would lead |
2f7e531ef3e7
sparse-revlog: skip the span check in the sparse-revlog case
Boris Feld <boris.feld@octobus.net>
parents:
40451
diff
changeset
|
617 # to too many reads. However, they do not happen in practice at all. So |
2f7e531ef3e7
sparse-revlog: skip the span check in the sparse-revlog case
Boris Feld <boris.feld@octobus.net>
parents:
40451
diff
changeset
|
618 # we skip the span check entirely. |
2f7e531ef3e7
sparse-revlog: skip the span check in the sparse-revlog case
Boris Feld <boris.feld@octobus.net>
parents:
40451
diff
changeset
|
619 if not revlog._sparserevlog and maxdist < deltainfo.distance: |
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
620 return False |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
621 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
622 # Bad delta from new delta size: |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
623 # |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
624 # If the delta size is larger than the target text, storing the |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
625 # delta will be inefficient. |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
626 if textlen < deltainfo.deltalen: |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
627 return False |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
628 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
629 # Bad delta from cumulated payload size: |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
630 # |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
631 # If the sum of delta get larger than K * target text length. |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
632 if textlen * LIMIT_DELTA2TEXT < deltainfo.compresseddeltalen: |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
633 return False |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
634 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
635 # Bad delta from chain length: |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
636 # |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
637 # If the number of delta in the chain gets too high. |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
638 if revlog._maxchainlen and revlog._maxchainlen < deltainfo.chainlen: |
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
639 return False |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
640 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
641 # bad delta from intermediate snapshot size limit |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
642 # |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
643 # If an intermediate snapshot size is higher than the limit. The |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
644 # limit exist to prevent endless chain of intermediate delta to be |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
645 # created. |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
646 if ( |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
647 deltainfo.snapshotdepth is not None |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
648 and (textlen >> deltainfo.snapshotdepth) < deltainfo.deltalen |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
649 ): |
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
650 return False |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
651 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
652 # bad delta if new intermediate snapshot is larger than the previous |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
653 # snapshot |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
654 if ( |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
655 deltainfo.snapshotdepth |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
656 and revlog.length(deltainfo.base) < deltainfo.deltalen |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
657 ): |
39330
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
658 return False |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
659 |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
660 return True |
655b5b465953
revlog: split functionality related to deltas computation in a new module
Boris Feld <boris.feld@octobus.net>
parents:
39329
diff
changeset
|
661 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
662 |
40978
42f59d3f714d
delta: exclude base candidate much smaller than the target
Boris Feld <boris.feld@octobus.net>
parents:
40957
diff
changeset
|
663 # If a revision's full text is that much bigger than a base candidate full |
42f59d3f714d
delta: exclude base candidate much smaller than the target
Boris Feld <boris.feld@octobus.net>
parents:
40957
diff
changeset
|
664 # text's, it is very unlikely that it will produce a valid delta. We no longer |
42f59d3f714d
delta: exclude base candidate much smaller than the target
Boris Feld <boris.feld@octobus.net>
parents:
40957
diff
changeset
|
665 # consider these candidates. |
41033
b373477948df
revlog: limit base to rev size ratio to 500 instead of 50
Boris Feld <boris.feld@octobus.net>
parents:
40979
diff
changeset
|
666 LIMIT_BASE2TEXT = 500 |
40978
42f59d3f714d
delta: exclude base candidate much smaller than the target
Boris Feld <boris.feld@octobus.net>
parents:
40957
diff
changeset
|
667 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
668 |
49611
2afee2176775
delta-find: expand a function definition and call before extendin it
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49610
diff
changeset
|
669 def _candidategroups( |
2afee2176775
delta-find: expand a function definition and call before extendin it
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49610
diff
changeset
|
670 revlog, |
2afee2176775
delta-find: expand a function definition and call before extendin it
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49610
diff
changeset
|
671 textlen, |
2afee2176775
delta-find: expand a function definition and call before extendin it
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49610
diff
changeset
|
672 p1, |
2afee2176775
delta-find: expand a function definition and call before extendin it
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49610
diff
changeset
|
673 p2, |
2afee2176775
delta-find: expand a function definition and call before extendin it
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49610
diff
changeset
|
674 cachedelta, |
49612
e706bb41fdb3
delta-find: move pre-filtering with other pre-filtering logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49611
diff
changeset
|
675 excluded_bases=None, |
e706bb41fdb3
delta-find: move pre-filtering with other pre-filtering logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49611
diff
changeset
|
676 target_rev=None, |
49681
c261a628e525
delta-find: use a single snapshot cache when applying a group to an object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49680
diff
changeset
|
677 snapshot_cache=None, |
49611
2afee2176775
delta-find: expand a function definition and call before extendin it
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49610
diff
changeset
|
678 ): |
39336
1c6ff52fe9cf
revlogdeltas: split candidate groups selection from the filtering logic
Boris Feld <boris.feld@octobus.net>
parents:
39335
diff
changeset
|
679 """Provides group of revision to be tested as delta base |
1c6ff52fe9cf
revlogdeltas: split candidate groups selection from the filtering logic
Boris Feld <boris.feld@octobus.net>
parents:
39335
diff
changeset
|
680 |
1c6ff52fe9cf
revlogdeltas: split candidate groups selection from the filtering logic
Boris Feld <boris.feld@octobus.net>
parents:
39335
diff
changeset
|
681 This top level function focus on emitting groups with unique and worthwhile |
1c6ff52fe9cf
revlogdeltas: split candidate groups selection from the filtering logic
Boris Feld <boris.feld@octobus.net>
parents:
39335
diff
changeset
|
682 content. See _raw_candidate_groups for details about the group order. |
39334
507f5b1dd7c8
revlogdeltas: extract _getcandidaterevs in a function
Boris Feld <boris.feld@octobus.net>
parents:
39333
diff
changeset
|
683 """ |
39336
1c6ff52fe9cf
revlogdeltas: split candidate groups selection from the filtering logic
Boris Feld <boris.feld@octobus.net>
parents:
39335
diff
changeset
|
684 # should we try to build a delta? |
1c6ff52fe9cf
revlogdeltas: split candidate groups selection from the filtering logic
Boris Feld <boris.feld@octobus.net>
parents:
39335
diff
changeset
|
685 if not (len(revlog) and revlog._storedeltachains): |
39497
5b308a4e6d03
snapshot: use None as a stop value when looking for a good delta
Boris Feld <boris.feld@octobus.net>
parents:
39496
diff
changeset
|
686 yield None |
39336
1c6ff52fe9cf
revlogdeltas: split candidate groups selection from the filtering logic
Boris Feld <boris.feld@octobus.net>
parents:
39335
diff
changeset
|
687 return |
1c6ff52fe9cf
revlogdeltas: split candidate groups selection from the filtering logic
Boris Feld <boris.feld@octobus.net>
parents:
39335
diff
changeset
|
688 |
50354
ca1522fe4ec8
delta-find: assume the target-rev if not specified
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49776
diff
changeset
|
689 if target_rev is None: |
ca1522fe4ec8
delta-find: assume the target-rev if not specified
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49776
diff
changeset
|
690 target_rev = len(revlog) |
ca1522fe4ec8
delta-find: assume the target-rev if not specified
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49776
diff
changeset
|
691 |
50355
0232571255d3
delta-find: never do anything fancy when general delta is off
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50354
diff
changeset
|
692 if not revlog._generaldelta: |
0232571255d3
delta-find: never do anything fancy when general delta is off
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50354
diff
changeset
|
693 # before general delta, there is only one possible delta base |
0232571255d3
delta-find: never do anything fancy when general delta is off
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50354
diff
changeset
|
694 yield (target_rev - 1,) |
0232571255d3
delta-find: never do anything fancy when general delta is off
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50354
diff
changeset
|
695 yield None |
0232571255d3
delta-find: never do anything fancy when general delta is off
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50354
diff
changeset
|
696 return |
0232571255d3
delta-find: never do anything fancy when general delta is off
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50354
diff
changeset
|
697 |
50698
f1b57672cb94
delta-find: remove dead code intended to deal with forced delta reuse
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50686
diff
changeset
|
698 # the DELTA_BASE_REUSE_FORCE case should have been taken care of sooner so |
f1b57672cb94
delta-find: remove dead code intended to deal with forced delta reuse
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50686
diff
changeset
|
699 # we should never end up asking such question. Adding the assert as a |
f1b57672cb94
delta-find: remove dead code intended to deal with forced delta reuse
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50686
diff
changeset
|
700 # safe-guard to detect anything that would be fishy in this regard. |
f1b57672cb94
delta-find: remove dead code intended to deal with forced delta reuse
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50686
diff
changeset
|
701 assert ( |
f1b57672cb94
delta-find: remove dead code intended to deal with forced delta reuse
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50686
diff
changeset
|
702 cachedelta is None |
f1b57672cb94
delta-find: remove dead code intended to deal with forced delta reuse
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50686
diff
changeset
|
703 or cachedelta[2] != DELTA_BASE_REUSE_FORCE |
f1b57672cb94
delta-find: remove dead code intended to deal with forced delta reuse
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50686
diff
changeset
|
704 or not revlog._generaldelta |
f1b57672cb94
delta-find: remove dead code intended to deal with forced delta reuse
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50686
diff
changeset
|
705 ) |
49776
acdb9a15137c
bundle: when forcing acceptance of incoming delta also accept snapshot
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49775
diff
changeset
|
706 |
39337
37957e07138c
revlogdeltas: move finddeltainfo filtering inside _candidategroups
Boris Feld <boris.feld@octobus.net>
parents:
39336
diff
changeset
|
707 deltalength = revlog.length |
37957e07138c
revlogdeltas: move finddeltainfo filtering inside _candidategroups
Boris Feld <boris.feld@octobus.net>
parents:
39336
diff
changeset
|
708 deltaparent = revlog.deltaparent |
40978
42f59d3f714d
delta: exclude base candidate much smaller than the target
Boris Feld <boris.feld@octobus.net>
parents:
40957
diff
changeset
|
709 sparse = revlog._sparserevlog |
39498
04b75f3a3f2a
snapshot: add refining logic at the findeltainfo level
Boris Feld <boris.feld@octobus.net>
parents:
39497
diff
changeset
|
710 good = None |
39337
37957e07138c
revlogdeltas: move finddeltainfo filtering inside _candidategroups
Boris Feld <boris.feld@octobus.net>
parents:
39336
diff
changeset
|
711 |
37957e07138c
revlogdeltas: move finddeltainfo filtering inside _candidategroups
Boris Feld <boris.feld@octobus.net>
parents:
39336
diff
changeset
|
712 deltas_limit = textlen * LIMIT_DELTA2TEXT |
49657
f5f113f1b011
delta-find: add a way to control the number of bases tested at the same time
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49615
diff
changeset
|
713 group_chunk_size = revlog._candidate_group_chunk_size |
39337
37957e07138c
revlogdeltas: move finddeltainfo filtering inside _candidategroups
Boris Feld <boris.feld@octobus.net>
parents:
39336
diff
changeset
|
714 |
42057
566daffc607d
cleanup: use set literals where possible
Martin von Zweigbergk <martinvonz@google.com>
parents:
41819
diff
changeset
|
715 tested = {nullrev} |
49610
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
716 candidates = _refinedgroups( |
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
717 revlog, |
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
718 p1, |
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
719 p2, |
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
720 cachedelta, |
49681
c261a628e525
delta-find: use a single snapshot cache when applying a group to an object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49680
diff
changeset
|
721 snapshot_cache=snapshot_cache, |
49610
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
722 ) |
39499
51cec7fb672e
snapshot: also use None as a stop value for `_refinegroup`
Boris Feld <boris.feld@octobus.net>
parents:
39498
diff
changeset
|
723 while True: |
39500
cc85ebb68ff9
snapshot: turn _refinedgroups into a coroutine
Boris Feld <boris.feld@octobus.net>
parents:
39499
diff
changeset
|
724 temptative = candidates.send(good) |
39499
51cec7fb672e
snapshot: also use None as a stop value for `_refinegroup`
Boris Feld <boris.feld@octobus.net>
parents:
39498
diff
changeset
|
725 if temptative is None: |
51cec7fb672e
snapshot: also use None as a stop value for `_refinegroup`
Boris Feld <boris.feld@octobus.net>
parents:
39498
diff
changeset
|
726 break |
39337
37957e07138c
revlogdeltas: move finddeltainfo filtering inside _candidategroups
Boris Feld <boris.feld@octobus.net>
parents:
39336
diff
changeset
|
727 group = [] |
37957e07138c
revlogdeltas: move finddeltainfo filtering inside _candidategroups
Boris Feld <boris.feld@octobus.net>
parents:
39336
diff
changeset
|
728 for rev in temptative: |
37957e07138c
revlogdeltas: move finddeltainfo filtering inside _candidategroups
Boris Feld <boris.feld@octobus.net>
parents:
39336
diff
changeset
|
729 # skip over empty delta (no need to include them in a chain) |
50357
8038493eb31a
delta-find: simply code that is now never invoqued without general delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50356
diff
changeset
|
730 while not (rev == nullrev or rev in tested or deltalength(rev)): |
39594
bdb41eaa8b59
snapshot: fix line order when skipping over empty deltas
Boris Feld <boris.feld@octobus.net>
parents:
39505
diff
changeset
|
731 tested.add(rev) |
39337
37957e07138c
revlogdeltas: move finddeltainfo filtering inside _candidategroups
Boris Feld <boris.feld@octobus.net>
parents:
39336
diff
changeset
|
732 rev = deltaparent(rev) |
40957
f960c51eebf3
delta: filter nullrev out first
Boris Feld <boris.feld@octobus.net>
parents:
40709
diff
changeset
|
733 # no need to try a delta against nullrev, this will be done as a |
f960c51eebf3
delta: filter nullrev out first
Boris Feld <boris.feld@octobus.net>
parents:
40709
diff
changeset
|
734 # last resort. |
f960c51eebf3
delta: filter nullrev out first
Boris Feld <boris.feld@octobus.net>
parents:
40709
diff
changeset
|
735 if rev == nullrev: |
f960c51eebf3
delta: filter nullrev out first
Boris Feld <boris.feld@octobus.net>
parents:
40709
diff
changeset
|
736 continue |
39337
37957e07138c
revlogdeltas: move finddeltainfo filtering inside _candidategroups
Boris Feld <boris.feld@octobus.net>
parents:
39336
diff
changeset
|
737 # filter out revision we tested already |
37957e07138c
revlogdeltas: move finddeltainfo filtering inside _candidategroups
Boris Feld <boris.feld@octobus.net>
parents:
39336
diff
changeset
|
738 if rev in tested: |
37957e07138c
revlogdeltas: move finddeltainfo filtering inside _candidategroups
Boris Feld <boris.feld@octobus.net>
parents:
39336
diff
changeset
|
739 continue |
49768
bcae90c53def
delta-find: add a delta-reuse policy that blindly accepts incoming deltas
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49681
diff
changeset
|
740 |
49612
e706bb41fdb3
delta-find: move pre-filtering with other pre-filtering logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49611
diff
changeset
|
741 # an higher authority deamed the base unworthy (e.g. censored) |
e706bb41fdb3
delta-find: move pre-filtering with other pre-filtering logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49611
diff
changeset
|
742 if excluded_bases is not None and rev in excluded_bases: |
49615
4956942c0416
delta-find: adjust the moment when we mark something as "tested"
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49614
diff
changeset
|
743 tested.add(rev) |
49612
e706bb41fdb3
delta-find: move pre-filtering with other pre-filtering logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49611
diff
changeset
|
744 continue |
e706bb41fdb3
delta-find: move pre-filtering with other pre-filtering logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49611
diff
changeset
|
745 # We are in some recomputation cases and that rev is too high in |
e706bb41fdb3
delta-find: move pre-filtering with other pre-filtering logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49611
diff
changeset
|
746 # the revlog |
e706bb41fdb3
delta-find: move pre-filtering with other pre-filtering logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49611
diff
changeset
|
747 if target_rev is not None and rev >= target_rev: |
49615
4956942c0416
delta-find: adjust the moment when we mark something as "tested"
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49614
diff
changeset
|
748 tested.add(rev) |
49612
e706bb41fdb3
delta-find: move pre-filtering with other pre-filtering logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49611
diff
changeset
|
749 continue |
39337
37957e07138c
revlogdeltas: move finddeltainfo filtering inside _candidategroups
Boris Feld <boris.feld@octobus.net>
parents:
39336
diff
changeset
|
750 # filter out delta base that will never produce good delta |
37957e07138c
revlogdeltas: move finddeltainfo filtering inside _candidategroups
Boris Feld <boris.feld@octobus.net>
parents:
39336
diff
changeset
|
751 if deltas_limit < revlog.length(rev): |
49615
4956942c0416
delta-find: adjust the moment when we mark something as "tested"
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49614
diff
changeset
|
752 tested.add(rev) |
39337
37957e07138c
revlogdeltas: move finddeltainfo filtering inside _candidategroups
Boris Feld <boris.feld@octobus.net>
parents:
39336
diff
changeset
|
753 continue |
40978
42f59d3f714d
delta: exclude base candidate much smaller than the target
Boris Feld <boris.feld@octobus.net>
parents:
40957
diff
changeset
|
754 if sparse and revlog.rawsize(rev) < (textlen // LIMIT_BASE2TEXT): |
49615
4956942c0416
delta-find: adjust the moment when we mark something as "tested"
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49614
diff
changeset
|
755 tested.add(rev) |
40978
42f59d3f714d
delta: exclude base candidate much smaller than the target
Boris Feld <boris.feld@octobus.net>
parents:
40957
diff
changeset
|
756 continue |
39337
37957e07138c
revlogdeltas: move finddeltainfo filtering inside _candidategroups
Boris Feld <boris.feld@octobus.net>
parents:
39336
diff
changeset
|
757 # no delta for rawtext-changing revs (see "candelta" for why) |
37957e07138c
revlogdeltas: move finddeltainfo filtering inside _candidategroups
Boris Feld <boris.feld@octobus.net>
parents:
39336
diff
changeset
|
758 if revlog.flags(rev) & REVIDX_RAWTEXT_CHANGING_FLAGS: |
49615
4956942c0416
delta-find: adjust the moment when we mark something as "tested"
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49614
diff
changeset
|
759 tested.add(rev) |
39337
37957e07138c
revlogdeltas: move finddeltainfo filtering inside _candidategroups
Boris Feld <boris.feld@octobus.net>
parents:
39336
diff
changeset
|
760 continue |
49610
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
761 |
40979
ba09db267cb6
delta: ignore base whose chains already don't match expectations
Boris Feld <boris.feld@octobus.net>
parents:
40978
diff
changeset
|
762 # If we reach here, we are about to build and test a delta. |
ba09db267cb6
delta: ignore base whose chains already don't match expectations
Boris Feld <boris.feld@octobus.net>
parents:
40978
diff
changeset
|
763 # The delta building process will compute the chaininfo in all |
ba09db267cb6
delta: ignore base whose chains already don't match expectations
Boris Feld <boris.feld@octobus.net>
parents:
40978
diff
changeset
|
764 # case, since that computation is cached, it is fine to access it |
ba09db267cb6
delta: ignore base whose chains already don't match expectations
Boris Feld <boris.feld@octobus.net>
parents:
40978
diff
changeset
|
765 # here too. |
ba09db267cb6
delta: ignore base whose chains already don't match expectations
Boris Feld <boris.feld@octobus.net>
parents:
40978
diff
changeset
|
766 chainlen, chainsize = revlog._chaininfo(rev) |
ba09db267cb6
delta: ignore base whose chains already don't match expectations
Boris Feld <boris.feld@octobus.net>
parents:
40978
diff
changeset
|
767 # if chain will be too long, skip base |
ba09db267cb6
delta: ignore base whose chains already don't match expectations
Boris Feld <boris.feld@octobus.net>
parents:
40978
diff
changeset
|
768 if revlog._maxchainlen and chainlen >= revlog._maxchainlen: |
49615
4956942c0416
delta-find: adjust the moment when we mark something as "tested"
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49614
diff
changeset
|
769 tested.add(rev) |
40979
ba09db267cb6
delta: ignore base whose chains already don't match expectations
Boris Feld <boris.feld@octobus.net>
parents:
40978
diff
changeset
|
770 continue |
ba09db267cb6
delta: ignore base whose chains already don't match expectations
Boris Feld <boris.feld@octobus.net>
parents:
40978
diff
changeset
|
771 # if chain already have too much data, skip base |
ba09db267cb6
delta: ignore base whose chains already don't match expectations
Boris Feld <boris.feld@octobus.net>
parents:
40978
diff
changeset
|
772 if deltas_limit < chainsize: |
49615
4956942c0416
delta-find: adjust the moment when we mark something as "tested"
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49614
diff
changeset
|
773 tested.add(rev) |
40979
ba09db267cb6
delta: ignore base whose chains already don't match expectations
Boris Feld <boris.feld@octobus.net>
parents:
40978
diff
changeset
|
774 continue |
42463
a0b26fc8fbba
deltas: skip if projected delta size does not match text size constraint
Valentin Gatien-Baron <vgatien-baron@janestreet.com>, Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42057
diff
changeset
|
775 if sparse and revlog.upperboundcomp is not None: |
a0b26fc8fbba
deltas: skip if projected delta size does not match text size constraint
Valentin Gatien-Baron <vgatien-baron@janestreet.com>, Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42057
diff
changeset
|
776 maxcomp = revlog.upperboundcomp |
a0b26fc8fbba
deltas: skip if projected delta size does not match text size constraint
Valentin Gatien-Baron <vgatien-baron@janestreet.com>, Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42057
diff
changeset
|
777 basenotsnap = (p1, p2, nullrev) |
a0b26fc8fbba
deltas: skip if projected delta size does not match text size constraint
Valentin Gatien-Baron <vgatien-baron@janestreet.com>, Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42057
diff
changeset
|
778 if rev not in basenotsnap and revlog.issnapshot(rev): |
a0b26fc8fbba
deltas: skip if projected delta size does not match text size constraint
Valentin Gatien-Baron <vgatien-baron@janestreet.com>, Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42057
diff
changeset
|
779 snapshotdepth = revlog.snapshotdepth(rev) |
a0b26fc8fbba
deltas: skip if projected delta size does not match text size constraint
Valentin Gatien-Baron <vgatien-baron@janestreet.com>, Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42057
diff
changeset
|
780 # If text is significantly larger than the base, we can |
a0b26fc8fbba
deltas: skip if projected delta size does not match text size constraint
Valentin Gatien-Baron <vgatien-baron@janestreet.com>, Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42057
diff
changeset
|
781 # expect the resulting delta to be proportional to the size |
a0b26fc8fbba
deltas: skip if projected delta size does not match text size constraint
Valentin Gatien-Baron <vgatien-baron@janestreet.com>, Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42057
diff
changeset
|
782 # difference |
a0b26fc8fbba
deltas: skip if projected delta size does not match text size constraint
Valentin Gatien-Baron <vgatien-baron@janestreet.com>, Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42057
diff
changeset
|
783 revsize = revlog.rawsize(rev) |
a0b26fc8fbba
deltas: skip if projected delta size does not match text size constraint
Valentin Gatien-Baron <vgatien-baron@janestreet.com>, Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42057
diff
changeset
|
784 rawsizedistance = max(textlen - revsize, 0) |
a0b26fc8fbba
deltas: skip if projected delta size does not match text size constraint
Valentin Gatien-Baron <vgatien-baron@janestreet.com>, Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42057
diff
changeset
|
785 # use an estimate of the compression upper bound. |
a0b26fc8fbba
deltas: skip if projected delta size does not match text size constraint
Valentin Gatien-Baron <vgatien-baron@janestreet.com>, Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42057
diff
changeset
|
786 lowestrealisticdeltalen = rawsizedistance // maxcomp |
a0b26fc8fbba
deltas: skip if projected delta size does not match text size constraint
Valentin Gatien-Baron <vgatien-baron@janestreet.com>, Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42057
diff
changeset
|
787 |
a0b26fc8fbba
deltas: skip if projected delta size does not match text size constraint
Valentin Gatien-Baron <vgatien-baron@janestreet.com>, Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42057
diff
changeset
|
788 # check the absolute constraint on the delta size |
a0b26fc8fbba
deltas: skip if projected delta size does not match text size constraint
Valentin Gatien-Baron <vgatien-baron@janestreet.com>, Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42057
diff
changeset
|
789 snapshotlimit = textlen >> snapshotdepth |
a0b26fc8fbba
deltas: skip if projected delta size does not match text size constraint
Valentin Gatien-Baron <vgatien-baron@janestreet.com>, Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42057
diff
changeset
|
790 if snapshotlimit < lowestrealisticdeltalen: |
a0b26fc8fbba
deltas: skip if projected delta size does not match text size constraint
Valentin Gatien-Baron <vgatien-baron@janestreet.com>, Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42057
diff
changeset
|
791 # delta lower bound is larger than accepted upper bound |
49615
4956942c0416
delta-find: adjust the moment when we mark something as "tested"
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49614
diff
changeset
|
792 tested.add(rev) |
42463
a0b26fc8fbba
deltas: skip if projected delta size does not match text size constraint
Valentin Gatien-Baron <vgatien-baron@janestreet.com>, Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42057
diff
changeset
|
793 continue |
a0b26fc8fbba
deltas: skip if projected delta size does not match text size constraint
Valentin Gatien-Baron <vgatien-baron@janestreet.com>, Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42057
diff
changeset
|
794 |
42464
66c27df1be84
deltas: skip if projected delta size is bigger than previous snapshot
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42463
diff
changeset
|
795 # check the relative constraint on the delta size |
66c27df1be84
deltas: skip if projected delta size is bigger than previous snapshot
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42463
diff
changeset
|
796 revlength = revlog.length(rev) |
66c27df1be84
deltas: skip if projected delta size is bigger than previous snapshot
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42463
diff
changeset
|
797 if revlength < lowestrealisticdeltalen: |
66c27df1be84
deltas: skip if projected delta size is bigger than previous snapshot
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42463
diff
changeset
|
798 # delta probable lower bound is larger than target base |
49615
4956942c0416
delta-find: adjust the moment when we mark something as "tested"
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49614
diff
changeset
|
799 tested.add(rev) |
42464
66c27df1be84
deltas: skip if projected delta size is bigger than previous snapshot
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42463
diff
changeset
|
800 continue |
66c27df1be84
deltas: skip if projected delta size is bigger than previous snapshot
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42463
diff
changeset
|
801 |
39337
37957e07138c
revlogdeltas: move finddeltainfo filtering inside _candidategroups
Boris Feld <boris.feld@octobus.net>
parents:
39336
diff
changeset
|
802 group.append(rev) |
39336
1c6ff52fe9cf
revlogdeltas: split candidate groups selection from the filtering logic
Boris Feld <boris.feld@octobus.net>
parents:
39335
diff
changeset
|
803 if group: |
49657
f5f113f1b011
delta-find: add a way to control the number of bases tested at the same time
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49615
diff
changeset
|
804 # When the size of the candidate group is big, it can result in a |
f5f113f1b011
delta-find: add a way to control the number of bases tested at the same time
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49615
diff
changeset
|
805 # quite significant performance impact. To reduce this, we can send |
f5f113f1b011
delta-find: add a way to control the number of bases tested at the same time
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49615
diff
changeset
|
806 # them in smaller batches until the new batch does not provide any |
f5f113f1b011
delta-find: add a way to control the number of bases tested at the same time
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49615
diff
changeset
|
807 # improvements. |
f5f113f1b011
delta-find: add a way to control the number of bases tested at the same time
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49615
diff
changeset
|
808 # |
f5f113f1b011
delta-find: add a way to control the number of bases tested at the same time
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49615
diff
changeset
|
809 # This might reduce the overall efficiency of the compression in |
f5f113f1b011
delta-find: add a way to control the number of bases tested at the same time
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49615
diff
changeset
|
810 # some corner cases, but that should also prevent very pathological |
f5f113f1b011
delta-find: add a way to control the number of bases tested at the same time
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49615
diff
changeset
|
811 # cases from being an issue. (eg. 20 000 candidates). |
f5f113f1b011
delta-find: add a way to control the number of bases tested at the same time
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49615
diff
changeset
|
812 # |
f5f113f1b011
delta-find: add a way to control the number of bases tested at the same time
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49615
diff
changeset
|
813 # XXX note that the ordering of the group becomes important as it |
f5f113f1b011
delta-find: add a way to control the number of bases tested at the same time
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49615
diff
changeset
|
814 # now impacts the final result. The current order is unprocessed |
f5f113f1b011
delta-find: add a way to control the number of bases tested at the same time
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49615
diff
changeset
|
815 # and can be improved. |
f5f113f1b011
delta-find: add a way to control the number of bases tested at the same time
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49615
diff
changeset
|
816 if group_chunk_size == 0: |
f5f113f1b011
delta-find: add a way to control the number of bases tested at the same time
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49615
diff
changeset
|
817 tested.update(group) |
f5f113f1b011
delta-find: add a way to control the number of bases tested at the same time
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49615
diff
changeset
|
818 good = yield tuple(group) |
f5f113f1b011
delta-find: add a way to control the number of bases tested at the same time
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49615
diff
changeset
|
819 else: |
f5f113f1b011
delta-find: add a way to control the number of bases tested at the same time
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49615
diff
changeset
|
820 prev_good = good |
f5f113f1b011
delta-find: add a way to control the number of bases tested at the same time
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49615
diff
changeset
|
821 for start in range(0, len(group), group_chunk_size): |
f5f113f1b011
delta-find: add a way to control the number of bases tested at the same time
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49615
diff
changeset
|
822 sub_group = group[start : start + group_chunk_size] |
f5f113f1b011
delta-find: add a way to control the number of bases tested at the same time
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49615
diff
changeset
|
823 tested.update(sub_group) |
f5f113f1b011
delta-find: add a way to control the number of bases tested at the same time
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49615
diff
changeset
|
824 good = yield tuple(sub_group) |
f5f113f1b011
delta-find: add a way to control the number of bases tested at the same time
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49615
diff
changeset
|
825 if prev_good == good: |
f5f113f1b011
delta-find: add a way to control the number of bases tested at the same time
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49615
diff
changeset
|
826 break |
f5f113f1b011
delta-find: add a way to control the number of bases tested at the same time
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49615
diff
changeset
|
827 |
39497
5b308a4e6d03
snapshot: use None as a stop value when looking for a good delta
Boris Feld <boris.feld@octobus.net>
parents:
39496
diff
changeset
|
828 yield None |
39336
1c6ff52fe9cf
revlogdeltas: split candidate groups selection from the filtering logic
Boris Feld <boris.feld@octobus.net>
parents:
39335
diff
changeset
|
829 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
830 |
49681
c261a628e525
delta-find: use a single snapshot cache when applying a group to an object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49680
diff
changeset
|
831 def _refinedgroups(revlog, p1, p2, cachedelta, snapshot_cache=None): |
39496
2f9f7889549b
snapshot: introduce an intermediate `_refinedgroups` generator
Boris Feld <boris.feld@octobus.net>
parents:
39495
diff
changeset
|
832 good = None |
39501
993d7e2c8b79
snapshot: make sure we'll never refine delta base from a reused source
Boris Feld <boris.feld@octobus.net>
parents:
39500
diff
changeset
|
833 # First we try to reuse a the delta contained in the bundle. |
993d7e2c8b79
snapshot: make sure we'll never refine delta base from a reused source
Boris Feld <boris.feld@octobus.net>
parents:
39500
diff
changeset
|
834 # (or from the source revlog) |
993d7e2c8b79
snapshot: make sure we'll never refine delta base from a reused source
Boris Feld <boris.feld@octobus.net>
parents:
39500
diff
changeset
|
835 # |
993d7e2c8b79
snapshot: make sure we'll never refine delta base from a reused source
Boris Feld <boris.feld@octobus.net>
parents:
39500
diff
changeset
|
836 # This logic only applies to general delta repositories and can be disabled |
993d7e2c8b79
snapshot: make sure we'll never refine delta base from a reused source
Boris Feld <boris.feld@octobus.net>
parents:
39500
diff
changeset
|
837 # through configuration. Disabling reuse source delta is useful when |
993d7e2c8b79
snapshot: make sure we'll never refine delta base from a reused source
Boris Feld <boris.feld@octobus.net>
parents:
39500
diff
changeset
|
838 # we want to make sure we recomputed "optimal" deltas. |
49610
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
839 debug_info = None |
49677
05db41701ece
find-delta: pass the cache-delta usage policy alongside the cache-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49675
diff
changeset
|
840 if cachedelta is not None and cachedelta[2] > DELTA_BASE_REUSE_NO: |
39501
993d7e2c8b79
snapshot: make sure we'll never refine delta base from a reused source
Boris Feld <boris.feld@octobus.net>
parents:
39500
diff
changeset
|
841 # Assume what we received from the server is a good choice |
993d7e2c8b79
snapshot: make sure we'll never refine delta base from a reused source
Boris Feld <boris.feld@octobus.net>
parents:
39500
diff
changeset
|
842 # build delta will reuse the cache |
49610
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
843 if debug_info is not None: |
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
844 debug_info['cached-delta.tested'] += 1 |
39501
993d7e2c8b79
snapshot: make sure we'll never refine delta base from a reused source
Boris Feld <boris.feld@octobus.net>
parents:
39500
diff
changeset
|
845 good = yield (cachedelta[0],) |
993d7e2c8b79
snapshot: make sure we'll never refine delta base from a reused source
Boris Feld <boris.feld@octobus.net>
parents:
39500
diff
changeset
|
846 if good is not None: |
49610
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
847 if debug_info is not None: |
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
848 debug_info['cached-delta.accepted'] += 1 |
39501
993d7e2c8b79
snapshot: make sure we'll never refine delta base from a reused source
Boris Feld <boris.feld@octobus.net>
parents:
39500
diff
changeset
|
849 yield None |
993d7e2c8b79
snapshot: make sure we'll never refine delta base from a reused source
Boris Feld <boris.feld@octobus.net>
parents:
39500
diff
changeset
|
850 return |
49681
c261a628e525
delta-find: use a single snapshot cache when applying a group to an object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49680
diff
changeset
|
851 if snapshot_cache is None: |
c261a628e525
delta-find: use a single snapshot cache when applying a group to an object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49680
diff
changeset
|
852 snapshot_cache = SnapshotCache() |
49610
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
853 groups = _rawgroups( |
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
854 revlog, |
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
855 p1, |
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
856 p2, |
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
857 cachedelta, |
49678
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
858 snapshot_cache, |
49610
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
859 ) |
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
860 for candidates in groups: |
39496
2f9f7889549b
snapshot: introduce an intermediate `_refinedgroups` generator
Boris Feld <boris.feld@octobus.net>
parents:
39495
diff
changeset
|
861 good = yield candidates |
2f9f7889549b
snapshot: introduce an intermediate `_refinedgroups` generator
Boris Feld <boris.feld@octobus.net>
parents:
39495
diff
changeset
|
862 if good is not None: |
2f9f7889549b
snapshot: introduce an intermediate `_refinedgroups` generator
Boris Feld <boris.feld@octobus.net>
parents:
39495
diff
changeset
|
863 break |
39502
e4d4361d0bcd
snapshot: try to refine new snapshot base down the chain
Boris Feld <boris.feld@octobus.net>
parents:
39501
diff
changeset
|
864 |
40428
bafa1c4bb7a8
sparse-revlog: only refine delta candidates in the sparse case (issue6006)
Boris Feld <boris.feld@octobus.net>
parents:
39777
diff
changeset
|
865 # If sparse revlog is enabled, we can try to refine the available deltas |
bafa1c4bb7a8
sparse-revlog: only refine delta candidates in the sparse case (issue6006)
Boris Feld <boris.feld@octobus.net>
parents:
39777
diff
changeset
|
866 if not revlog._sparserevlog: |
bafa1c4bb7a8
sparse-revlog: only refine delta candidates in the sparse case (issue6006)
Boris Feld <boris.feld@octobus.net>
parents:
39777
diff
changeset
|
867 yield None |
bafa1c4bb7a8
sparse-revlog: only refine delta candidates in the sparse case (issue6006)
Boris Feld <boris.feld@octobus.net>
parents:
39777
diff
changeset
|
868 return |
bafa1c4bb7a8
sparse-revlog: only refine delta candidates in the sparse case (issue6006)
Boris Feld <boris.feld@octobus.net>
parents:
39777
diff
changeset
|
869 |
39502
e4d4361d0bcd
snapshot: try to refine new snapshot base down the chain
Boris Feld <boris.feld@octobus.net>
parents:
39501
diff
changeset
|
870 # if we have a refinable value, try to refine it |
e4d4361d0bcd
snapshot: try to refine new snapshot base down the chain
Boris Feld <boris.feld@octobus.net>
parents:
39501
diff
changeset
|
871 if good is not None and good not in (p1, p2) and revlog.issnapshot(good): |
e4d4361d0bcd
snapshot: try to refine new snapshot base down the chain
Boris Feld <boris.feld@octobus.net>
parents:
39501
diff
changeset
|
872 # refine snapshot down |
e4d4361d0bcd
snapshot: try to refine new snapshot base down the chain
Boris Feld <boris.feld@octobus.net>
parents:
39501
diff
changeset
|
873 previous = None |
e4d4361d0bcd
snapshot: try to refine new snapshot base down the chain
Boris Feld <boris.feld@octobus.net>
parents:
39501
diff
changeset
|
874 while previous != good: |
e4d4361d0bcd
snapshot: try to refine new snapshot base down the chain
Boris Feld <boris.feld@octobus.net>
parents:
39501
diff
changeset
|
875 previous = good |
e4d4361d0bcd
snapshot: try to refine new snapshot base down the chain
Boris Feld <boris.feld@octobus.net>
parents:
39501
diff
changeset
|
876 base = revlog.deltaparent(good) |
e4d4361d0bcd
snapshot: try to refine new snapshot base down the chain
Boris Feld <boris.feld@octobus.net>
parents:
39501
diff
changeset
|
877 if base == nullrev: |
e4d4361d0bcd
snapshot: try to refine new snapshot base down the chain
Boris Feld <boris.feld@octobus.net>
parents:
39501
diff
changeset
|
878 break |
e4d4361d0bcd
snapshot: try to refine new snapshot base down the chain
Boris Feld <boris.feld@octobus.net>
parents:
39501
diff
changeset
|
879 good = yield (base,) |
39503
5aef5afa8654
snapshot: refine candidate snapshot base upward
Boris Feld <boris.feld@octobus.net>
parents:
39502
diff
changeset
|
880 # refine snapshot up |
49678
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
881 if not snapshot_cache.snapshots: |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
882 snapshot_cache.update(revlog, good + 1) |
39503
5aef5afa8654
snapshot: refine candidate snapshot base upward
Boris Feld <boris.feld@octobus.net>
parents:
39502
diff
changeset
|
883 previous = None |
5aef5afa8654
snapshot: refine candidate snapshot base upward
Boris Feld <boris.feld@octobus.net>
parents:
39502
diff
changeset
|
884 while good != previous: |
5aef5afa8654
snapshot: refine candidate snapshot base upward
Boris Feld <boris.feld@octobus.net>
parents:
39502
diff
changeset
|
885 previous = good |
49678
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
886 children = tuple(sorted(c for c in snapshot_cache.snapshots[good])) |
39503
5aef5afa8654
snapshot: refine candidate snapshot base upward
Boris Feld <boris.feld@octobus.net>
parents:
39502
diff
changeset
|
887 good = yield children |
5aef5afa8654
snapshot: refine candidate snapshot base upward
Boris Feld <boris.feld@octobus.net>
parents:
39502
diff
changeset
|
888 |
49610
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
889 if debug_info is not None: |
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
890 if good is None: |
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
891 debug_info['no-solution'] += 1 |
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
892 |
39499
51cec7fb672e
snapshot: also use None as a stop value for `_refinegroup`
Boris Feld <boris.feld@octobus.net>
parents:
39498
diff
changeset
|
893 yield None |
39496
2f9f7889549b
snapshot: introduce an intermediate `_refinedgroups` generator
Boris Feld <boris.feld@octobus.net>
parents:
39495
diff
changeset
|
894 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
895 |
49678
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
896 def _rawgroups(revlog, p1, p2, cachedelta, snapshot_cache=None): |
39336
1c6ff52fe9cf
revlogdeltas: split candidate groups selection from the filtering logic
Boris Feld <boris.feld@octobus.net>
parents:
39335
diff
changeset
|
897 """Provides group of revision to be tested as delta base |
1c6ff52fe9cf
revlogdeltas: split candidate groups selection from the filtering logic
Boris Feld <boris.feld@octobus.net>
parents:
39335
diff
changeset
|
898 |
1c6ff52fe9cf
revlogdeltas: split candidate groups selection from the filtering logic
Boris Feld <boris.feld@octobus.net>
parents:
39335
diff
changeset
|
899 This lower level function focus on emitting delta theorically interresting |
1c6ff52fe9cf
revlogdeltas: split candidate groups selection from the filtering logic
Boris Feld <boris.feld@octobus.net>
parents:
39335
diff
changeset
|
900 without looking it any practical details. |
1c6ff52fe9cf
revlogdeltas: split candidate groups selection from the filtering logic
Boris Feld <boris.feld@octobus.net>
parents:
39335
diff
changeset
|
901 |
1c6ff52fe9cf
revlogdeltas: split candidate groups selection from the filtering logic
Boris Feld <boris.feld@octobus.net>
parents:
39335
diff
changeset
|
902 The group order aims at providing fast or small candidates first. |
39334
507f5b1dd7c8
revlogdeltas: extract _getcandidaterevs in a function
Boris Feld <boris.feld@octobus.net>
parents:
39333
diff
changeset
|
903 """ |
50357
8038493eb31a
delta-find: simply code that is now never invoqued without general delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50356
diff
changeset
|
904 # Why search for delta base if we cannot use a delta base ? |
8038493eb31a
delta-find: simply code that is now never invoqued without general delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50356
diff
changeset
|
905 assert revlog._generaldelta |
8038493eb31a
delta-find: simply code that is now never invoqued without general delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50356
diff
changeset
|
906 # also see issue6056 |
8038493eb31a
delta-find: simply code that is now never invoqued without general delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50356
diff
changeset
|
907 sparse = revlog._sparserevlog |
39334
507f5b1dd7c8
revlogdeltas: extract _getcandidaterevs in a function
Boris Feld <boris.feld@octobus.net>
parents:
39333
diff
changeset
|
908 curr = len(revlog) |
507f5b1dd7c8
revlogdeltas: extract _getcandidaterevs in a function
Boris Feld <boris.feld@octobus.net>
parents:
39333
diff
changeset
|
909 prev = curr - 1 |
39492
a33f394b2bfd
snapshot: try intermediate snapshot against parents' base
Boris Feld <boris.feld@octobus.net>
parents:
39488
diff
changeset
|
910 deltachain = lambda rev: revlog._deltachain(rev)[0] |
39334
507f5b1dd7c8
revlogdeltas: extract _getcandidaterevs in a function
Boris Feld <boris.feld@octobus.net>
parents:
39333
diff
changeset
|
911 |
50357
8038493eb31a
delta-find: simply code that is now never invoqued without general delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50356
diff
changeset
|
912 # exclude already lazy tested base if any |
8038493eb31a
delta-find: simply code that is now never invoqued without general delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50356
diff
changeset
|
913 parents = [p for p in (p1, p2) if p != nullrev] |
39334
507f5b1dd7c8
revlogdeltas: extract _getcandidaterevs in a function
Boris Feld <boris.feld@octobus.net>
parents:
39333
diff
changeset
|
914 |
50357
8038493eb31a
delta-find: simply code that is now never invoqued without general delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50356
diff
changeset
|
915 if not revlog._deltabothparents and len(parents) == 2: |
8038493eb31a
delta-find: simply code that is now never invoqued without general delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50356
diff
changeset
|
916 parents.sort() |
8038493eb31a
delta-find: simply code that is now never invoqued without general delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50356
diff
changeset
|
917 # To minimize the chance of having to build a fulltext, |
8038493eb31a
delta-find: simply code that is now never invoqued without general delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50356
diff
changeset
|
918 # pick first whichever parent is closest to us (max rev) |
8038493eb31a
delta-find: simply code that is now never invoqued without general delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50356
diff
changeset
|
919 yield (parents[1],) |
8038493eb31a
delta-find: simply code that is now never invoqued without general delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50356
diff
changeset
|
920 # then the other one (min rev) if the first did not fit |
8038493eb31a
delta-find: simply code that is now never invoqued without general delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50356
diff
changeset
|
921 yield (parents[0],) |
8038493eb31a
delta-find: simply code that is now never invoqued without general delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50356
diff
changeset
|
922 elif len(parents) > 0: |
8038493eb31a
delta-find: simply code that is now never invoqued without general delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50356
diff
changeset
|
923 # Test all parents (1 or 2), and keep the best candidate |
8038493eb31a
delta-find: simply code that is now never invoqued without general delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50356
diff
changeset
|
924 yield parents |
39334
507f5b1dd7c8
revlogdeltas: extract _getcandidaterevs in a function
Boris Feld <boris.feld@octobus.net>
parents:
39333
diff
changeset
|
925 |
39492
a33f394b2bfd
snapshot: try intermediate snapshot against parents' base
Boris Feld <boris.feld@octobus.net>
parents:
39488
diff
changeset
|
926 if sparse and parents: |
49678
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
927 if snapshot_cache is None: |
49613
5447c1507c86
delta-find: small documentation update
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49612
diff
changeset
|
928 # map: base-rev: [snapshot-revs] |
49678
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
929 snapshot_cache = SnapshotCache() |
39492
a33f394b2bfd
snapshot: try intermediate snapshot against parents' base
Boris Feld <boris.feld@octobus.net>
parents:
39488
diff
changeset
|
930 # See if we can use an existing snapshot in the parent chains to use as |
a33f394b2bfd
snapshot: try intermediate snapshot against parents' base
Boris Feld <boris.feld@octobus.net>
parents:
39488
diff
changeset
|
931 # a base for a new intermediate-snapshot |
39494
e72130f58f5d
snapshot: consider all snapshots in the parents' chains
Boris Feld <boris.feld@octobus.net>
parents:
39493
diff
changeset
|
932 # |
e72130f58f5d
snapshot: consider all snapshots in the parents' chains
Boris Feld <boris.feld@octobus.net>
parents:
39493
diff
changeset
|
933 # search for snapshot in parents delta chain |
e72130f58f5d
snapshot: consider all snapshots in the parents' chains
Boris Feld <boris.feld@octobus.net>
parents:
39493
diff
changeset
|
934 # map: snapshot-level: snapshot-rev |
e72130f58f5d
snapshot: consider all snapshots in the parents' chains
Boris Feld <boris.feld@octobus.net>
parents:
39493
diff
changeset
|
935 parents_snaps = collections.defaultdict(set) |
39504
05a165dc4f55
snapshot: extract parent chain computation
Boris Feld <boris.feld@octobus.net>
parents:
39503
diff
changeset
|
936 candidate_chains = [deltachain(p) for p in parents] |
05a165dc4f55
snapshot: extract parent chain computation
Boris Feld <boris.feld@octobus.net>
parents:
39503
diff
changeset
|
937 for chain in candidate_chains: |
05a165dc4f55
snapshot: extract parent chain computation
Boris Feld <boris.feld@octobus.net>
parents:
39503
diff
changeset
|
938 for idx, s in enumerate(chain): |
39494
e72130f58f5d
snapshot: consider all snapshots in the parents' chains
Boris Feld <boris.feld@octobus.net>
parents:
39493
diff
changeset
|
939 if not revlog.issnapshot(s): |
e72130f58f5d
snapshot: consider all snapshots in the parents' chains
Boris Feld <boris.feld@octobus.net>
parents:
39493
diff
changeset
|
940 break |
e72130f58f5d
snapshot: consider all snapshots in the parents' chains
Boris Feld <boris.feld@octobus.net>
parents:
39493
diff
changeset
|
941 parents_snaps[idx].add(s) |
39495
6a53842727c1
snapshot: consider unrelated snapshots at a similar level first
Boris Feld <boris.feld@octobus.net>
parents:
39494
diff
changeset
|
942 snapfloor = min(parents_snaps[0]) + 1 |
49678
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
943 snapshot_cache.update(revlog, snapfloor) |
39505
c6b8eab5db19
snapshot: also consider the snapshot chain of one unrelated revision
Boris Feld <boris.feld@octobus.net>
parents:
39504
diff
changeset
|
944 # search for the highest "unrelated" revision |
c6b8eab5db19
snapshot: also consider the snapshot chain of one unrelated revision
Boris Feld <boris.feld@octobus.net>
parents:
39504
diff
changeset
|
945 # |
c6b8eab5db19
snapshot: also consider the snapshot chain of one unrelated revision
Boris Feld <boris.feld@octobus.net>
parents:
39504
diff
changeset
|
946 # Adding snapshots used by "unrelated" revision increase the odd we |
c6b8eab5db19
snapshot: also consider the snapshot chain of one unrelated revision
Boris Feld <boris.feld@octobus.net>
parents:
39504
diff
changeset
|
947 # reuse an independant, yet better snapshot chain. |
c6b8eab5db19
snapshot: also consider the snapshot chain of one unrelated revision
Boris Feld <boris.feld@octobus.net>
parents:
39504
diff
changeset
|
948 # |
c6b8eab5db19
snapshot: also consider the snapshot chain of one unrelated revision
Boris Feld <boris.feld@octobus.net>
parents:
39504
diff
changeset
|
949 # XXX instead of building a set of revisions, we could lazily enumerate |
c6b8eab5db19
snapshot: also consider the snapshot chain of one unrelated revision
Boris Feld <boris.feld@octobus.net>
parents:
39504
diff
changeset
|
950 # over the chains. That would be more efficient, however we stick to |
c6b8eab5db19
snapshot: also consider the snapshot chain of one unrelated revision
Boris Feld <boris.feld@octobus.net>
parents:
39504
diff
changeset
|
951 # simple code for now. |
c6b8eab5db19
snapshot: also consider the snapshot chain of one unrelated revision
Boris Feld <boris.feld@octobus.net>
parents:
39504
diff
changeset
|
952 all_revs = set() |
c6b8eab5db19
snapshot: also consider the snapshot chain of one unrelated revision
Boris Feld <boris.feld@octobus.net>
parents:
39504
diff
changeset
|
953 for chain in candidate_chains: |
c6b8eab5db19
snapshot: also consider the snapshot chain of one unrelated revision
Boris Feld <boris.feld@octobus.net>
parents:
39504
diff
changeset
|
954 all_revs.update(chain) |
c6b8eab5db19
snapshot: also consider the snapshot chain of one unrelated revision
Boris Feld <boris.feld@octobus.net>
parents:
39504
diff
changeset
|
955 other = None |
c6b8eab5db19
snapshot: also consider the snapshot chain of one unrelated revision
Boris Feld <boris.feld@octobus.net>
parents:
39504
diff
changeset
|
956 for r in revlog.revs(prev, snapfloor): |
c6b8eab5db19
snapshot: also consider the snapshot chain of one unrelated revision
Boris Feld <boris.feld@octobus.net>
parents:
39504
diff
changeset
|
957 if r not in all_revs: |
c6b8eab5db19
snapshot: also consider the snapshot chain of one unrelated revision
Boris Feld <boris.feld@octobus.net>
parents:
39504
diff
changeset
|
958 other = r |
c6b8eab5db19
snapshot: also consider the snapshot chain of one unrelated revision
Boris Feld <boris.feld@octobus.net>
parents:
39504
diff
changeset
|
959 break |
c6b8eab5db19
snapshot: also consider the snapshot chain of one unrelated revision
Boris Feld <boris.feld@octobus.net>
parents:
39504
diff
changeset
|
960 if other is not None: |
c6b8eab5db19
snapshot: also consider the snapshot chain of one unrelated revision
Boris Feld <boris.feld@octobus.net>
parents:
39504
diff
changeset
|
961 # To avoid unfair competition, we won't use unrelated intermediate |
c6b8eab5db19
snapshot: also consider the snapshot chain of one unrelated revision
Boris Feld <boris.feld@octobus.net>
parents:
39504
diff
changeset
|
962 # snapshot that are deeper than the ones from the parent delta |
c6b8eab5db19
snapshot: also consider the snapshot chain of one unrelated revision
Boris Feld <boris.feld@octobus.net>
parents:
39504
diff
changeset
|
963 # chain. |
c6b8eab5db19
snapshot: also consider the snapshot chain of one unrelated revision
Boris Feld <boris.feld@octobus.net>
parents:
39504
diff
changeset
|
964 max_depth = max(parents_snaps.keys()) |
c6b8eab5db19
snapshot: also consider the snapshot chain of one unrelated revision
Boris Feld <boris.feld@octobus.net>
parents:
39504
diff
changeset
|
965 chain = deltachain(other) |
49614
01ccb45b7393
delta-find: rename a variable for clarity
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49613
diff
changeset
|
966 for depth, s in enumerate(chain): |
39505
c6b8eab5db19
snapshot: also consider the snapshot chain of one unrelated revision
Boris Feld <boris.feld@octobus.net>
parents:
39504
diff
changeset
|
967 if s < snapfloor: |
c6b8eab5db19
snapshot: also consider the snapshot chain of one unrelated revision
Boris Feld <boris.feld@octobus.net>
parents:
39504
diff
changeset
|
968 continue |
49614
01ccb45b7393
delta-find: rename a variable for clarity
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49613
diff
changeset
|
969 if max_depth < depth: |
39505
c6b8eab5db19
snapshot: also consider the snapshot chain of one unrelated revision
Boris Feld <boris.feld@octobus.net>
parents:
39504
diff
changeset
|
970 break |
c6b8eab5db19
snapshot: also consider the snapshot chain of one unrelated revision
Boris Feld <boris.feld@octobus.net>
parents:
39504
diff
changeset
|
971 if not revlog.issnapshot(s): |
c6b8eab5db19
snapshot: also consider the snapshot chain of one unrelated revision
Boris Feld <boris.feld@octobus.net>
parents:
39504
diff
changeset
|
972 break |
49614
01ccb45b7393
delta-find: rename a variable for clarity
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49613
diff
changeset
|
973 parents_snaps[depth].add(s) |
39494
e72130f58f5d
snapshot: consider all snapshots in the parents' chains
Boris Feld <boris.feld@octobus.net>
parents:
39493
diff
changeset
|
974 # Test them as possible intermediate snapshot base |
e72130f58f5d
snapshot: consider all snapshots in the parents' chains
Boris Feld <boris.feld@octobus.net>
parents:
39493
diff
changeset
|
975 # We test them from highest to lowest level. High level one are more |
e72130f58f5d
snapshot: consider all snapshots in the parents' chains
Boris Feld <boris.feld@octobus.net>
parents:
39493
diff
changeset
|
976 # likely to result in small delta |
39495
6a53842727c1
snapshot: consider unrelated snapshots at a similar level first
Boris Feld <boris.feld@octobus.net>
parents:
39494
diff
changeset
|
977 floor = None |
39494
e72130f58f5d
snapshot: consider all snapshots in the parents' chains
Boris Feld <boris.feld@octobus.net>
parents:
39493
diff
changeset
|
978 for idx, snaps in sorted(parents_snaps.items(), reverse=True): |
39495
6a53842727c1
snapshot: consider unrelated snapshots at a similar level first
Boris Feld <boris.feld@octobus.net>
parents:
39494
diff
changeset
|
979 siblings = set() |
6a53842727c1
snapshot: consider unrelated snapshots at a similar level first
Boris Feld <boris.feld@octobus.net>
parents:
39494
diff
changeset
|
980 for s in snaps: |
49678
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
981 siblings.update(snapshot_cache.snapshots[s]) |
39495
6a53842727c1
snapshot: consider unrelated snapshots at a similar level first
Boris Feld <boris.feld@octobus.net>
parents:
39494
diff
changeset
|
982 # Before considering making a new intermediate snapshot, we check |
6a53842727c1
snapshot: consider unrelated snapshots at a similar level first
Boris Feld <boris.feld@octobus.net>
parents:
39494
diff
changeset
|
983 # if an existing snapshot, children of base we consider, would be |
6a53842727c1
snapshot: consider unrelated snapshots at a similar level first
Boris Feld <boris.feld@octobus.net>
parents:
39494
diff
changeset
|
984 # suitable. |
6a53842727c1
snapshot: consider unrelated snapshots at a similar level first
Boris Feld <boris.feld@octobus.net>
parents:
39494
diff
changeset
|
985 # |
6a53842727c1
snapshot: consider unrelated snapshots at a similar level first
Boris Feld <boris.feld@octobus.net>
parents:
39494
diff
changeset
|
986 # It give a change to reuse a delta chain "unrelated" to the |
6a53842727c1
snapshot: consider unrelated snapshots at a similar level first
Boris Feld <boris.feld@octobus.net>
parents:
39494
diff
changeset
|
987 # current revision instead of starting our own. Without such |
6a53842727c1
snapshot: consider unrelated snapshots at a similar level first
Boris Feld <boris.feld@octobus.net>
parents:
39494
diff
changeset
|
988 # re-use, topological branches would keep reopening new chains. |
6a53842727c1
snapshot: consider unrelated snapshots at a similar level first
Boris Feld <boris.feld@octobus.net>
parents:
39494
diff
changeset
|
989 # Creating more and more snapshot as the repository grow. |
6a53842727c1
snapshot: consider unrelated snapshots at a similar level first
Boris Feld <boris.feld@octobus.net>
parents:
39494
diff
changeset
|
990 |
6a53842727c1
snapshot: consider unrelated snapshots at a similar level first
Boris Feld <boris.feld@octobus.net>
parents:
39494
diff
changeset
|
991 if floor is not None: |
6a53842727c1
snapshot: consider unrelated snapshots at a similar level first
Boris Feld <boris.feld@octobus.net>
parents:
39494
diff
changeset
|
992 # We only do this for siblings created after the one in our |
6a53842727c1
snapshot: consider unrelated snapshots at a similar level first
Boris Feld <boris.feld@octobus.net>
parents:
39494
diff
changeset
|
993 # parent's delta chain. Those created before has less chances |
6a53842727c1
snapshot: consider unrelated snapshots at a similar level first
Boris Feld <boris.feld@octobus.net>
parents:
39494
diff
changeset
|
994 # to be valid base since our ancestors had to create a new |
6a53842727c1
snapshot: consider unrelated snapshots at a similar level first
Boris Feld <boris.feld@octobus.net>
parents:
39494
diff
changeset
|
995 # snapshot. |
6a53842727c1
snapshot: consider unrelated snapshots at a similar level first
Boris Feld <boris.feld@octobus.net>
parents:
39494
diff
changeset
|
996 siblings = [r for r in siblings if floor < r] |
6a53842727c1
snapshot: consider unrelated snapshots at a similar level first
Boris Feld <boris.feld@octobus.net>
parents:
39494
diff
changeset
|
997 yield tuple(sorted(siblings)) |
6a53842727c1
snapshot: consider unrelated snapshots at a similar level first
Boris Feld <boris.feld@octobus.net>
parents:
39494
diff
changeset
|
998 # then test the base from our parent's delta chain. |
39494
e72130f58f5d
snapshot: consider all snapshots in the parents' chains
Boris Feld <boris.feld@octobus.net>
parents:
39493
diff
changeset
|
999 yield tuple(sorted(snaps)) |
39495
6a53842727c1
snapshot: consider unrelated snapshots at a similar level first
Boris Feld <boris.feld@octobus.net>
parents:
39494
diff
changeset
|
1000 floor = min(snaps) |
39493
3ca144f1c8dd
snapshot: search for unrelated but reusable full-snapshot
Boris Feld <boris.feld@octobus.net>
parents:
39492
diff
changeset
|
1001 # No suitable base found in the parent chain, search if any full |
3ca144f1c8dd
snapshot: search for unrelated but reusable full-snapshot
Boris Feld <boris.feld@octobus.net>
parents:
39492
diff
changeset
|
1002 # snapshots emitted since parent's base would be a suitable base for an |
3ca144f1c8dd
snapshot: search for unrelated but reusable full-snapshot
Boris Feld <boris.feld@octobus.net>
parents:
39492
diff
changeset
|
1003 # intermediate snapshot. |
3ca144f1c8dd
snapshot: search for unrelated but reusable full-snapshot
Boris Feld <boris.feld@octobus.net>
parents:
39492
diff
changeset
|
1004 # |
3ca144f1c8dd
snapshot: search for unrelated but reusable full-snapshot
Boris Feld <boris.feld@octobus.net>
parents:
39492
diff
changeset
|
1005 # It give a chance to reuse a delta chain unrelated to the current |
3ca144f1c8dd
snapshot: search for unrelated but reusable full-snapshot
Boris Feld <boris.feld@octobus.net>
parents:
39492
diff
changeset
|
1006 # revisions instead of starting our own. Without such re-use, |
3ca144f1c8dd
snapshot: search for unrelated but reusable full-snapshot
Boris Feld <boris.feld@octobus.net>
parents:
39492
diff
changeset
|
1007 # topological branches would keep reopening new full chains. Creating |
3ca144f1c8dd
snapshot: search for unrelated but reusable full-snapshot
Boris Feld <boris.feld@octobus.net>
parents:
39492
diff
changeset
|
1008 # more and more snapshot as the repository grow. |
49680
40e24d82b513
delta-find: make sure we only use newer full snapshot as candidate
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49679
diff
changeset
|
1009 full = [r for r in snapshot_cache.snapshots[nullrev] if snapfloor <= r] |
40e24d82b513
delta-find: make sure we only use newer full snapshot as candidate
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49679
diff
changeset
|
1010 yield tuple(sorted(full)) |
39492
a33f394b2bfd
snapshot: try intermediate snapshot against parents' base
Boris Feld <boris.feld@octobus.net>
parents:
39488
diff
changeset
|
1011 |
39505
c6b8eab5db19
snapshot: also consider the snapshot chain of one unrelated revision
Boris Feld <boris.feld@octobus.net>
parents:
39504
diff
changeset
|
1012 if not sparse: |
c6b8eab5db19
snapshot: also consider the snapshot chain of one unrelated revision
Boris Feld <boris.feld@octobus.net>
parents:
39504
diff
changeset
|
1013 # other approach failed try against prev to hopefully save us a |
c6b8eab5db19
snapshot: also consider the snapshot chain of one unrelated revision
Boris Feld <boris.feld@octobus.net>
parents:
39504
diff
changeset
|
1014 # fulltext. |
c6b8eab5db19
snapshot: also consider the snapshot chain of one unrelated revision
Boris Feld <boris.feld@octobus.net>
parents:
39504
diff
changeset
|
1015 yield (prev,) |
39334
507f5b1dd7c8
revlogdeltas: extract _getcandidaterevs in a function
Boris Feld <boris.feld@octobus.net>
parents:
39333
diff
changeset
|
1016 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1017 |
49678
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1018 class SnapshotCache: |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1019 __slots__ = ('snapshots', '_start_rev', '_end_rev') |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1020 |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1021 def __init__(self): |
49679
b670eb3dd6c9
delta-find: use sets instead of list in the snapshot cache
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49678
diff
changeset
|
1022 self.snapshots = collections.defaultdict(set) |
49678
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1023 self._start_rev = None |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1024 self._end_rev = None |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1025 |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1026 def update(self, revlog, start_rev=0): |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1027 """find snapshots from start_rev to tip""" |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1028 nb_revs = len(revlog) |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1029 end_rev = nb_revs - 1 |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1030 if start_rev > end_rev: |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1031 return # range is empty |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1032 |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1033 if self._start_rev is None: |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1034 assert self._end_rev is None |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1035 self._update(revlog, start_rev, end_rev) |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1036 elif not (self._start_rev <= start_rev and end_rev <= self._end_rev): |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1037 if start_rev < self._start_rev: |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1038 self._update(revlog, start_rev, self._start_rev - 1) |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1039 if self._end_rev < end_rev: |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1040 self._update(revlog, self._end_rev + 1, end_rev) |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1041 |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1042 if self._start_rev is None: |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1043 assert self._end_rev is None |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1044 self._end_rev = end_rev |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1045 self._start_rev = start_rev |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1046 else: |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1047 self._start_rev = min(self._start_rev, start_rev) |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1048 self._end_rev = max(self._end_rev, end_rev) |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1049 assert self._start_rev <= self._end_rev, ( |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1050 self._start_rev, |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1051 self._end_rev, |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1052 ) |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1053 |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1054 def _update(self, revlog, start_rev, end_rev): |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1055 """internal method that actually do update content""" |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1056 assert self._start_rev is None or ( |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1057 start_rev < self._start_rev or start_rev > self._end_rev |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1058 ), (self._start_rev, self._end_rev, start_rev, end_rev) |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1059 assert self._start_rev is None or ( |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1060 end_rev < self._start_rev or end_rev > self._end_rev |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1061 ), (self._start_rev, self._end_rev, start_rev, end_rev) |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1062 cache = self.snapshots |
50686
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50657
diff
changeset
|
1063 if util.safehasattr(revlog.index, 'findsnapshots'): |
49678
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1064 revlog.index.findsnapshots(cache, start_rev, end_rev) |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1065 else: |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1066 deltaparent = revlog.deltaparent |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1067 issnapshot = revlog.issnapshot |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1068 for rev in revlog.revs(start_rev, end_rev): |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1069 if issnapshot(rev): |
49679
b670eb3dd6c9
delta-find: use sets instead of list in the snapshot cache
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49678
diff
changeset
|
1070 cache[deltaparent(rev)].add(rev) |
49678
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1071 |
efbbc2f9121e
delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49677
diff
changeset
|
1072 |
48946
642e31cb55f0
py3: use class X: instead of class X(object):
Gregory Szorc <gregory.szorc@gmail.com>
parents:
48875
diff
changeset
|
1073 class deltacomputer: |
49610
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
1074 def __init__( |
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
1075 self, |
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
1076 revlog, |
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
1077 write_debug=None, |
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
1078 debug_search=False, |
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
1079 debug_info=None, |
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
1080 ): |
35738
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1081 self.revlog = revlog |
49226
e6b7c6fbeb48
deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
1082 self._write_debug = write_debug |
50649
c84cc0ac77e4
delta-fine: use the `_debug_search` attribute directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50648
diff
changeset
|
1083 if write_debug is None: |
c84cc0ac77e4
delta-fine: use the `_debug_search` attribute directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50648
diff
changeset
|
1084 self._debug_search = False |
c84cc0ac77e4
delta-fine: use the `_debug_search` attribute directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50648
diff
changeset
|
1085 else: |
c84cc0ac77e4
delta-fine: use the `_debug_search` attribute directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50648
diff
changeset
|
1086 self._debug_search = debug_search |
49610
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
1087 self._debug_info = debug_info |
49681
c261a628e525
delta-find: use a single snapshot cache when applying a group to an object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49680
diff
changeset
|
1088 self._snapshot_cache = SnapshotCache() |
35738
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1089 |
50648
5d210ff4b657
delta-find: move the `gather_debug` logic in a property
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50357
diff
changeset
|
1090 @property |
5d210ff4b657
delta-find: move the `gather_debug` logic in a property
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50357
diff
changeset
|
1091 def _gather_debug(self): |
5d210ff4b657
delta-find: move the `gather_debug` logic in a property
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50357
diff
changeset
|
1092 return self._write_debug is not None or self._debug_info is not None |
5d210ff4b657
delta-find: move the `gather_debug` logic in a property
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50357
diff
changeset
|
1093 |
35738
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1094 def buildtext(self, revinfo, fh): |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1095 """Builds a fulltext version of a revision |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1096 |
47399
34cc102c73f5
revlog: move `revisioninfo` in `revlogutils`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47253
diff
changeset
|
1097 revinfo: revisioninfo instance that contains all needed info |
35738
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1098 fh: file handle to either the .i or the .d revlog file, |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1099 depending on whether it is inlined or not |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1100 """ |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1101 btext = revinfo.btext |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1102 if btext[0] is not None: |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1103 return btext[0] |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1104 |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1105 revlog = self.revlog |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1106 cachedelta = revinfo.cachedelta |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1107 baserev = cachedelta[0] |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1108 delta = cachedelta[1] |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1109 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1110 fulltext = btext[0] = _textfromdelta( |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1111 fh, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1112 revlog, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1113 baserev, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1114 delta, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1115 revinfo.p1, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1116 revinfo.p2, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1117 revinfo.flags, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1118 revinfo.node, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1119 ) |
39331
fd0150a3c2fe
revlogdeltas: extra fulltext building in its own function
Boris Feld <boris.feld@octobus.net>
parents:
39330
diff
changeset
|
1120 return fulltext |
35738
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1121 |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1122 def _builddeltadiff(self, base, revinfo, fh): |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1123 revlog = self.revlog |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1124 t = self.buildtext(revinfo, fh) |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1125 if revlog.iscensored(base): |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1126 # deltas based on a censored revision must replace the |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1127 # full content in one patch, so delta works everywhere |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1128 header = mdiff.replacediffheader(revlog.rawsize(base), len(t)) |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1129 delta = header + t |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1130 else: |
42780
7a89b044eea4
rawdata: update callers in delta utils
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42468
diff
changeset
|
1131 ptext = revlog.rawdata(base, _df=fh) |
35738
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1132 delta = mdiff.textdiff(ptext, t) |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1133 |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1134 return delta |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1135 |
50356
2a6949ab9d23
delta-find: add a simple safeguard to prevent bad non-general-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50355
diff
changeset
|
1136 def _builddeltainfo(self, revinfo, base, fh, target_rev=None): |
35738
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1137 # can we use the cached delta? |
42465
6e9ba867a946
delta: move some delta chain related computation earlier in deltainfo
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42464
diff
changeset
|
1138 revlog = self.revlog |
6e9ba867a946
delta: move some delta chain related computation earlier in deltainfo
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42464
diff
changeset
|
1139 chainbase = revlog.chainbase(base) |
6e9ba867a946
delta: move some delta chain related computation earlier in deltainfo
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42464
diff
changeset
|
1140 if revlog._generaldelta: |
6e9ba867a946
delta: move some delta chain related computation earlier in deltainfo
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42464
diff
changeset
|
1141 deltabase = base |
6e9ba867a946
delta: move some delta chain related computation earlier in deltainfo
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42464
diff
changeset
|
1142 else: |
50356
2a6949ab9d23
delta-find: add a simple safeguard to prevent bad non-general-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50355
diff
changeset
|
1143 if target_rev is not None and base != target_rev - 1: |
2a6949ab9d23
delta-find: add a simple safeguard to prevent bad non-general-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50355
diff
changeset
|
1144 msg = ( |
2a6949ab9d23
delta-find: add a simple safeguard to prevent bad non-general-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50355
diff
changeset
|
1145 b'general delta cannot use delta for something else ' |
2a6949ab9d23
delta-find: add a simple safeguard to prevent bad non-general-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50355
diff
changeset
|
1146 b'than `prev`: %d<-%d' |
2a6949ab9d23
delta-find: add a simple safeguard to prevent bad non-general-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50355
diff
changeset
|
1147 ) |
2a6949ab9d23
delta-find: add a simple safeguard to prevent bad non-general-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50355
diff
changeset
|
1148 msg %= (base, target_rev) |
2a6949ab9d23
delta-find: add a simple safeguard to prevent bad non-general-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50355
diff
changeset
|
1149 raise error.ProgrammingError(msg) |
42465
6e9ba867a946
delta: move some delta chain related computation earlier in deltainfo
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42464
diff
changeset
|
1150 deltabase = chainbase |
6e9ba867a946
delta: move some delta chain related computation earlier in deltainfo
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42464
diff
changeset
|
1151 snapshotdepth = None |
6e9ba867a946
delta: move some delta chain related computation earlier in deltainfo
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42464
diff
changeset
|
1152 if revlog._sparserevlog and deltabase == nullrev: |
6e9ba867a946
delta: move some delta chain related computation earlier in deltainfo
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42464
diff
changeset
|
1153 snapshotdepth = 0 |
6e9ba867a946
delta: move some delta chain related computation earlier in deltainfo
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42464
diff
changeset
|
1154 elif revlog._sparserevlog and revlog.issnapshot(deltabase): |
6e9ba867a946
delta: move some delta chain related computation earlier in deltainfo
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42464
diff
changeset
|
1155 # A delta chain should always be one full snapshot, |
6e9ba867a946
delta: move some delta chain related computation earlier in deltainfo
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42464
diff
changeset
|
1156 # zero or more semi-snapshots, and zero or more deltas |
6e9ba867a946
delta: move some delta chain related computation earlier in deltainfo
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42464
diff
changeset
|
1157 p1, p2 = revlog.rev(revinfo.p1), revlog.rev(revinfo.p2) |
6e9ba867a946
delta: move some delta chain related computation earlier in deltainfo
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42464
diff
changeset
|
1158 if deltabase not in (p1, p2) and revlog.issnapshot(deltabase): |
6e9ba867a946
delta: move some delta chain related computation earlier in deltainfo
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42464
diff
changeset
|
1159 snapshotdepth = len(revlog._deltachain(deltabase)[0]) |
39595
a911932d5003
revlog: reuse cached delta for identical base revision (issue5975)
Boris Feld <boris.feld@octobus.net>
parents:
39594
diff
changeset
|
1160 delta = None |
a911932d5003
revlog: reuse cached delta for identical base revision (issue5975)
Boris Feld <boris.feld@octobus.net>
parents:
39594
diff
changeset
|
1161 if revinfo.cachedelta: |
49675
0fca63953810
find-delta: minor preparatory change
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49674
diff
changeset
|
1162 cachebase = revinfo.cachedelta[0] |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1163 # check if the diff still apply |
39595
a911932d5003
revlog: reuse cached delta for identical base revision (issue5975)
Boris Feld <boris.feld@octobus.net>
parents:
39594
diff
changeset
|
1164 currentbase = cachebase |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1165 while ( |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1166 currentbase != nullrev |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1167 and currentbase != base |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1168 and self.revlog.length(currentbase) == 0 |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1169 ): |
39595
a911932d5003
revlog: reuse cached delta for identical base revision (issue5975)
Boris Feld <boris.feld@octobus.net>
parents:
39594
diff
changeset
|
1170 currentbase = self.revlog.deltaparent(currentbase) |
41819
688fc33e105d
storage: introduce a `revlog.reuse-external-delta` config
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
41447
diff
changeset
|
1171 if self.revlog._lazydelta and currentbase == base: |
39595
a911932d5003
revlog: reuse cached delta for identical base revision (issue5975)
Boris Feld <boris.feld@octobus.net>
parents:
39594
diff
changeset
|
1172 delta = revinfo.cachedelta[1] |
a911932d5003
revlog: reuse cached delta for identical base revision (issue5975)
Boris Feld <boris.feld@octobus.net>
parents:
39594
diff
changeset
|
1173 if delta is None: |
35738
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1174 delta = self._builddeltadiff(base, revinfo, fh) |
50649
c84cc0ac77e4
delta-fine: use the `_debug_search` attribute directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50648
diff
changeset
|
1175 if self._debug_search: |
49228
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1176 msg = b"DBG-DELTAS-SEARCH: uncompressed-delta-size=%d\n" |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1177 msg %= len(delta) |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1178 self._write_debug(msg) |
42467
c1c1872d25d1
deltas: skip if projected compressed size does not match text size constraint
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42466
diff
changeset
|
1179 # snapshotdept need to be neither None nor 0 level snapshot |
c1c1872d25d1
deltas: skip if projected compressed size does not match text size constraint
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42466
diff
changeset
|
1180 if revlog.upperboundcomp is not None and snapshotdepth: |
c1c1872d25d1
deltas: skip if projected compressed size does not match text size constraint
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42466
diff
changeset
|
1181 lowestrealisticdeltalen = len(delta) // revlog.upperboundcomp |
c1c1872d25d1
deltas: skip if projected compressed size does not match text size constraint
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42466
diff
changeset
|
1182 snapshotlimit = revinfo.textlen >> snapshotdepth |
50649
c84cc0ac77e4
delta-fine: use the `_debug_search` attribute directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50648
diff
changeset
|
1183 if self._debug_search: |
49228
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1184 msg = b"DBG-DELTAS-SEARCH: projected-lower-size=%d\n" |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1185 msg %= lowestrealisticdeltalen |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1186 self._write_debug(msg) |
42467
c1c1872d25d1
deltas: skip if projected compressed size does not match text size constraint
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42466
diff
changeset
|
1187 if snapshotlimit < lowestrealisticdeltalen: |
50649
c84cc0ac77e4
delta-fine: use the `_debug_search` attribute directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50648
diff
changeset
|
1188 if self._debug_search: |
49228
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1189 msg = b"DBG-DELTAS-SEARCH: DISCARDED (snapshot limit)\n" |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1190 self._write_debug(msg) |
42467
c1c1872d25d1
deltas: skip if projected compressed size does not match text size constraint
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42466
diff
changeset
|
1191 return None |
42468
9b5fbe5ead89
deltas: skip if projected compressed size is bigger than previous snapshot
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42467
diff
changeset
|
1192 if revlog.length(base) < lowestrealisticdeltalen: |
50649
c84cc0ac77e4
delta-fine: use the `_debug_search` attribute directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50648
diff
changeset
|
1193 if self._debug_search: |
49228
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1194 msg = b"DBG-DELTAS-SEARCH: DISCARDED (prev size)\n" |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1195 self._write_debug(msg) |
42468
9b5fbe5ead89
deltas: skip if projected compressed size is bigger than previous snapshot
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42467
diff
changeset
|
1196 return None |
35738
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1197 header, data = revlog.compress(delta) |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1198 deltalen = len(header) + len(data) |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1199 offset = revlog.end(len(revlog) - 1) |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1200 dist = deltalen + offset - revlog.start(chainbase) |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1201 chainlen, compresseddeltalen = revlog._chaininfo(base) |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1202 chainlen += 1 |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1203 compresseddeltalen += deltalen |
39154
e0da43e2f71f
revlog: compute snapshot depth on delta info
Boris Feld <boris.feld@octobus.net>
parents:
39152
diff
changeset
|
1204 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1205 return _deltainfo( |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1206 dist, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1207 deltalen, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1208 (header, data), |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1209 deltabase, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1210 chainbase, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1211 chainlen, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1212 compresseddeltalen, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1213 snapshotdepth, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1214 ) |
35738
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1215 |
47456
93f4e183b3f5
deltas: at a `target_rev` parameter to finddeltainfo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47452
diff
changeset
|
1216 def _fullsnapshotinfo(self, fh, revinfo, curr): |
39333
5d343a24bff5
revlogdeltas: always return a delta info object in finddeltainfo
Boris Feld <boris.feld@octobus.net>
parents:
39332
diff
changeset
|
1217 rawtext = self.buildtext(revinfo, fh) |
5d343a24bff5
revlogdeltas: always return a delta info object in finddeltainfo
Boris Feld <boris.feld@octobus.net>
parents:
39332
diff
changeset
|
1218 data = self.revlog.compress(rawtext) |
5d343a24bff5
revlogdeltas: always return a delta info object in finddeltainfo
Boris Feld <boris.feld@octobus.net>
parents:
39332
diff
changeset
|
1219 compresseddeltalen = deltalen = dist = len(data[1]) + len(data[0]) |
5d343a24bff5
revlogdeltas: always return a delta info object in finddeltainfo
Boris Feld <boris.feld@octobus.net>
parents:
39332
diff
changeset
|
1220 deltabase = chainbase = curr |
5d343a24bff5
revlogdeltas: always return a delta info object in finddeltainfo
Boris Feld <boris.feld@octobus.net>
parents:
39332
diff
changeset
|
1221 snapshotdepth = 0 |
5d343a24bff5
revlogdeltas: always return a delta info object in finddeltainfo
Boris Feld <boris.feld@octobus.net>
parents:
39332
diff
changeset
|
1222 chainlen = 1 |
5d343a24bff5
revlogdeltas: always return a delta info object in finddeltainfo
Boris Feld <boris.feld@octobus.net>
parents:
39332
diff
changeset
|
1223 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1224 return _deltainfo( |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1225 dist, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1226 deltalen, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1227 data, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1228 deltabase, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1229 chainbase, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1230 chainlen, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1231 compresseddeltalen, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1232 snapshotdepth, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1233 ) |
39333
5d343a24bff5
revlogdeltas: always return a delta info object in finddeltainfo
Boris Feld <boris.feld@octobus.net>
parents:
39332
diff
changeset
|
1234 |
47456
93f4e183b3f5
deltas: at a `target_rev` parameter to finddeltainfo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47452
diff
changeset
|
1235 def finddeltainfo(self, revinfo, fh, excluded_bases=None, target_rev=None): |
35738
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1236 """Find an acceptable delta against a candidate revision |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1237 |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1238 revinfo: information about the revision (instance of _revisioninfo) |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1239 fh: file handle to either the .i or the .d revlog file, |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1240 depending on whether it is inlined or not |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1241 |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1242 Returns the first acceptable candidate revision, as ordered by |
39334
507f5b1dd7c8
revlogdeltas: extract _getcandidaterevs in a function
Boris Feld <boris.feld@octobus.net>
parents:
39333
diff
changeset
|
1243 _candidategroups |
39333
5d343a24bff5
revlogdeltas: always return a delta info object in finddeltainfo
Boris Feld <boris.feld@octobus.net>
parents:
39332
diff
changeset
|
1244 |
5d343a24bff5
revlogdeltas: always return a delta info object in finddeltainfo
Boris Feld <boris.feld@octobus.net>
parents:
39332
diff
changeset
|
1245 If no suitable deltabase is found, we return delta info for a full |
5d343a24bff5
revlogdeltas: always return a delta info object in finddeltainfo
Boris Feld <boris.feld@octobus.net>
parents:
39332
diff
changeset
|
1246 snapshot. |
47401
1efe3cdef53a
revlog: add a ways to blacklist some revision when searching for a delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47399
diff
changeset
|
1247 |
1efe3cdef53a
revlog: add a ways to blacklist some revision when searching for a delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47399
diff
changeset
|
1248 `excluded_bases` is an optional set of revision that cannot be used as |
1efe3cdef53a
revlog: add a ways to blacklist some revision when searching for a delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47399
diff
changeset
|
1249 a delta base. Use this to recompute delta suitable in censor or strip |
1efe3cdef53a
revlog: add a ways to blacklist some revision when searching for a delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47399
diff
changeset
|
1250 context. |
35738
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1251 """ |
47456
93f4e183b3f5
deltas: at a `target_rev` parameter to finddeltainfo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47452
diff
changeset
|
1252 if target_rev is None: |
47457
f8330a3fc39f
censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47456
diff
changeset
|
1253 target_rev = len(self.revlog) |
47456
93f4e183b3f5
deltas: at a `target_rev` parameter to finddeltainfo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47452
diff
changeset
|
1254 |
50653
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1255 gather_debug = self._gather_debug |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1256 cachedelta = revinfo.cachedelta |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1257 revlog = self.revlog |
50657
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1258 p1r = p2r = None |
50653
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1259 |
50657
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1260 if excluded_bases is None: |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1261 excluded_bases = set() |
50653
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1262 |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1263 if gather_debug: |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1264 start = util.timer() |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1265 dbg = self._one_dbg_data() |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1266 dbg['revision'] = target_rev |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1267 target_revlog = b"UNKNOWN" |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1268 target_type = self.revlog.target[0] |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1269 target_key = self.revlog.target[1] |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1270 if target_type == KIND_CHANGELOG: |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1271 target_revlog = b'CHANGELOG:' |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1272 elif target_type == KIND_MANIFESTLOG: |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1273 target_revlog = b'MANIFESTLOG:' |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1274 if target_key: |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1275 target_revlog += b'%s:' % target_key |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1276 elif target_type == KIND_FILELOG: |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1277 target_revlog = b'FILELOG:' |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1278 if target_key: |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1279 target_revlog += b'%s:' % target_key |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1280 dbg['target-revlog'] = target_revlog |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1281 p1r = revlog.rev(revinfo.p1) |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1282 p2r = revlog.rev(revinfo.p2) |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1283 if p1r != nullrev: |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1284 p1_chain_len = revlog._chaininfo(p1r)[0] |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1285 else: |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1286 p1_chain_len = -1 |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1287 if p2r != nullrev: |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1288 p2_chain_len = revlog._chaininfo(p2r)[0] |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1289 else: |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1290 p2_chain_len = -1 |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1291 dbg['p1-chain-len'] = p1_chain_len |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1292 dbg['p2-chain-len'] = p2_chain_len |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1293 |
50654
bfb6404089a5
delta-find: gather the condition to blindly use a full snapshot together
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50653
diff
changeset
|
1294 # 1) if the revision is empty, no amount of delta can beat it |
bfb6404089a5
delta-find: gather the condition to blindly use a full snapshot together
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50653
diff
changeset
|
1295 # |
bfb6404089a5
delta-find: gather the condition to blindly use a full snapshot together
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50653
diff
changeset
|
1296 # 2) no delta for flag processor revision (see "candelta" for why) |
bfb6404089a5
delta-find: gather the condition to blindly use a full snapshot together
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50653
diff
changeset
|
1297 # not calling candelta since only one revision needs test, also to |
bfb6404089a5
delta-find: gather the condition to blindly use a full snapshot together
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50653
diff
changeset
|
1298 # avoid overhead fetching flags again. |
bfb6404089a5
delta-find: gather the condition to blindly use a full snapshot together
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50653
diff
changeset
|
1299 if not revinfo.textlen or revinfo.flags & REVIDX_RAWTEXT_CHANGING_FLAGS: |
50655
485c9410b75a
deltafind: issue debug information when we fast-path rivial case too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50654
diff
changeset
|
1300 deltainfo = self._fullsnapshotinfo(fh, revinfo, target_rev) |
485c9410b75a
deltafind: issue debug information when we fast-path rivial case too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50654
diff
changeset
|
1301 if gather_debug: |
485c9410b75a
deltafind: issue debug information when we fast-path rivial case too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50654
diff
changeset
|
1302 end = util.timer() |
485c9410b75a
deltafind: issue debug information when we fast-path rivial case too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50654
diff
changeset
|
1303 dbg['duration'] = end - start |
485c9410b75a
deltafind: issue debug information when we fast-path rivial case too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50654
diff
changeset
|
1304 dbg[ |
485c9410b75a
deltafind: issue debug information when we fast-path rivial case too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50654
diff
changeset
|
1305 'delta-base' |
485c9410b75a
deltafind: issue debug information when we fast-path rivial case too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50654
diff
changeset
|
1306 ] = deltainfo.base # pytype: disable=attribute-error |
485c9410b75a
deltafind: issue debug information when we fast-path rivial case too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50654
diff
changeset
|
1307 dbg['search_round_count'] = 0 |
50657
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1308 dbg['using-cached-base'] = False |
50655
485c9410b75a
deltafind: issue debug information when we fast-path rivial case too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50654
diff
changeset
|
1309 dbg['delta_try_count'] = 0 |
485c9410b75a
deltafind: issue debug information when we fast-path rivial case too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50654
diff
changeset
|
1310 dbg['type'] = b"full" |
485c9410b75a
deltafind: issue debug information when we fast-path rivial case too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50654
diff
changeset
|
1311 dbg['snapshot-depth'] = 0 |
485c9410b75a
deltafind: issue debug information when we fast-path rivial case too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50654
diff
changeset
|
1312 self._dbg_process_data(dbg) |
485c9410b75a
deltafind: issue debug information when we fast-path rivial case too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50654
diff
changeset
|
1313 return deltainfo |
39085
dbb3e9e44fce
revlog: do not search for delta for empty content
Boris Feld <boris.feld@octobus.net>
parents:
39084
diff
changeset
|
1314 |
50657
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1315 deltainfo = None |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1316 |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1317 # If this source delta are to be forcibly reuse, let us comply early. |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1318 if ( |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1319 revlog._generaldelta |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1320 and revinfo.cachedelta is not None |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1321 and revinfo.cachedelta[2] == DELTA_BASE_REUSE_FORCE |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1322 ): |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1323 base = revinfo.cachedelta[0] |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1324 if base == nullrev: |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1325 dbg_type = b"full" |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1326 deltainfo = self._fullsnapshotinfo(fh, revinfo, target_rev) |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1327 if gather_debug: |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1328 snapshotdepth = 0 |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1329 elif base not in excluded_bases: |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1330 delta = revinfo.cachedelta[1] |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1331 header, data = revlog.compress(delta) |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1332 deltalen = len(header) + len(data) |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1333 if gather_debug: |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1334 offset = revlog.end(len(revlog) - 1) |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1335 chainbase = revlog.chainbase(base) |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1336 distance = deltalen + offset - revlog.start(chainbase) |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1337 chainlen, compresseddeltalen = revlog._chaininfo(base) |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1338 chainlen += 1 |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1339 compresseddeltalen += deltalen |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1340 if base == p1r or base == p2r: |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1341 dbg_type = b"delta" |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1342 snapshotdepth = None |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1343 elif not revlog.issnapshot(base): |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1344 snapshotdepth = None |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1345 else: |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1346 dbg_type = b"snapshot" |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1347 snapshotdepth = revlog.snapshotdepth(base) + 1 |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1348 else: |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1349 distance = None |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1350 chainbase = None |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1351 chainlen = None |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1352 compresseddeltalen = None |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1353 snapshotdepth = None |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1354 deltainfo = _deltainfo( |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1355 distance=distance, |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1356 deltalen=deltalen, |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1357 data=(header, data), |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1358 base=base, |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1359 chainbase=chainbase, |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1360 chainlen=chainlen, |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1361 compresseddeltalen=compresseddeltalen, |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1362 snapshotdepth=snapshotdepth, |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1363 ) |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1364 |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1365 if deltainfo is not None: |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1366 if gather_debug: |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1367 end = util.timer() |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1368 dbg['duration'] = end - start |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1369 dbg[ |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1370 'delta-base' |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1371 ] = deltainfo.base # pytype: disable=attribute-error |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1372 dbg['search_round_count'] = 0 |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1373 dbg['using-cached-base'] = True |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1374 dbg['delta_try_count'] = 0 |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1375 dbg['type'] = b"full" |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1376 if snapshotdepth is None: |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1377 dbg['snapshot-depth'] = 0 |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1378 else: |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1379 dbg['snapshot-depth'] = snapshotdepth |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1380 self._dbg_process_data(dbg) |
e77ca247b85b
delta-find: fix pulled-delta-reuse-policy=forced behavior
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50655
diff
changeset
|
1381 return deltainfo |
47401
1efe3cdef53a
revlog: add a ways to blacklist some revision when searching for a delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47399
diff
changeset
|
1382 |
49226
e6b7c6fbeb48
deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
1383 # count the number of different delta we tried (for debug purpose) |
e6b7c6fbeb48
deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
1384 dbg_try_count = 0 |
e6b7c6fbeb48
deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
1385 # count the number of "search round" we did. (for debug purpose) |
e6b7c6fbeb48
deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
1386 dbg_try_rounds = 0 |
e6b7c6fbeb48
deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
1387 dbg_type = b'unknown' |
e6b7c6fbeb48
deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
1388 |
50653
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1389 if p1r is None: |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1390 p1r = revlog.rev(revinfo.p1) |
f930b1b1d671
delta-find: initialize the debug information much sooner (when possible)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50652
diff
changeset
|
1391 p2r = revlog.rev(revinfo.p2) |
49226
e6b7c6fbeb48
deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
1392 |
50649
c84cc0ac77e4
delta-fine: use the `_debug_search` attribute directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50648
diff
changeset
|
1393 if self._debug_search: |
49228
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1394 msg = b"DBG-DELTAS-SEARCH: SEARCH rev=%d\n" |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1395 msg %= target_rev |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1396 self._write_debug(msg) |
49226
e6b7c6fbeb48
deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
1397 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1398 groups = _candidategroups( |
49611
2afee2176775
delta-find: expand a function definition and call before extendin it
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49610
diff
changeset
|
1399 self.revlog, |
2afee2176775
delta-find: expand a function definition and call before extendin it
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49610
diff
changeset
|
1400 revinfo.textlen, |
2afee2176775
delta-find: expand a function definition and call before extendin it
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49610
diff
changeset
|
1401 p1r, |
2afee2176775
delta-find: expand a function definition and call before extendin it
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49610
diff
changeset
|
1402 p2r, |
2afee2176775
delta-find: expand a function definition and call before extendin it
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49610
diff
changeset
|
1403 cachedelta, |
49612
e706bb41fdb3
delta-find: move pre-filtering with other pre-filtering logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49611
diff
changeset
|
1404 excluded_bases, |
e706bb41fdb3
delta-find: move pre-filtering with other pre-filtering logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49611
diff
changeset
|
1405 target_rev, |
49681
c261a628e525
delta-find: use a single snapshot cache when applying a group to an object
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49680
diff
changeset
|
1406 snapshot_cache=self._snapshot_cache, |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42992
diff
changeset
|
1407 ) |
39497
5b308a4e6d03
snapshot: use None as a stop value when looking for a good delta
Boris Feld <boris.feld@octobus.net>
parents:
39496
diff
changeset
|
1408 candidaterevs = next(groups) |
5b308a4e6d03
snapshot: use None as a stop value when looking for a good delta
Boris Feld <boris.feld@octobus.net>
parents:
39496
diff
changeset
|
1409 while candidaterevs is not None: |
49226
e6b7c6fbeb48
deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
1410 dbg_try_rounds += 1 |
50649
c84cc0ac77e4
delta-fine: use the `_debug_search` attribute directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50648
diff
changeset
|
1411 if self._debug_search: |
49228
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1412 prev = None |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1413 if deltainfo is not None: |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1414 prev = deltainfo.base |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1415 |
49608
78ba41878f2e
delta-find: add debug information about reuse of cached data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49228
diff
changeset
|
1416 if ( |
78ba41878f2e
delta-find: add debug information about reuse of cached data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49228
diff
changeset
|
1417 cachedelta is not None |
78ba41878f2e
delta-find: add debug information about reuse of cached data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49228
diff
changeset
|
1418 and len(candidaterevs) == 1 |
78ba41878f2e
delta-find: add debug information about reuse of cached data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49228
diff
changeset
|
1419 and cachedelta[0] in candidaterevs |
78ba41878f2e
delta-find: add debug information about reuse of cached data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49228
diff
changeset
|
1420 ): |
78ba41878f2e
delta-find: add debug information about reuse of cached data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49228
diff
changeset
|
1421 round_type = b"cached-delta" |
50652
ebb292ffdf4d
delta-find: fix `parents` round detection
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50651
diff
changeset
|
1422 elif p1r in candidaterevs or p2r in candidaterevs: |
49228
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1423 round_type = b"parents" |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1424 elif prev is not None and all(c < prev for c in candidaterevs): |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1425 round_type = b"refine-down" |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1426 elif prev is not None and all(c > prev for c in candidaterevs): |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1427 round_type = b"refine-up" |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1428 else: |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1429 round_type = b"search-down" |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1430 msg = b"DBG-DELTAS-SEARCH: ROUND #%d - %d candidates - %s\n" |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1431 msg %= (dbg_try_rounds, len(candidaterevs), round_type) |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1432 self._write_debug(msg) |
35738
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1433 nominateddeltas = [] |
39498
04b75f3a3f2a
snapshot: add refining logic at the findeltainfo level
Boris Feld <boris.feld@octobus.net>
parents:
39497
diff
changeset
|
1434 if deltainfo is not None: |
50649
c84cc0ac77e4
delta-fine: use the `_debug_search` attribute directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50648
diff
changeset
|
1435 if self._debug_search: |
49228
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1436 msg = ( |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1437 b"DBG-DELTAS-SEARCH: CONTENDER: rev=%d - length=%d\n" |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1438 ) |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1439 msg %= (deltainfo.base, deltainfo.deltalen) |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1440 self._write_debug(msg) |
39498
04b75f3a3f2a
snapshot: add refining logic at the findeltainfo level
Boris Feld <boris.feld@octobus.net>
parents:
39497
diff
changeset
|
1441 # if we already found a good delta, |
04b75f3a3f2a
snapshot: add refining logic at the findeltainfo level
Boris Feld <boris.feld@octobus.net>
parents:
39497
diff
changeset
|
1442 # challenge it against refined candidates |
04b75f3a3f2a
snapshot: add refining logic at the findeltainfo level
Boris Feld <boris.feld@octobus.net>
parents:
39497
diff
changeset
|
1443 nominateddeltas.append(deltainfo) |
35738
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1444 for candidaterev in candidaterevs: |
50649
c84cc0ac77e4
delta-fine: use the `_debug_search` attribute directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50648
diff
changeset
|
1445 if self._debug_search: |
49228
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1446 msg = b"DBG-DELTAS-SEARCH: CANDIDATE: rev=%d\n" |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1447 msg %= candidaterev |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1448 self._write_debug(msg) |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1449 candidate_type = None |
50652
ebb292ffdf4d
delta-find: fix `parents` round detection
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50651
diff
changeset
|
1450 if candidaterev == p1r: |
49228
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1451 candidate_type = b"p1" |
50652
ebb292ffdf4d
delta-find: fix `parents` round detection
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50651
diff
changeset
|
1452 elif candidaterev == p2r: |
49228
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1453 candidate_type = b"p2" |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1454 elif self.revlog.issnapshot(candidaterev): |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1455 candidate_type = b"snapshot-%d" |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1456 candidate_type %= self.revlog.snapshotdepth( |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1457 candidaterev |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1458 ) |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1459 |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1460 if candidate_type is not None: |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1461 msg = b"DBG-DELTAS-SEARCH: type=%s\n" |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1462 msg %= candidate_type |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1463 self._write_debug(msg) |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1464 msg = b"DBG-DELTAS-SEARCH: size=%d\n" |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1465 msg %= self.revlog.length(candidaterev) |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1466 self._write_debug(msg) |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1467 msg = b"DBG-DELTAS-SEARCH: base=%d\n" |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1468 msg %= self.revlog.deltaparent(candidaterev) |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1469 self._write_debug(msg) |
49612
e706bb41fdb3
delta-find: move pre-filtering with other pre-filtering logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49611
diff
changeset
|
1470 |
49226
e6b7c6fbeb48
deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
1471 dbg_try_count += 1 |
49228
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1472 |
50649
c84cc0ac77e4
delta-fine: use the `_debug_search` attribute directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50648
diff
changeset
|
1473 if self._debug_search: |
49228
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1474 delta_start = util.timer() |
50356
2a6949ab9d23
delta-find: add a simple safeguard to prevent bad non-general-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50355
diff
changeset
|
1475 candidatedelta = self._builddeltainfo( |
2a6949ab9d23
delta-find: add a simple safeguard to prevent bad non-general-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50355
diff
changeset
|
1476 revinfo, |
2a6949ab9d23
delta-find: add a simple safeguard to prevent bad non-general-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50355
diff
changeset
|
1477 candidaterev, |
2a6949ab9d23
delta-find: add a simple safeguard to prevent bad non-general-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50355
diff
changeset
|
1478 fh, |
2a6949ab9d23
delta-find: add a simple safeguard to prevent bad non-general-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50355
diff
changeset
|
1479 target_rev=target_rev, |
2a6949ab9d23
delta-find: add a simple safeguard to prevent bad non-general-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50355
diff
changeset
|
1480 ) |
50649
c84cc0ac77e4
delta-fine: use the `_debug_search` attribute directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50648
diff
changeset
|
1481 if self._debug_search: |
49228
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1482 delta_end = util.timer() |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1483 msg = b"DBG-DELTAS-SEARCH: delta-search-time=%f\n" |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1484 msg %= delta_end - delta_start |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1485 self._write_debug(msg) |
42466
465f2d0df9ae
deltas: accept and skip None return for delta info
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42465
diff
changeset
|
1486 if candidatedelta is not None: |
49674
5af4a0a73e4c
find-delta: rename _isgooddeltainfo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49657
diff
changeset
|
1487 if is_good_delta_info(self.revlog, candidatedelta, revinfo): |
50649
c84cc0ac77e4
delta-fine: use the `_debug_search` attribute directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50648
diff
changeset
|
1488 if self._debug_search: |
49228
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1489 msg = b"DBG-DELTAS-SEARCH: DELTA: length=%d (GOOD)\n" |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1490 msg %= candidatedelta.deltalen |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1491 self._write_debug(msg) |
42466
465f2d0df9ae
deltas: accept and skip None return for delta info
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
42465
diff
changeset
|
1492 nominateddeltas.append(candidatedelta) |
50649
c84cc0ac77e4
delta-fine: use the `_debug_search` attribute directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50648
diff
changeset
|
1493 elif self._debug_search: |
49228
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1494 msg = b"DBG-DELTAS-SEARCH: DELTA: length=%d (BAD)\n" |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1495 msg %= candidatedelta.deltalen |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1496 self._write_debug(msg) |
50649
c84cc0ac77e4
delta-fine: use the `_debug_search` attribute directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50648
diff
changeset
|
1497 elif self._debug_search: |
49228
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1498 msg = b"DBG-DELTAS-SEARCH: NO-DELTA\n" |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1499 self._write_debug(msg) |
35738
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1500 if nominateddeltas: |
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1501 deltainfo = min(nominateddeltas, key=lambda x: x.deltalen) |
39498
04b75f3a3f2a
snapshot: add refining logic at the findeltainfo level
Boris Feld <boris.feld@octobus.net>
parents:
39497
diff
changeset
|
1502 if deltainfo is not None: |
04b75f3a3f2a
snapshot: add refining logic at the findeltainfo level
Boris Feld <boris.feld@octobus.net>
parents:
39497
diff
changeset
|
1503 candidaterevs = groups.send(deltainfo.base) |
04b75f3a3f2a
snapshot: add refining logic at the findeltainfo level
Boris Feld <boris.feld@octobus.net>
parents:
39497
diff
changeset
|
1504 else: |
04b75f3a3f2a
snapshot: add refining logic at the findeltainfo level
Boris Feld <boris.feld@octobus.net>
parents:
39497
diff
changeset
|
1505 candidaterevs = next(groups) |
35738
f90f6fd130c1
revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents:
35737
diff
changeset
|
1506 |
39333
5d343a24bff5
revlogdeltas: always return a delta info object in finddeltainfo
Boris Feld <boris.feld@octobus.net>
parents:
39332
diff
changeset
|
1507 if deltainfo is None: |
49226
e6b7c6fbeb48
deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
1508 dbg_type = b"full" |
47456
93f4e183b3f5
deltas: at a `target_rev` parameter to finddeltainfo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47452
diff
changeset
|
1509 deltainfo = self._fullsnapshotinfo(fh, revinfo, target_rev) |
49228
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1510 elif deltainfo.snapshotdepth: # pytype: disable=attribute-error |
49226
e6b7c6fbeb48
deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
1511 dbg_type = b"snapshot" |
e6b7c6fbeb48
deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
1512 else: |
e6b7c6fbeb48
deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
1513 dbg_type = b"delta" |
e6b7c6fbeb48
deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
1514 |
49610
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
1515 if gather_debug: |
49226
e6b7c6fbeb48
deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
1516 end = util.timer() |
49775
d57b966cdeb1
delta-find: properly report full snapshot used from cache as such
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49768
diff
changeset
|
1517 if dbg_type == b'full': |
d57b966cdeb1
delta-find: properly report full snapshot used from cache as such
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49768
diff
changeset
|
1518 used_cached = ( |
d57b966cdeb1
delta-find: properly report full snapshot used from cache as such
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49768
diff
changeset
|
1519 cachedelta is not None |
d57b966cdeb1
delta-find: properly report full snapshot used from cache as such
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49768
diff
changeset
|
1520 and dbg_try_rounds == 0 |
d57b966cdeb1
delta-find: properly report full snapshot used from cache as such
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49768
diff
changeset
|
1521 and dbg_try_count == 0 |
d57b966cdeb1
delta-find: properly report full snapshot used from cache as such
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49768
diff
changeset
|
1522 and cachedelta[0] == nullrev |
d57b966cdeb1
delta-find: properly report full snapshot used from cache as such
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49768
diff
changeset
|
1523 ) |
d57b966cdeb1
delta-find: properly report full snapshot used from cache as such
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49768
diff
changeset
|
1524 else: |
d57b966cdeb1
delta-find: properly report full snapshot used from cache as such
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49768
diff
changeset
|
1525 used_cached = ( |
d57b966cdeb1
delta-find: properly report full snapshot used from cache as such
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49768
diff
changeset
|
1526 cachedelta is not None |
d57b966cdeb1
delta-find: properly report full snapshot used from cache as such
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49768
diff
changeset
|
1527 and dbg_try_rounds == 1 |
d57b966cdeb1
delta-find: properly report full snapshot used from cache as such
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49768
diff
changeset
|
1528 and dbg_try_count == 1 |
d57b966cdeb1
delta-find: properly report full snapshot used from cache as such
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49768
diff
changeset
|
1529 and deltainfo.base == cachedelta[0] |
d57b966cdeb1
delta-find: properly report full snapshot used from cache as such
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49768
diff
changeset
|
1530 ) |
50651
d1dc4a03125e
delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50650
diff
changeset
|
1531 dbg['duration'] = end - start |
d1dc4a03125e
delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50650
diff
changeset
|
1532 dbg[ |
d1dc4a03125e
delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50650
diff
changeset
|
1533 'delta-base' |
d1dc4a03125e
delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50650
diff
changeset
|
1534 ] = deltainfo.base # pytype: disable=attribute-error |
d1dc4a03125e
delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50650
diff
changeset
|
1535 dbg['search_round_count'] = dbg_try_rounds |
d1dc4a03125e
delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50650
diff
changeset
|
1536 dbg['using-cached-base'] = used_cached |
d1dc4a03125e
delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50650
diff
changeset
|
1537 dbg['delta_try_count'] = dbg_try_count |
d1dc4a03125e
delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50650
diff
changeset
|
1538 dbg['type'] = dbg_type |
49228
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1539 if ( |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1540 deltainfo.snapshotdepth # pytype: disable=attribute-error |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1541 is not None |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1542 ): |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1543 dbg[ |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1544 'snapshot-depth' |
b909dd35d9ab
deltas: add a debug-delta-find command to analyse delta search
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49226
diff
changeset
|
1545 ] = deltainfo.snapshotdepth # pytype: disable=attribute-error |
49226
e6b7c6fbeb48
deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
1546 else: |
e6b7c6fbeb48
deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
1547 dbg['snapshot-depth'] = 0 |
50650
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1548 self._dbg_process_data(dbg) |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1549 return deltainfo |
49226
e6b7c6fbeb48
deltas: add code to display information about the result of `finddeltainfo`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
1550 |
50651
d1dc4a03125e
delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50650
diff
changeset
|
1551 def _one_dbg_data(self): |
d1dc4a03125e
delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50650
diff
changeset
|
1552 return { |
d1dc4a03125e
delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50650
diff
changeset
|
1553 'duration': None, |
d1dc4a03125e
delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50650
diff
changeset
|
1554 'revision': None, |
d1dc4a03125e
delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50650
diff
changeset
|
1555 'delta-base': None, |
d1dc4a03125e
delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50650
diff
changeset
|
1556 'search_round_count': None, |
d1dc4a03125e
delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50650
diff
changeset
|
1557 'using-cached-base': None, |
d1dc4a03125e
delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50650
diff
changeset
|
1558 'delta_try_count': None, |
d1dc4a03125e
delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50650
diff
changeset
|
1559 'type': None, |
d1dc4a03125e
delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50650
diff
changeset
|
1560 'p1-chain-len': None, |
d1dc4a03125e
delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50650
diff
changeset
|
1561 'p2-chain-len': None, |
d1dc4a03125e
delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50650
diff
changeset
|
1562 'snapshot-depth': None, |
d1dc4a03125e
delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50650
diff
changeset
|
1563 'target-revlog': None, |
d1dc4a03125e
delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50650
diff
changeset
|
1564 } |
d1dc4a03125e
delta-find: intrduce a `_one_dbg_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50650
diff
changeset
|
1565 |
50650
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1566 def _dbg_process_data(self, dbg): |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1567 if self._debug_info is not None: |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1568 self._debug_info.append(dbg) |
49610
35d4c2124073
debug: add an option to display statistic about a unbundling operation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49608
diff
changeset
|
1569 |
50650
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1570 if self._write_debug is not None: |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1571 msg = ( |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1572 b"DBG-DELTAS:" |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1573 b" %-12s" |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1574 b" rev=%d:" |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1575 b" delta-base=%d" |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1576 b" is-cached=%d" |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1577 b" - search-rounds=%d" |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1578 b" try-count=%d" |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1579 b" - delta-type=%-6s" |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1580 b" snap-depth=%d" |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1581 b" - p1-chain-length=%d" |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1582 b" p2-chain-length=%d" |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1583 b" - duration=%f" |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1584 b"\n" |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1585 ) |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1586 msg %= ( |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1587 dbg["target-revlog"], |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1588 dbg["revision"], |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1589 dbg["delta-base"], |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1590 dbg["using-cached-base"], |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1591 dbg["search_round_count"], |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1592 dbg["delta_try_count"], |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1593 dbg["type"], |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1594 dbg["snapshot-depth"], |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1595 dbg["p1-chain-len"], |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1596 dbg["p2-chain-len"], |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1597 dbg["duration"], |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1598 ) |
4d84b6d52e93
delta-find: move final debug processing in a `_dbg_process_data` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50649
diff
changeset
|
1599 self._write_debug(msg) |
47452
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1600 |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1601 |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1602 def delta_compression(default_compression_header, deltainfo): |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1603 """return (COMPRESSION_MODE, deltainfo) |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1604 |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1605 used by revlog v2+ format to dispatch between PLAIN and DEFAULT |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1606 compression. |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1607 """ |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1608 h, d = deltainfo.data |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1609 compression_mode = COMP_MODE_INLINE |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1610 if not h and not d: |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1611 # not data to store at all... declare them uncompressed |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1612 compression_mode = COMP_MODE_PLAIN |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1613 elif not h: |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1614 t = d[0:1] |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1615 if t == b'\0': |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1616 compression_mode = COMP_MODE_PLAIN |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1617 elif t == default_compression_header: |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1618 compression_mode = COMP_MODE_DEFAULT |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1619 elif h == b'u': |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1620 # we have a more efficient way to declare uncompressed |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1621 h = b'' |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1622 compression_mode = COMP_MODE_PLAIN |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1623 deltainfo = drop_u_compression(deltainfo) |
c6844912c327
revlog: factor the logic to determine the delta compression out
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47401
diff
changeset
|
1624 return compression_mode, deltainfo |