annotate mercurial/revlog.py @ 38716:c67093e81a3e

revlog: extract `deltainfo.distance` for future conditional redefinition This commit exist to make the next one clearer.
author Paul Morelle <paul.morelle@octobus.net>
date Mon, 04 Jun 2018 12:12:00 +0200
parents 4ac3c2078567
children aa21a9ad46ea
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
8226
8b2cd04a6e97 put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents: 8225
diff changeset
1 # revlog.py - storage back-end for mercurial
8b2cd04a6e97 put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents: 8225
diff changeset
2 #
8b2cd04a6e97 put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents: 8225
diff changeset
3 # Copyright 2005-2007 Matt Mackall <mpm@selenic.com>
8b2cd04a6e97 put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents: 8225
diff changeset
4 #
8b2cd04a6e97 put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents: 8225
diff changeset
5 # This software may be used and distributed according to the terms of the
10263
25e572394f5c Update license to GPLv2+
Matt Mackall <mpm@selenic.com>
parents: 10047
diff changeset
6 # GNU General Public License version 2 or any later version.
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
7
8227
0a9542703300 turn some comments back into module docstrings
Martin Geisler <mg@lazybytes.net>
parents: 8226
diff changeset
8 """Storage back-end for Mercurial.
0a9542703300 turn some comments back into module docstrings
Martin Geisler <mg@lazybytes.net>
parents: 8226
diff changeset
9
0a9542703300 turn some comments back into module docstrings
Martin Geisler <mg@lazybytes.net>
parents: 8226
diff changeset
10 This provides efficient delta storage with O(1) retrieve and append
0a9542703300 turn some comments back into module docstrings
Martin Geisler <mg@lazybytes.net>
parents: 8226
diff changeset
11 and O(changes) merge between branches.
0a9542703300 turn some comments back into module docstrings
Martin Geisler <mg@lazybytes.net>
parents: 8226
diff changeset
12 """
0a9542703300 turn some comments back into module docstrings
Martin Geisler <mg@lazybytes.net>
parents: 8226
diff changeset
13
27361
29f50344fa83 revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27251
diff changeset
14 from __future__ import absolute_import
29f50344fa83 revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27251
diff changeset
15
25113
0ca8410ea345 util: drop alias for collections.deque
Martin von Zweigbergk <martinvonz@google.com>
parents: 24454
diff changeset
16 import collections
35973
4d66993bdcff revlog: add a _datareadfp context manager for data access needs
Boris Feld <boris.feld@octobus.net>
parents: 35972
diff changeset
17 import contextlib
27361
29f50344fa83 revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27251
diff changeset
18 import errno
29339
a9e010cd66e1 revlog: use hashlib.sha1 directly instead of through util
Augie Fackler <raf@durin42.com>
parents: 27650
diff changeset
19 import hashlib
34880
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
20 import heapq
27430
e240e914d226 revlog: seek to end of file before writing (issue4943)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26705
diff changeset
21 import os
37442
0596d27457c6 revlog: move parsemeta() and packmeta() from filelog (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37084
diff changeset
22 import re
27361
29f50344fa83 revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27251
diff changeset
23 import struct
29f50344fa83 revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27251
diff changeset
24 import zlib
29f50344fa83 revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27251
diff changeset
25
29f50344fa83 revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27251
diff changeset
26 # import stuff from node for others to import from revlog
29f50344fa83 revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27251
diff changeset
27 from .node import (
29f50344fa83 revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27251
diff changeset
28 bin,
29f50344fa83 revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27251
diff changeset
29 hex,
29f50344fa83 revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27251
diff changeset
30 nullid,
29f50344fa83 revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27251
diff changeset
31 nullrev,
37449
a0d71618074f revlog: detect pseudo file nodeids to raise WdirUnsupported exception
Yuya Nishihara <yuya@tcha.org>
parents: 37443
diff changeset
32 wdirfilenodeids,
32684
af854b1b36f8 revlog: add support for partial matching of wdir node id
Yuya Nishihara <yuya@tcha.org>
parents: 32659
diff changeset
33 wdirhex,
32659
7b17f9de6d3e revlog: map rev(wdirid) to WdirUnsupported exception
Yuya Nishihara <yuya@tcha.org>
parents: 32443
diff changeset
34 wdirid,
32403
a28c76e1cea9 revlog: raise WdirUnsupported when wdirrev is passed
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32393
diff changeset
35 wdirrev,
27361
29f50344fa83 revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27251
diff changeset
36 )
29f50344fa83 revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27251
diff changeset
37 from .i18n import _
35638
edc9330acac1 revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents: 35637
diff changeset
38 from .thirdparty import (
edc9330acac1 revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents: 35637
diff changeset
39 attr,
edc9330acac1 revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents: 35637
diff changeset
40 )
27361
29f50344fa83 revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27251
diff changeset
41 from . import (
29f50344fa83 revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27251
diff changeset
42 ancestor,
29f50344fa83 revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27251
diff changeset
43 error,
29f50344fa83 revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27251
diff changeset
44 mdiff,
32372
df448de7cf3b parsers: switch to policy importer
Yuya Nishihara <yuya@tcha.org>
parents: 32315
diff changeset
45 policy,
31574
a8e55d6f1d67 revlog: use pycompat.maplist to eagerly evaluate map on Python 3
Augie Fackler <augie@google.com>
parents: 31504
diff changeset
46 pycompat,
27361
29f50344fa83 revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27251
diff changeset
47 templatefilters,
29f50344fa83 revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27251
diff changeset
48 util,
29f50344fa83 revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27251
diff changeset
49 )
37084
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module
Yuya Nishihara <yuya@tcha.org>
parents: 36760
diff changeset
50 from .utils import (
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module
Yuya Nishihara <yuya@tcha.org>
parents: 36760
diff changeset
51 stringutil,
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module
Yuya Nishihara <yuya@tcha.org>
parents: 36760
diff changeset
52 )
36
da28286bf6b7 Add smart node lookup by substring or by rev number
mpm@selenic.com
parents: 26
diff changeset
53
32372
df448de7cf3b parsers: switch to policy importer
Yuya Nishihara <yuya@tcha.org>
parents: 32315
diff changeset
54 parsers = policy.importmod(r'parsers')
df448de7cf3b parsers: switch to policy importer
Yuya Nishihara <yuya@tcha.org>
parents: 32315
diff changeset
55
30817
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
56 # Aliased for performance.
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
57 _zlibdecompress = zlib.decompress
5007
3addf4531643 revlog: localize some fastpath functions
Matt Mackall <mpm@selenic.com>
parents: 5006
diff changeset
58
11746
46ac30b17978 revlog: add shallow header flag
Vishakh H <vsh426@gmail.com>
parents: 11745
diff changeset
59 # revlog header flags
2072
74d3f5336b66 Implement revlogng.
mason@suse.com
parents: 2002
diff changeset
60 REVLOGV0 = 0
32315
67026d65a4fc revlog: rename constants (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32307
diff changeset
61 REVLOGV1 = 1
32697
19b9fc40cc51 revlog: skeleton support for version 2 revlogs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32684
diff changeset
62 # Dummy value until file format is finalized.
19b9fc40cc51 revlog: skeleton support for version 2 revlogs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32684
diff changeset
63 # Reminder: change the bounds check in revlog.__init__ when this is changed.
19b9fc40cc51 revlog: skeleton support for version 2 revlogs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32684
diff changeset
64 REVLOGV2 = 0xDEAD
32315
67026d65a4fc revlog: rename constants (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32307
diff changeset
65 FLAG_INLINE_DATA = (1 << 16)
67026d65a4fc revlog: rename constants (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32307
diff changeset
66 FLAG_GENERALDELTA = (1 << 17)
67026d65a4fc revlog: rename constants (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32307
diff changeset
67 REVLOG_DEFAULT_FLAGS = FLAG_INLINE_DATA
67026d65a4fc revlog: rename constants (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32307
diff changeset
68 REVLOG_DEFAULT_FORMAT = REVLOGV1
2222
c9e264b115e6 Use revlogng and inlined data files by default
mason@suse.com
parents: 2177
diff changeset
69 REVLOG_DEFAULT_VERSION = REVLOG_DEFAULT_FORMAT | REVLOG_DEFAULT_FLAGS
32315
67026d65a4fc revlog: rename constants (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32307
diff changeset
70 REVLOGV1_FLAGS = FLAG_INLINE_DATA | FLAG_GENERALDELTA
32697
19b9fc40cc51 revlog: skeleton support for version 2 revlogs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32684
diff changeset
71 REVLOGV2_FLAGS = REVLOGV1_FLAGS
11746
46ac30b17978 revlog: add shallow header flag
Vishakh H <vsh426@gmail.com>
parents: 11745
diff changeset
72
46ac30b17978 revlog: add shallow header flag
Vishakh H <vsh426@gmail.com>
parents: 11745
diff changeset
73 # revlog index flags
23855
4f23081c901e revlog: define censored flag for revlogng index
Mike Edgar <adgar@google.com>
parents: 23338
diff changeset
74 REVIDX_ISCENSORED = (1 << 15) # revision has censor metadata, must be verified
30829
08b34c3a6f74 revlog: give EXTSTORED flag value to narrowhg
Martin von Zweigbergk <martinvonz@google.com>
parents: 30818
diff changeset
75 REVIDX_ELLIPSIS = (1 << 14) # revision hash does not match data (narrowhg)
08b34c3a6f74 revlog: give EXTSTORED flag value to narrowhg
Martin von Zweigbergk <martinvonz@google.com>
parents: 30818
diff changeset
76 REVIDX_EXTSTORED = (1 << 13) # revision data is stored externally
23855
4f23081c901e revlog: define censored flag for revlogng index
Mike Edgar <adgar@google.com>
parents: 23338
diff changeset
77 REVIDX_DEFAULT_FLAGS = 0
30745
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
78 # stable order in which flags need to be processed and their processors applied
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
79 REVIDX_FLAGS_ORDER = [
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
80 REVIDX_ISCENSORED,
30829
08b34c3a6f74 revlog: give EXTSTORED flag value to narrowhg
Martin von Zweigbergk <martinvonz@google.com>
parents: 30818
diff changeset
81 REVIDX_ELLIPSIS,
30746
9cb0bb0f29f0 revlog: REVIDX_EXTSTORED flag
Remi Chaintron <remi@fb.com>
parents: 30745
diff changeset
82 REVIDX_EXTSTORED,
30745
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
83 ]
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
84 REVIDX_KNOWN_FLAGS = util.bitsfrom(REVIDX_FLAGS_ORDER)
36743
d031609b3cb7 changegroup: do not delta lfs revisions
Jun Wu <quark@fb.com>
parents: 35738
diff changeset
85 # bitmark for flags that could cause rawdata content change
d031609b3cb7 changegroup: do not delta lfs revisions
Jun Wu <quark@fb.com>
parents: 35738
diff changeset
86 REVIDX_RAWTEXT_CHANGING_FLAGS = REVIDX_ISCENSORED | REVIDX_EXTSTORED
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
87
10916
9c84395a338e add documentation for revlog._prereadsize
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 10914
diff changeset
88 # max size of revlog with inline data
9c84395a338e add documentation for revlog._prereadsize
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 10914
diff changeset
89 _maxinline = 131072
13253
61c9bc3da402 revlog: remove lazy index
Matt Mackall <mpm@selenic.com>
parents: 13239
diff changeset
90 _chunksize = 1048576
10913
f2ecc5733c89 revlog: factor out _maxinline global.
Greg Ward <greg-hg@gerg.ca>
parents: 10404
diff changeset
91
7633
08cabecfa8a8 errors: move revlog errors
Matt Mackall <mpm@selenic.com>
parents: 7365
diff changeset
92 RevlogError = error.RevlogError
08cabecfa8a8 errors: move revlog errors
Matt Mackall <mpm@selenic.com>
parents: 7365
diff changeset
93 LookupError = error.LookupError
22934
8a096d4d0862 revlog: support importing censored file revision tombstones
Mike Edgar <adgar@google.com>
parents: 22785
diff changeset
94 CensoredNodeError = error.CensoredNodeError
30745
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
95 ProgrammingError = error.ProgrammingError
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
96
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
97 # Store flag processors (cf. 'addflagprocessor()' to register)
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
98 _flagprocessors = {
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
99 REVIDX_ISCENSORED: None,
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
100 }
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
101
37442
0596d27457c6 revlog: move parsemeta() and packmeta() from filelog (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37084
diff changeset
102 _mdre = re.compile('\1\n')
0596d27457c6 revlog: move parsemeta() and packmeta() from filelog (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37084
diff changeset
103 def parsemeta(text):
0596d27457c6 revlog: move parsemeta() and packmeta() from filelog (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37084
diff changeset
104 """return (metadatadict, metadatasize)"""
0596d27457c6 revlog: move parsemeta() and packmeta() from filelog (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37084
diff changeset
105 # text can be buffer, so we can't use .startswith or .index
0596d27457c6 revlog: move parsemeta() and packmeta() from filelog (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37084
diff changeset
106 if text[:2] != '\1\n':
0596d27457c6 revlog: move parsemeta() and packmeta() from filelog (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37084
diff changeset
107 return None, None
0596d27457c6 revlog: move parsemeta() and packmeta() from filelog (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37084
diff changeset
108 s = _mdre.search(text, 2).start()
0596d27457c6 revlog: move parsemeta() and packmeta() from filelog (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37084
diff changeset
109 mtext = text[2:s]
0596d27457c6 revlog: move parsemeta() and packmeta() from filelog (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37084
diff changeset
110 meta = {}
0596d27457c6 revlog: move parsemeta() and packmeta() from filelog (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37084
diff changeset
111 for l in mtext.splitlines():
0596d27457c6 revlog: move parsemeta() and packmeta() from filelog (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37084
diff changeset
112 k, v = l.split(": ", 1)
0596d27457c6 revlog: move parsemeta() and packmeta() from filelog (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37084
diff changeset
113 meta[k] = v
0596d27457c6 revlog: move parsemeta() and packmeta() from filelog (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37084
diff changeset
114 return meta, (s + 2)
0596d27457c6 revlog: move parsemeta() and packmeta() from filelog (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37084
diff changeset
115
0596d27457c6 revlog: move parsemeta() and packmeta() from filelog (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37084
diff changeset
116 def packmeta(meta, text):
0596d27457c6 revlog: move parsemeta() and packmeta() from filelog (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37084
diff changeset
117 keys = sorted(meta)
0596d27457c6 revlog: move parsemeta() and packmeta() from filelog (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37084
diff changeset
118 metatext = "".join("%s: %s\n" % (k, meta[k]) for k in keys)
0596d27457c6 revlog: move parsemeta() and packmeta() from filelog (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37084
diff changeset
119 return "\1\n%s\1\n%s" % (metatext, text)
0596d27457c6 revlog: move parsemeta() and packmeta() from filelog (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37084
diff changeset
120
37443
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
121 def _censoredtext(text):
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
122 m, offs = parsemeta(text)
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
123 return m and "censored" in m
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
124
30745
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
125 def addflagprocessor(flag, processor):
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
126 """Register a flag processor on a revision data flag.
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
127
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
128 Invariant:
36743
d031609b3cb7 changegroup: do not delta lfs revisions
Jun Wu <quark@fb.com>
parents: 35738
diff changeset
129 - Flags need to be defined in REVIDX_KNOWN_FLAGS and REVIDX_FLAGS_ORDER,
d031609b3cb7 changegroup: do not delta lfs revisions
Jun Wu <quark@fb.com>
parents: 35738
diff changeset
130 and REVIDX_RAWTEXT_CHANGING_FLAGS if they can alter rawtext.
30745
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
131 - Only one flag processor can be registered on a specific flag.
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
132 - flagprocessors must be 3-tuples of functions (read, write, raw) with the
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
133 following signatures:
31749
17d0dab7b2b6 revlog: clarify flagprocessor documentation
Jun Wu <quark@fb.com>
parents: 31722
diff changeset
134 - (read) f(self, rawtext) -> text, bool
17d0dab7b2b6 revlog: clarify flagprocessor documentation
Jun Wu <quark@fb.com>
parents: 31722
diff changeset
135 - (write) f(self, text) -> rawtext, bool
17d0dab7b2b6 revlog: clarify flagprocessor documentation
Jun Wu <quark@fb.com>
parents: 31722
diff changeset
136 - (raw) f(self, rawtext) -> bool
17d0dab7b2b6 revlog: clarify flagprocessor documentation
Jun Wu <quark@fb.com>
parents: 31722
diff changeset
137 "text" is presented to the user. "rawtext" is stored in revlog data, not
17d0dab7b2b6 revlog: clarify flagprocessor documentation
Jun Wu <quark@fb.com>
parents: 31722
diff changeset
138 directly visible to the user.
30745
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
139 The boolean returned by these transforms is used to determine whether
31749
17d0dab7b2b6 revlog: clarify flagprocessor documentation
Jun Wu <quark@fb.com>
parents: 31722
diff changeset
140 the returned text can be used for hash integrity checking. For example,
17d0dab7b2b6 revlog: clarify flagprocessor documentation
Jun Wu <quark@fb.com>
parents: 31722
diff changeset
141 if "write" returns False, then "text" is used to generate hash. If
17d0dab7b2b6 revlog: clarify flagprocessor documentation
Jun Wu <quark@fb.com>
parents: 31722
diff changeset
142 "write" returns True, that basically means "rawtext" returned by "write"
17d0dab7b2b6 revlog: clarify flagprocessor documentation
Jun Wu <quark@fb.com>
parents: 31722
diff changeset
143 should be used to generate hash. Usually, "write" and "read" return
17d0dab7b2b6 revlog: clarify flagprocessor documentation
Jun Wu <quark@fb.com>
parents: 31722
diff changeset
144 different booleans. And "raw" returns a same boolean as "write".
30745
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
145
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
146 Note: The 'raw' transform is used for changegroup generation and in some
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
147 debug commands. In this case the transform only indicates whether the
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
148 contents can be used for hash integrity checks.
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
149 """
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
150 if not flag & REVIDX_KNOWN_FLAGS:
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
151 msg = _("cannot register processor on unknown flag '%#x'.") % (flag)
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
152 raise ProgrammingError(msg)
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
153 if flag not in REVIDX_FLAGS_ORDER:
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
154 msg = _("flag '%#x' undefined in REVIDX_FLAGS_ORDER.") % (flag)
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
155 raise ProgrammingError(msg)
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
156 if flag in _flagprocessors:
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
157 msg = _("cannot register multiple processors on flag '%#x'.") % (flag)
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
158 raise error.Abort(msg)
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
159 _flagprocessors[flag] = processor
6703
bacfee67c1a9 LookupError should have same __str__ as RevlogError
Alexander Solovyov <piranha@piranha.org.ua>
parents: 6700
diff changeset
160
4987
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
161 def getoffset(q):
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
162 return int(q >> 16)
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
163
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
164 def gettype(q):
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
165 return int(q & 0xFFFF)
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
166
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
167 def offset_type(offset, type):
30543
03fae9048fa1 revlog: ensure that flags do not overflow 2 bytes
Cotizo Sima <cotizo@fb.com>
parents: 30391
diff changeset
168 if (type & ~REVIDX_KNOWN_FLAGS) != 0:
03fae9048fa1 revlog: ensure that flags do not overflow 2 bytes
Cotizo Sima <cotizo@fb.com>
parents: 30391
diff changeset
169 raise ValueError('unknown revlog index flags')
31504
73aa13bc8dac revlog: use int instead of long
Augie Fackler <augie@google.com>
parents: 31369
diff changeset
170 return int(int(offset) << 16 | type)
4987
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
171
29339
a9e010cd66e1 revlog: use hashlib.sha1 directly instead of through util
Augie Fackler <raf@durin42.com>
parents: 27650
diff changeset
172 _nullhash = hashlib.sha1(nullid)
7883
c63c30ae9e39 revlog: faster hash computation when one of the parent node is null
Nicolas Dumazet <nicdumz.commits@gmail.com>
parents: 7874
diff changeset
173
1091
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
174 def hash(text, p1, p2):
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
175 """generate a hash from the given text and its parent hashes
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
176
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
177 This hash combines both the current file contents and its history
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
178 in a manner that makes it easy to distinguish nodes with the same
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
179 content in the revision graph.
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
180 """
7883
c63c30ae9e39 revlog: faster hash computation when one of the parent node is null
Nicolas Dumazet <nicdumz.commits@gmail.com>
parents: 7874
diff changeset
181 # As of now, if one of the parent node is null, p2 is null
c63c30ae9e39 revlog: faster hash computation when one of the parent node is null
Nicolas Dumazet <nicdumz.commits@gmail.com>
parents: 7874
diff changeset
182 if p2 == nullid:
c63c30ae9e39 revlog: faster hash computation when one of the parent node is null
Nicolas Dumazet <nicdumz.commits@gmail.com>
parents: 7874
diff changeset
183 # deep copy of a hash is faster than creating one
22784
0f4e655136ef revlog: mark nullhash as module-private
Augie Fackler <raf@durin42.com>
parents: 22389
diff changeset
184 s = _nullhash.copy()
7883
c63c30ae9e39 revlog: faster hash computation when one of the parent node is null
Nicolas Dumazet <nicdumz.commits@gmail.com>
parents: 7874
diff changeset
185 s.update(p1)
c63c30ae9e39 revlog: faster hash computation when one of the parent node is null
Nicolas Dumazet <nicdumz.commits@gmail.com>
parents: 7874
diff changeset
186 else:
c63c30ae9e39 revlog: faster hash computation when one of the parent node is null
Nicolas Dumazet <nicdumz.commits@gmail.com>
parents: 7874
diff changeset
187 # none of the parent nodes are nullid
33391
943b8c37f49d revlog: micro-optimize the computation of hashes
Alex Gaynor <agaynor@mozilla.com>
parents: 33260
diff changeset
188 if p1 < p2:
943b8c37f49d revlog: micro-optimize the computation of hashes
Alex Gaynor <agaynor@mozilla.com>
parents: 33260
diff changeset
189 a = p1
943b8c37f49d revlog: micro-optimize the computation of hashes
Alex Gaynor <agaynor@mozilla.com>
parents: 33260
diff changeset
190 b = p2
943b8c37f49d revlog: micro-optimize the computation of hashes
Alex Gaynor <agaynor@mozilla.com>
parents: 33260
diff changeset
191 else:
943b8c37f49d revlog: micro-optimize the computation of hashes
Alex Gaynor <agaynor@mozilla.com>
parents: 33260
diff changeset
192 a = p2
943b8c37f49d revlog: micro-optimize the computation of hashes
Alex Gaynor <agaynor@mozilla.com>
parents: 33260
diff changeset
193 b = p1
943b8c37f49d revlog: micro-optimize the computation of hashes
Alex Gaynor <agaynor@mozilla.com>
parents: 33260
diff changeset
194 s = hashlib.sha1(a)
943b8c37f49d revlog: micro-optimize the computation of hashes
Alex Gaynor <agaynor@mozilla.com>
parents: 33260
diff changeset
195 s.update(b)
1091
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
196 s.update(text)
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
197 return s.digest()
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
198
38637
e33f784f2a44 revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents: 38636
diff changeset
199 class _testrevlog(object):
e33f784f2a44 revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents: 38636
diff changeset
200 """minimalist fake revlog to use in doctests"""
e33f784f2a44 revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents: 38636
diff changeset
201
e33f784f2a44 revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents: 38636
diff changeset
202 def __init__(self, data, density=0.5, mingap=0):
e33f784f2a44 revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents: 38636
diff changeset
203 """data is an list of revision payload boundaries"""
e33f784f2a44 revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents: 38636
diff changeset
204 self._data = data
e33f784f2a44 revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents: 38636
diff changeset
205 self._srdensitythreshold = density
e33f784f2a44 revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents: 38636
diff changeset
206 self._srmingapsize = mingap
e33f784f2a44 revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents: 38636
diff changeset
207
e33f784f2a44 revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents: 38636
diff changeset
208 def start(self, rev):
e33f784f2a44 revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents: 38636
diff changeset
209 if rev == 0:
e33f784f2a44 revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents: 38636
diff changeset
210 return 0
e33f784f2a44 revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents: 38636
diff changeset
211 return self._data[rev - 1]
e33f784f2a44 revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents: 38636
diff changeset
212
e33f784f2a44 revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents: 38636
diff changeset
213 def end(self, rev):
e33f784f2a44 revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents: 38636
diff changeset
214 return self._data[rev]
e33f784f2a44 revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents: 38636
diff changeset
215
e33f784f2a44 revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents: 38636
diff changeset
216 def length(self, rev):
e33f784f2a44 revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents: 38636
diff changeset
217 return self.end(rev) - self.start(rev)
e33f784f2a44 revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents: 38636
diff changeset
218
34898
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents: 34881
diff changeset
219 def _trimchunk(revlog, revs, startidx, endidx=None):
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents: 34881
diff changeset
220 """returns revs[startidx:endidx] without empty trailing revs
38638
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
221
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
222 Doctest Setup
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
223 >>> revlog = _testrevlog([
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
224 ... 5, #0
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
225 ... 10, #1
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
226 ... 12, #2
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
227 ... 12, #3 (empty)
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
228 ... 17, #4
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
229 ... 21, #5
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
230 ... 21, #6 (empty)
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
231 ... ])
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
232
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
233 Contiguous cases:
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
234 >>> _trimchunk(revlog, [0, 1, 2, 3, 4, 5, 6], 0)
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
235 [0, 1, 2, 3, 4, 5]
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
236 >>> _trimchunk(revlog, [0, 1, 2, 3, 4, 5, 6], 0, 5)
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
237 [0, 1, 2, 3, 4]
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
238 >>> _trimchunk(revlog, [0, 1, 2, 3, 4, 5, 6], 0, 4)
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
239 [0, 1, 2]
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
240 >>> _trimchunk(revlog, [0, 1, 2, 3, 4, 5, 6], 2, 4)
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
241 [2]
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
242 >>> _trimchunk(revlog, [0, 1, 2, 3, 4, 5, 6], 3)
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
243 [3, 4, 5]
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
244 >>> _trimchunk(revlog, [0, 1, 2, 3, 4, 5, 6], 3, 5)
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
245 [3, 4]
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
246
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
247 Discontiguous cases:
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
248 >>> _trimchunk(revlog, [1, 3, 5, 6], 0)
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
249 [1, 3, 5]
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
250 >>> _trimchunk(revlog, [1, 3, 5, 6], 0, 2)
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
251 [1]
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
252 >>> _trimchunk(revlog, [1, 3, 5, 6], 1, 3)
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
253 [3, 5]
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
254 >>> _trimchunk(revlog, [1, 3, 5, 6], 1)
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
255 [3, 5]
34898
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents: 34881
diff changeset
256 """
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents: 34881
diff changeset
257 length = revlog.length
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents: 34881
diff changeset
258
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents: 34881
diff changeset
259 if endidx is None:
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents: 34881
diff changeset
260 endidx = len(revs)
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents: 34881
diff changeset
261
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents: 34881
diff changeset
262 # Trim empty revs at the end, but never the very first revision of a chain
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents: 34881
diff changeset
263 while endidx > 1 and endidx > startidx and length(revs[endidx - 1]) == 0:
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents: 34881
diff changeset
264 endidx -= 1
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents: 34881
diff changeset
265
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents: 34881
diff changeset
266 return revs[startidx:endidx]
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents: 34881
diff changeset
267
38636
25534e0cf0d9 revlog: _segmentspan computes the byte span of a segment
Paul Morelle <paul.morelle@octobus.net>
parents: 38635
diff changeset
268 def _segmentspan(revlog, revs):
25534e0cf0d9 revlog: _segmentspan computes the byte span of a segment
Paul Morelle <paul.morelle@octobus.net>
parents: 38635
diff changeset
269 """Get the byte span of a segment of revisions
25534e0cf0d9 revlog: _segmentspan computes the byte span of a segment
Paul Morelle <paul.morelle@octobus.net>
parents: 38635
diff changeset
270
25534e0cf0d9 revlog: _segmentspan computes the byte span of a segment
Paul Morelle <paul.morelle@octobus.net>
parents: 38635
diff changeset
271 revs is a sorted array of revision numbers
38639
2dd4cf273804 revlog: add a doctest for _segmentspan
Boris Feld <boris.feld@octobus.net>
parents: 38638
diff changeset
272
2dd4cf273804 revlog: add a doctest for _segmentspan
Boris Feld <boris.feld@octobus.net>
parents: 38638
diff changeset
273 >>> revlog = _testrevlog([
2dd4cf273804 revlog: add a doctest for _segmentspan
Boris Feld <boris.feld@octobus.net>
parents: 38638
diff changeset
274 ... 5, #0
2dd4cf273804 revlog: add a doctest for _segmentspan
Boris Feld <boris.feld@octobus.net>
parents: 38638
diff changeset
275 ... 10, #1
2dd4cf273804 revlog: add a doctest for _segmentspan
Boris Feld <boris.feld@octobus.net>
parents: 38638
diff changeset
276 ... 12, #2
2dd4cf273804 revlog: add a doctest for _segmentspan
Boris Feld <boris.feld@octobus.net>
parents: 38638
diff changeset
277 ... 12, #3 (empty)
2dd4cf273804 revlog: add a doctest for _segmentspan
Boris Feld <boris.feld@octobus.net>
parents: 38638
diff changeset
278 ... 17, #4
2dd4cf273804 revlog: add a doctest for _segmentspan
Boris Feld <boris.feld@octobus.net>
parents: 38638
diff changeset
279 ... ])
2dd4cf273804 revlog: add a doctest for _segmentspan
Boris Feld <boris.feld@octobus.net>
parents: 38638
diff changeset
280
2dd4cf273804 revlog: add a doctest for _segmentspan
Boris Feld <boris.feld@octobus.net>
parents: 38638
diff changeset
281 >>> _segmentspan(revlog, [0, 1, 2, 3, 4])
2dd4cf273804 revlog: add a doctest for _segmentspan
Boris Feld <boris.feld@octobus.net>
parents: 38638
diff changeset
282 17
2dd4cf273804 revlog: add a doctest for _segmentspan
Boris Feld <boris.feld@octobus.net>
parents: 38638
diff changeset
283 >>> _segmentspan(revlog, [0, 4])
2dd4cf273804 revlog: add a doctest for _segmentspan
Boris Feld <boris.feld@octobus.net>
parents: 38638
diff changeset
284 17
2dd4cf273804 revlog: add a doctest for _segmentspan
Boris Feld <boris.feld@octobus.net>
parents: 38638
diff changeset
285 >>> _segmentspan(revlog, [3, 4])
2dd4cf273804 revlog: add a doctest for _segmentspan
Boris Feld <boris.feld@octobus.net>
parents: 38638
diff changeset
286 5
2dd4cf273804 revlog: add a doctest for _segmentspan
Boris Feld <boris.feld@octobus.net>
parents: 38638
diff changeset
287 >>> _segmentspan(revlog, [1, 2, 3,])
2dd4cf273804 revlog: add a doctest for _segmentspan
Boris Feld <boris.feld@octobus.net>
parents: 38638
diff changeset
288 7
2dd4cf273804 revlog: add a doctest for _segmentspan
Boris Feld <boris.feld@octobus.net>
parents: 38638
diff changeset
289 >>> _segmentspan(revlog, [1, 3])
2dd4cf273804 revlog: add a doctest for _segmentspan
Boris Feld <boris.feld@octobus.net>
parents: 38638
diff changeset
290 7
38636
25534e0cf0d9 revlog: _segmentspan computes the byte span of a segment
Paul Morelle <paul.morelle@octobus.net>
parents: 38635
diff changeset
291 """
25534e0cf0d9 revlog: _segmentspan computes the byte span of a segment
Paul Morelle <paul.morelle@octobus.net>
parents: 38635
diff changeset
292 if not revs:
25534e0cf0d9 revlog: _segmentspan computes the byte span of a segment
Paul Morelle <paul.morelle@octobus.net>
parents: 38635
diff changeset
293 return 0
25534e0cf0d9 revlog: _segmentspan computes the byte span of a segment
Paul Morelle <paul.morelle@octobus.net>
parents: 38635
diff changeset
294 return revlog.end(revs[-1]) - revlog.start(revs[0])
25534e0cf0d9 revlog: _segmentspan computes the byte span of a segment
Paul Morelle <paul.morelle@octobus.net>
parents: 38635
diff changeset
295
38643
967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents: 38642
diff changeset
296 def _slicechunk(revlog, revs, targetsize=None):
34824
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
297 """slice revs to reduce the amount of unrelated data to be read from disk.
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
298
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
299 ``revs`` is sliced into groups that should be read in one time.
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
300 Assume that revs are sorted.
38640
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
301
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
302 The initial chunk is sliced until the overall density (payload/chunks-span
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
303 ratio) is above `revlog._srdensitythreshold`. No gap smaller than
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
304 `revlog._srmingapsize` is skipped.
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
305
38643
967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents: 38642
diff changeset
306 If `targetsize` is set, no chunk larger than `targetsize` will be yield.
967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents: 38642
diff changeset
307 For consistency with other slicing choice, this limit won't go lower than
967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents: 38642
diff changeset
308 `revlog._srmingapsize`.
967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents: 38642
diff changeset
309
967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents: 38642
diff changeset
310 If individual revisions chunk are larger than this limit, they will still
967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents: 38642
diff changeset
311 be raised individually.
967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents: 38642
diff changeset
312
38640
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
313 >>> revlog = _testrevlog([
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
314 ... 5, #00 (5)
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
315 ... 10, #01 (5)
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
316 ... 12, #02 (2)
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
317 ... 12, #03 (empty)
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
318 ... 27, #04 (15)
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
319 ... 31, #05 (4)
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
320 ... 31, #06 (empty)
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
321 ... 42, #07 (11)
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
322 ... 47, #08 (5)
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
323 ... 47, #09 (empty)
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
324 ... 48, #10 (1)
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
325 ... 51, #11 (3)
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
326 ... 74, #12 (23)
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
327 ... 85, #13 (11)
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
328 ... 86, #14 (1)
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
329 ... 91, #15 (5)
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
330 ... ])
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
331
38655
cd1c484e31e8 revlog: adjust doctest examples to be portable to Python 3
Augie Fackler <augie@google.com>
parents: 38644
diff changeset
332 >>> list(_slicechunk(revlog, list(range(16))))
38640
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
333 [[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]]
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
334 >>> list(_slicechunk(revlog, [0, 15]))
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
335 [[0], [15]]
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
336 >>> list(_slicechunk(revlog, [0, 11, 15]))
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
337 [[0], [11], [15]]
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
338 >>> list(_slicechunk(revlog, [0, 11, 13, 15]))
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
339 [[0], [11, 13, 15]]
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
340 >>> list(_slicechunk(revlog, [1, 2, 3, 5, 8, 10, 11, 14]))
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
341 [[1, 2], [5, 8, 10, 11], [14]]
38643
967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents: 38642
diff changeset
342
967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents: 38642
diff changeset
343 Slicing with a maximum chunk size
967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents: 38642
diff changeset
344 >>> list(_slicechunk(revlog, [0, 11, 13, 15], 15))
967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents: 38642
diff changeset
345 [[0], [11], [13], [15]]
967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents: 38642
diff changeset
346 >>> list(_slicechunk(revlog, [0, 11, 13, 15], 20))
967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents: 38642
diff changeset
347 [[0], [11], [13, 15]]
34824
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
348 """
38643
967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents: 38642
diff changeset
349 if targetsize is not None:
967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents: 38642
diff changeset
350 targetsize = max(targetsize, revlog._srmingapsize)
38641
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
351 for chunk in _slicechunktodensity(revlog, revs,
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
352 revlog._srdensitythreshold,
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
353 revlog._srmingapsize):
38643
967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents: 38642
diff changeset
354 for subchunk in _slicechunktosize(revlog, chunk, targetsize):
967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents: 38642
diff changeset
355 yield subchunk
38641
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
356
38642
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
357 def _slicechunktosize(revlog, revs, targetsize):
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
358 """slice revs to match the target size
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
359
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
360 This is intended to be used on chunk that density slicing selected by that
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
361 are still too large compared to the read garantee of revlog. This might
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
362 happens when "minimal gap size" interrupted the slicing or when chain are
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
363 built in a way that create large blocks next to each other.
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
364
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
365 >>> revlog = _testrevlog([
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
366 ... 3, #0 (3)
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
367 ... 5, #1 (2)
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
368 ... 6, #2 (1)
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
369 ... 8, #3 (2)
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
370 ... 8, #4 (empty)
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
371 ... 11, #5 (3)
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
372 ... 12, #6 (1)
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
373 ... 13, #7 (1)
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
374 ... 14, #8 (1)
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
375 ... ])
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
376
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
377 Cases where chunk is already small enough
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
378 >>> list(_slicechunktosize(revlog, [0], 3))
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
379 [[0]]
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
380 >>> list(_slicechunktosize(revlog, [6, 7], 3))
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
381 [[6, 7]]
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
382 >>> list(_slicechunktosize(revlog, [0], None))
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
383 [[0]]
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
384 >>> list(_slicechunktosize(revlog, [6, 7], None))
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
385 [[6, 7]]
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
386
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
387 cases where we need actual slicing
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
388 >>> list(_slicechunktosize(revlog, [0, 1], 3))
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
389 [[0], [1]]
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
390 >>> list(_slicechunktosize(revlog, [1, 3], 3))
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
391 [[1], [3]]
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
392 >>> list(_slicechunktosize(revlog, [1, 2, 3], 3))
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
393 [[1, 2], [3]]
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
394 >>> list(_slicechunktosize(revlog, [3, 5], 3))
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
395 [[3], [5]]
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
396 >>> list(_slicechunktosize(revlog, [3, 4, 5], 3))
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
397 [[3], [5]]
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
398 >>> list(_slicechunktosize(revlog, [5, 6, 7, 8], 3))
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
399 [[5], [6, 7, 8]]
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
400 >>> list(_slicechunktosize(revlog, [0, 1, 2, 3, 4, 5, 6, 7, 8], 3))
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
401 [[0], [1, 2], [3], [5], [6, 7, 8]]
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
402
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
403 Case with too large individual chunk (must return valid chunk)
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
404 >>> list(_slicechunktosize(revlog, [0, 1], 2))
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
405 [[0], [1]]
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
406 >>> list(_slicechunktosize(revlog, [1, 3], 1))
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
407 [[1], [3]]
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
408 >>> list(_slicechunktosize(revlog, [3, 4, 5], 2))
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
409 [[3], [5]]
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
410 """
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
411 assert targetsize is None or 0 <= targetsize
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
412 if targetsize is None or _segmentspan(revlog, revs) <= targetsize:
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
413 yield revs
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
414 return
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
415
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
416 startrevidx = 0
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
417 startdata = revlog.start(revs[0])
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
418 endrevidx = 0
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
419 iterrevs = enumerate(revs)
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
420 next(iterrevs) # skip first rev.
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
421 for idx, r in iterrevs:
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
422 span = revlog.end(r) - startdata
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
423 if span <= targetsize:
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
424 endrevidx = idx
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
425 else:
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
426 chunk = _trimchunk(revlog, revs, startrevidx, endrevidx + 1)
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
427 if chunk:
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
428 yield chunk
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
429 startrevidx = idx
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
430 startdata = revlog.start(r)
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
431 endrevidx = idx
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
432 yield _trimchunk(revlog, revs, startrevidx)
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
433
38641
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
434 def _slicechunktodensity(revlog, revs, targetdensity=0.5, mingapsize=0):
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
435 """slice revs to reduce the amount of unrelated data to be read from disk.
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
436
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
437 ``revs`` is sliced into groups that should be read in one time.
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
438 Assume that revs are sorted.
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
439
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
440 The initial chunk is sliced until the overall density (payload/chunks-span
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
441 ratio) is above `targetdensity`. No gap smaller than `mingapsize` is
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
442 skipped.
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
443
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
444 >>> revlog = _testrevlog([
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
445 ... 5, #00 (5)
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
446 ... 10, #01 (5)
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
447 ... 12, #02 (2)
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
448 ... 12, #03 (empty)
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
449 ... 27, #04 (15)
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
450 ... 31, #05 (4)
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
451 ... 31, #06 (empty)
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
452 ... 42, #07 (11)
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
453 ... 47, #08 (5)
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
454 ... 47, #09 (empty)
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
455 ... 48, #10 (1)
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
456 ... 51, #11 (3)
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
457 ... 74, #12 (23)
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
458 ... 85, #13 (11)
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
459 ... 86, #14 (1)
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
460 ... 91, #15 (5)
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
461 ... ])
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
462
38655
cd1c484e31e8 revlog: adjust doctest examples to be portable to Python 3
Augie Fackler <augie@google.com>
parents: 38644
diff changeset
463 >>> list(_slicechunktodensity(revlog, list(range(16))))
38641
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
464 [[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]]
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
465 >>> list(_slicechunktodensity(revlog, [0, 15]))
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
466 [[0], [15]]
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
467 >>> list(_slicechunktodensity(revlog, [0, 11, 15]))
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
468 [[0], [11], [15]]
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
469 >>> list(_slicechunktodensity(revlog, [0, 11, 13, 15]))
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
470 [[0], [11, 13, 15]]
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
471 >>> list(_slicechunktodensity(revlog, [1, 2, 3, 5, 8, 10, 11, 14]))
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
472 [[1, 2], [5, 8, 10, 11], [14]]
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
473 >>> list(_slicechunktodensity(revlog, [1, 2, 3, 5, 8, 10, 11, 14],
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
474 ... mingapsize=20))
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
475 [[1, 2, 3, 5, 8, 10, 11], [14]]
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
476 >>> list(_slicechunktodensity(revlog, [1, 2, 3, 5, 8, 10, 11, 14],
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
477 ... targetdensity=0.95))
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
478 [[1, 2], [5], [8, 10, 11], [14]]
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
479 >>> list(_slicechunktodensity(revlog, [1, 2, 3, 5, 8, 10, 11, 14],
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
480 ... targetdensity=0.95, mingapsize=12))
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
481 [[1, 2], [5, 8, 10, 11], [14]]
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
482 """
34824
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
483 start = revlog.start
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
484 length = revlog.length
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
485
34880
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
486 if len(revs) <= 1:
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
487 yield revs
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
488 return
34824
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
489
38636
25534e0cf0d9 revlog: _segmentspan computes the byte span of a segment
Paul Morelle <paul.morelle@octobus.net>
parents: 38635
diff changeset
490 readdata = deltachainspan = _segmentspan(revlog, revs)
34880
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
491
38641
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
492 if deltachainspan < mingapsize:
38635
d083ae26c325 revlog: early return in _slicechunk when span is already small enough
Boris Feld <boris.feld@octobus.net>
parents: 38634
diff changeset
493 yield revs
d083ae26c325 revlog: early return in _slicechunk when span is already small enough
Boris Feld <boris.feld@octobus.net>
parents: 38634
diff changeset
494 return
d083ae26c325 revlog: early return in _slicechunk when span is already small enough
Boris Feld <boris.feld@octobus.net>
parents: 38634
diff changeset
495
34880
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
496 chainpayload = sum(length(r) for r in revs)
34824
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
497
34880
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
498 if deltachainspan:
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
499 density = chainpayload / float(deltachainspan)
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
500 else:
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
501 density = 1.0
34824
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
502
38641
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
503 if density >= targetdensity:
38634
f0ea8b847831 revlog: early return in _slicechunk when density is already good
Paul Morelle <paul.morelle@octobus.net>
parents: 38632
diff changeset
504 yield revs
f0ea8b847831 revlog: early return in _slicechunk when density is already good
Paul Morelle <paul.morelle@octobus.net>
parents: 38632
diff changeset
505 return
f0ea8b847831 revlog: early return in _slicechunk when density is already good
Paul Morelle <paul.morelle@octobus.net>
parents: 38632
diff changeset
506
34880
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
507 # Store the gaps in a heap to have them sorted by decreasing size
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
508 gapsheap = []
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
509 heapq.heapify(gapsheap)
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
510 prevend = None
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
511 for i, rev in enumerate(revs):
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
512 revstart = start(rev)
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
513 revlen = length(rev)
34824
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
514
34898
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents: 34881
diff changeset
515 # Skip empty revisions to form larger holes
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents: 34881
diff changeset
516 if revlen == 0:
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents: 34881
diff changeset
517 continue
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents: 34881
diff changeset
518
34880
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
519 if prevend is not None:
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
520 gapsize = revstart - prevend
34881
8c9b08a0c48c sparse-read: skip gaps too small to be worth splitting
Paul Morelle <paul.morelle@octobus.net>
parents: 34880
diff changeset
521 # only consider holes that are large enough
38641
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
522 if gapsize > mingapsize:
34880
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
523 heapq.heappush(gapsheap, (-gapsize, i))
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
524
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
525 prevend = revstart + revlen
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
526
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
527 # Collect the indices of the largest holes until the density is acceptable
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
528 indicesheap = []
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
529 heapq.heapify(indicesheap)
38641
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
530 while gapsheap and density < targetdensity:
34880
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
531 oppgapsize, gapidx = heapq.heappop(gapsheap)
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
532
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
533 heapq.heappush(indicesheap, gapidx)
34824
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
534
34880
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
535 # the gap sizes are stored as negatives to be sorted decreasingly
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
536 # by the heap
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
537 readdata -= (-oppgapsize)
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
538 if readdata > 0:
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
539 density = chainpayload / float(readdata)
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
540 else:
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
541 density = 1.0
34824
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
542
34880
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
543 # Cut the revs at collected indices
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
544 previdx = 0
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
545 while indicesheap:
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
546 idx = heapq.heappop(indicesheap)
34898
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents: 34881
diff changeset
547
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents: 34881
diff changeset
548 chunk = _trimchunk(revlog, revs, previdx, idx)
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents: 34881
diff changeset
549 if chunk:
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents: 34881
diff changeset
550 yield chunk
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents: 34881
diff changeset
551
34880
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
552 previdx = idx
34898
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents: 34881
diff changeset
553
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents: 34881
diff changeset
554 chunk = _trimchunk(revlog, revs, previdx)
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents: 34881
diff changeset
555 if chunk:
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents: 34881
diff changeset
556 yield chunk
34824
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
557
35638
edc9330acac1 revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents: 35637
diff changeset
558 @attr.s(slots=True, frozen=True)
edc9330acac1 revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents: 35637
diff changeset
559 class _deltainfo(object):
edc9330acac1 revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents: 35637
diff changeset
560 distance = attr.ib()
edc9330acac1 revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents: 35637
diff changeset
561 deltalen = attr.ib()
edc9330acac1 revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents: 35637
diff changeset
562 data = attr.ib()
edc9330acac1 revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents: 35637
diff changeset
563 base = attr.ib()
edc9330acac1 revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents: 35637
diff changeset
564 chainbase = attr.ib()
edc9330acac1 revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents: 35637
diff changeset
565 chainlen = attr.ib()
edc9330acac1 revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents: 35637
diff changeset
566 compresseddeltalen = attr.ib()
edc9330acac1 revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents: 35637
diff changeset
567
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
568 class _deltacomputer(object):
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
569 def __init__(self, revlog):
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
570 self.revlog = revlog
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
571
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
572 def _getcandidaterevs(self, p1, p2, cachedelta):
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
573 """
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
574 Provides revisions that present an interest to be diffed against,
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
575 grouped by level of easiness.
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
576 """
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
577 revlog = self.revlog
38114
f79ba1d1b4b1 revlog: in _getcandidaterevs, shorten revlog._generaldelta to gdelta
Paul Morelle <paul.morelle@octobus.net>
parents: 38102
diff changeset
578 gdelta = revlog._generaldelta
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
579 curr = len(revlog)
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
580 prev = curr - 1
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
581 p1r, p2r = revlog.rev(p1), revlog.rev(p2)
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
582
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
583 # should we try to build a delta?
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
584 if prev != nullrev and revlog.storedeltachains:
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
585 tested = set()
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
586 # This condition is true most of the time when processing
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
587 # changegroup data into a generaldelta repo. The only time it
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
588 # isn't true is if this is the first revision in a delta chain
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
589 # or if ``format.generaldelta=true`` disabled ``lazydeltabase``.
38114
f79ba1d1b4b1 revlog: in _getcandidaterevs, shorten revlog._generaldelta to gdelta
Paul Morelle <paul.morelle@octobus.net>
parents: 38102
diff changeset
590 if cachedelta and gdelta and revlog._lazydeltabase:
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
591 # Assume what we received from the server is a good choice
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
592 # build delta will reuse the cache
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
593 yield (cachedelta[0],)
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
594 tested.add(cachedelta[0])
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
595
38114
f79ba1d1b4b1 revlog: in _getcandidaterevs, shorten revlog._generaldelta to gdelta
Paul Morelle <paul.morelle@octobus.net>
parents: 38102
diff changeset
596 if gdelta:
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
597 # exclude already lazy tested base if any
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
598 parents = [p for p in (p1r, p2r)
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
599 if p != nullrev and p not in tested]
38102
9bf0bd4d7a2e revlog: suggest other parent when a parent was refused for a delta (issue5481)
Paul Morelle <paul.morelle@octobus.net>
parents: 38090
diff changeset
600
9bf0bd4d7a2e revlog: suggest other parent when a parent was refused for a delta (issue5481)
Paul Morelle <paul.morelle@octobus.net>
parents: 38090
diff changeset
601 if not revlog._aggressivemergedeltas and len(parents) == 2:
9bf0bd4d7a2e revlog: suggest other parent when a parent was refused for a delta (issue5481)
Paul Morelle <paul.morelle@octobus.net>
parents: 38090
diff changeset
602 parents.sort()
9bf0bd4d7a2e revlog: suggest other parent when a parent was refused for a delta (issue5481)
Paul Morelle <paul.morelle@octobus.net>
parents: 38090
diff changeset
603 # To minimize the chance of having to build a fulltext,
9bf0bd4d7a2e revlog: suggest other parent when a parent was refused for a delta (issue5481)
Paul Morelle <paul.morelle@octobus.net>
parents: 38090
diff changeset
604 # pick first whichever parent is closest to us (max rev)
9bf0bd4d7a2e revlog: suggest other parent when a parent was refused for a delta (issue5481)
Paul Morelle <paul.morelle@octobus.net>
parents: 38090
diff changeset
605 yield (parents[1],)
9bf0bd4d7a2e revlog: suggest other parent when a parent was refused for a delta (issue5481)
Paul Morelle <paul.morelle@octobus.net>
parents: 38090
diff changeset
606 # then the other one (min rev) if the first did not fit
9bf0bd4d7a2e revlog: suggest other parent when a parent was refused for a delta (issue5481)
Paul Morelle <paul.morelle@octobus.net>
parents: 38090
diff changeset
607 yield (parents[0],)
9bf0bd4d7a2e revlog: suggest other parent when a parent was refused for a delta (issue5481)
Paul Morelle <paul.morelle@octobus.net>
parents: 38090
diff changeset
608 tested.update(parents)
9bf0bd4d7a2e revlog: suggest other parent when a parent was refused for a delta (issue5481)
Paul Morelle <paul.morelle@octobus.net>
parents: 38090
diff changeset
609 elif len(parents) > 0:
9bf0bd4d7a2e revlog: suggest other parent when a parent was refused for a delta (issue5481)
Paul Morelle <paul.morelle@octobus.net>
parents: 38090
diff changeset
610 # Test all parents (1 or 2), and keep the best candidate
9bf0bd4d7a2e revlog: suggest other parent when a parent was refused for a delta (issue5481)
Paul Morelle <paul.morelle@octobus.net>
parents: 38090
diff changeset
611 yield parents
9bf0bd4d7a2e revlog: suggest other parent when a parent was refused for a delta (issue5481)
Paul Morelle <paul.morelle@octobus.net>
parents: 38090
diff changeset
612 tested.update(parents)
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
613
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
614 if prev not in tested:
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
615 # other approach failed try against prev to hopefully save us a
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
616 # fulltext.
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
617 yield (prev,)
38168
faa015417348 revlog: make getcandidaterevs more consistent about updating tested revs set
Paul Morelle <paul.morelle@octobus.net>
parents: 38117
diff changeset
618 tested.add(prev)
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
619
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
620 def buildtext(self, revinfo, fh):
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
621 """Builds a fulltext version of a revision
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
622
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
623 revinfo: _revisioninfo instance that contains all needed info
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
624 fh: file handle to either the .i or the .d revlog file,
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
625 depending on whether it is inlined or not
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
626 """
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
627 btext = revinfo.btext
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
628 if btext[0] is not None:
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
629 return btext[0]
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
630
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
631 revlog = self.revlog
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
632 cachedelta = revinfo.cachedelta
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
633 flags = revinfo.flags
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
634 node = revinfo.node
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
635
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
636 baserev = cachedelta[0]
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
637 delta = cachedelta[1]
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
638 # special case deltas which replace entire base; no need to decode
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
639 # base revision. this neatly avoids censored bases, which throw when
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
640 # they're decoded.
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
641 hlen = struct.calcsize(">lll")
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
642 if delta[:hlen] == mdiff.replacediffheader(revlog.rawsize(baserev),
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
643 len(delta) - hlen):
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
644 btext[0] = delta[hlen:]
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
645 else:
36748
369aadf7a326 revlog: resolve lfs rawtext to vanilla rawtext before applying delta
Jun Wu <quark@fb.com>
parents: 36744
diff changeset
646 # deltabase is rawtext before changed by flag processors, which is
369aadf7a326 revlog: resolve lfs rawtext to vanilla rawtext before applying delta
Jun Wu <quark@fb.com>
parents: 36744
diff changeset
647 # equivalent to non-raw text
369aadf7a326 revlog: resolve lfs rawtext to vanilla rawtext before applying delta
Jun Wu <quark@fb.com>
parents: 36744
diff changeset
648 basetext = revlog.revision(baserev, _df=fh, raw=False)
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
649 btext[0] = mdiff.patch(basetext, delta)
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
650
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
651 try:
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
652 res = revlog._processflags(btext[0], flags, 'read', raw=True)
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
653 btext[0], validatehash = res
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
654 if validatehash:
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
655 revlog.checkhash(btext[0], node, p1=revinfo.p1, p2=revinfo.p2)
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
656 if flags & REVIDX_ISCENSORED:
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
657 raise RevlogError(_('node %s is not censored') % node)
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
658 except CensoredNodeError:
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
659 # must pass the censored index flag to add censored revisions
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
660 if not flags & REVIDX_ISCENSORED:
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
661 raise
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
662 return btext[0]
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
663
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
664 def _builddeltadiff(self, base, revinfo, fh):
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
665 revlog = self.revlog
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
666 t = self.buildtext(revinfo, fh)
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
667 if revlog.iscensored(base):
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
668 # deltas based on a censored revision must replace the
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
669 # full content in one patch, so delta works everywhere
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
670 header = mdiff.replacediffheader(revlog.rawsize(base), len(t))
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
671 delta = header + t
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
672 else:
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
673 ptext = revlog.revision(base, _df=fh, raw=True)
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
674 delta = mdiff.textdiff(ptext, t)
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
675
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
676 return delta
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
677
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
678 def _builddeltainfo(self, revinfo, base, fh):
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
679 # can we use the cached delta?
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
680 if revinfo.cachedelta and revinfo.cachedelta[0] == base:
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
681 delta = revinfo.cachedelta[1]
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
682 else:
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
683 delta = self._builddeltadiff(base, revinfo, fh)
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
684 revlog = self.revlog
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
685 header, data = revlog.compress(delta)
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
686 deltalen = len(header) + len(data)
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
687 chainbase = revlog.chainbase(base)
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
688 offset = revlog.end(len(revlog) - 1)
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
689 dist = deltalen + offset - revlog.start(chainbase)
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
690 if revlog._generaldelta:
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
691 deltabase = base
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
692 else:
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
693 deltabase = chainbase
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
694 chainlen, compresseddeltalen = revlog._chaininfo(base)
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
695 chainlen += 1
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
696 compresseddeltalen += deltalen
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
697 return _deltainfo(dist, deltalen, (header, data), deltabase,
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
698 chainbase, chainlen, compresseddeltalen)
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
699
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
700 def finddeltainfo(self, revinfo, fh):
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
701 """Find an acceptable delta against a candidate revision
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
702
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
703 revinfo: information about the revision (instance of _revisioninfo)
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
704 fh: file handle to either the .i or the .d revlog file,
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
705 depending on whether it is inlined or not
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
706
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
707 Returns the first acceptable candidate revision, as ordered by
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
708 _getcandidaterevs
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
709 """
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
710 cachedelta = revinfo.cachedelta
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
711 p1 = revinfo.p1
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
712 p2 = revinfo.p2
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
713 revlog = self.revlog
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
714
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
715 deltainfo = None
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
716 for candidaterevs in self._getcandidaterevs(p1, p2, cachedelta):
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
717 nominateddeltas = []
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
718 for candidaterev in candidaterevs:
36744
33275ab5e837 revlog: do not use delta for lfs revisions
Jun Wu <quark@fb.com>
parents: 36743
diff changeset
719 # no delta for rawtext-changing revs (see "candelta" for why)
33275ab5e837 revlog: do not use delta for lfs revisions
Jun Wu <quark@fb.com>
parents: 36743
diff changeset
720 if revlog.flags(candidaterev) & REVIDX_RAWTEXT_CHANGING_FLAGS:
33275ab5e837 revlog: do not use delta for lfs revisions
Jun Wu <quark@fb.com>
parents: 36743
diff changeset
721 continue
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
722 candidatedelta = self._builddeltainfo(revinfo, candidaterev, fh)
38117
69ec6f98cfa6 revlog: isgooddeltainfo takes the whole revinfo object
Paul Morelle <paul.morelle@octobus.net>
parents: 38116
diff changeset
723 if revlog._isgooddeltainfo(candidatedelta, revinfo):
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
724 nominateddeltas.append(candidatedelta)
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
725 if nominateddeltas:
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
726 deltainfo = min(nominateddeltas, key=lambda x: x.deltalen)
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
727 break
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
728
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
729 return deltainfo
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
730
35641
c41bb85c09ff revlog: group revision info into a dedicated structure
Paul Morelle <paul.morelle@octobus.net>
parents: 35640
diff changeset
731 @attr.s(slots=True, frozen=True)
c41bb85c09ff revlog: group revision info into a dedicated structure
Paul Morelle <paul.morelle@octobus.net>
parents: 35640
diff changeset
732 class _revisioninfo(object):
c41bb85c09ff revlog: group revision info into a dedicated structure
Paul Morelle <paul.morelle@octobus.net>
parents: 35640
diff changeset
733 """Information about a revision that allows building its fulltext
c41bb85c09ff revlog: group revision info into a dedicated structure
Paul Morelle <paul.morelle@octobus.net>
parents: 35640
diff changeset
734 node: expected hash of the revision
c41bb85c09ff revlog: group revision info into a dedicated structure
Paul Morelle <paul.morelle@octobus.net>
parents: 35640
diff changeset
735 p1, p2: parent revs of the revision
c41bb85c09ff revlog: group revision info into a dedicated structure
Paul Morelle <paul.morelle@octobus.net>
parents: 35640
diff changeset
736 btext: built text cache consisting of a one-element list
c41bb85c09ff revlog: group revision info into a dedicated structure
Paul Morelle <paul.morelle@octobus.net>
parents: 35640
diff changeset
737 cachedelta: (baserev, uncompressed_delta) or None
c41bb85c09ff revlog: group revision info into a dedicated structure
Paul Morelle <paul.morelle@octobus.net>
parents: 35640
diff changeset
738 flags: flags associated to the revision storage
c41bb85c09ff revlog: group revision info into a dedicated structure
Paul Morelle <paul.morelle@octobus.net>
parents: 35640
diff changeset
739
c41bb85c09ff revlog: group revision info into a dedicated structure
Paul Morelle <paul.morelle@octobus.net>
parents: 35640
diff changeset
740 One of btext[0] or cachedelta must be set.
c41bb85c09ff revlog: group revision info into a dedicated structure
Paul Morelle <paul.morelle@octobus.net>
parents: 35640
diff changeset
741 """
c41bb85c09ff revlog: group revision info into a dedicated structure
Paul Morelle <paul.morelle@octobus.net>
parents: 35640
diff changeset
742 node = attr.ib()
c41bb85c09ff revlog: group revision info into a dedicated structure
Paul Morelle <paul.morelle@octobus.net>
parents: 35640
diff changeset
743 p1 = attr.ib()
c41bb85c09ff revlog: group revision info into a dedicated structure
Paul Morelle <paul.morelle@octobus.net>
parents: 35640
diff changeset
744 p2 = attr.ib()
c41bb85c09ff revlog: group revision info into a dedicated structure
Paul Morelle <paul.morelle@octobus.net>
parents: 35640
diff changeset
745 btext = attr.ib()
35737
d99b07bc69fb revlog: refactor out _finddeltainfo from _addrevision
Paul Morelle <paul.morelle@octobus.net>
parents: 35641
diff changeset
746 textlen = attr.ib()
35641
c41bb85c09ff revlog: group revision info into a dedicated structure
Paul Morelle <paul.morelle@octobus.net>
parents: 35640
diff changeset
747 cachedelta = attr.ib()
c41bb85c09ff revlog: group revision info into a dedicated structure
Paul Morelle <paul.morelle@octobus.net>
parents: 35640
diff changeset
748 flags = attr.ib()
c41bb85c09ff revlog: group revision info into a dedicated structure
Paul Morelle <paul.morelle@octobus.net>
parents: 35640
diff changeset
749
18585
b280f3bfc8a0 revlog: document v0 format
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 18090
diff changeset
750 # index v0:
b280f3bfc8a0 revlog: document v0 format
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 18090
diff changeset
751 # 4 bytes: offset
b280f3bfc8a0 revlog: document v0 format
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 18090
diff changeset
752 # 4 bytes: compressed length
b280f3bfc8a0 revlog: document v0 format
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 18090
diff changeset
753 # 4 bytes: base rev
b280f3bfc8a0 revlog: document v0 format
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 18090
diff changeset
754 # 4 bytes: link rev
25891
c73fada78589 revlog: correct comment about size of v0 index format
Yuya Nishihara <yuya@tcha.org>
parents: 25822
diff changeset
755 # 20 bytes: parent 1 nodeid
c73fada78589 revlog: correct comment about size of v0 index format
Yuya Nishihara <yuya@tcha.org>
parents: 25822
diff changeset
756 # 20 bytes: parent 2 nodeid
c73fada78589 revlog: correct comment about size of v0 index format
Yuya Nishihara <yuya@tcha.org>
parents: 25822
diff changeset
757 # 20 bytes: nodeid
33392
ac6446611ad2 revlog: use struct.Struct instances for slight performance wins
Alex Gaynor <agaynor@mozilla.com>
parents: 33391
diff changeset
758 indexformatv0 = struct.Struct(">4l20s20s20s")
ac6446611ad2 revlog: use struct.Struct instances for slight performance wins
Alex Gaynor <agaynor@mozilla.com>
parents: 33391
diff changeset
759 indexformatv0_pack = indexformatv0.pack
ac6446611ad2 revlog: use struct.Struct instances for slight performance wins
Alex Gaynor <agaynor@mozilla.com>
parents: 33391
diff changeset
760 indexformatv0_unpack = indexformatv0.unpack
4918
e017d3a82e1d revlog: raise offset/type helpers to global scope
Matt Mackall <mpm@selenic.com>
parents: 4746
diff changeset
761
4972
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
762 class revlogoldio(object):
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
763 def __init__(self):
33392
ac6446611ad2 revlog: use struct.Struct instances for slight performance wins
Alex Gaynor <agaynor@mozilla.com>
parents: 33391
diff changeset
764 self.size = indexformatv0.size
4972
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
765
13264
8439526fb407 revlog/parseindex: no need to pass the file around
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 13259
diff changeset
766 def parseindex(self, data, inline):
4977
6cb30bc4ca32 revlog: parse revlogv0 indexes into v1 internally
Matt Mackall <mpm@selenic.com>
parents: 4976
diff changeset
767 s = self.size
4972
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
768 index = []
27637
b502138f5faa cleanup: remove superfluous space after space after equals (python)
timeless <timeless@mozdev.org>
parents: 27475
diff changeset
769 nodemap = {nullid: nullrev}
4973
a386a6e4fe46 revlog: simplify the v0 parser
Matt Mackall <mpm@selenic.com>
parents: 4972
diff changeset
770 n = off = 0
a386a6e4fe46 revlog: simplify the v0 parser
Matt Mackall <mpm@selenic.com>
parents: 4972
diff changeset
771 l = len(data)
a386a6e4fe46 revlog: simplify the v0 parser
Matt Mackall <mpm@selenic.com>
parents: 4972
diff changeset
772 while off + s <= l:
a386a6e4fe46 revlog: simplify the v0 parser
Matt Mackall <mpm@selenic.com>
parents: 4972
diff changeset
773 cur = data[off:off + s]
a386a6e4fe46 revlog: simplify the v0 parser
Matt Mackall <mpm@selenic.com>
parents: 4972
diff changeset
774 off += s
33392
ac6446611ad2 revlog: use struct.Struct instances for slight performance wins
Alex Gaynor <agaynor@mozilla.com>
parents: 33391
diff changeset
775 e = indexformatv0_unpack(cur)
4977
6cb30bc4ca32 revlog: parse revlogv0 indexes into v1 internally
Matt Mackall <mpm@selenic.com>
parents: 4976
diff changeset
776 # transform to revlogv1 format
6cb30bc4ca32 revlog: parse revlogv0 indexes into v1 internally
Matt Mackall <mpm@selenic.com>
parents: 4976
diff changeset
777 e2 = (offset_type(e[0], 0), e[1], -1, e[2], e[3],
5544
686899a7de5b revlog: make revlogv0 loading more robust against corruption
Matt Mackall <mpm@selenic.com>
parents: 5453
diff changeset
778 nodemap.get(e[4], nullrev), nodemap.get(e[5], nullrev), e[6])
4977
6cb30bc4ca32 revlog: parse revlogv0 indexes into v1 internally
Matt Mackall <mpm@selenic.com>
parents: 4976
diff changeset
779 index.append(e2)
6cb30bc4ca32 revlog: parse revlogv0 indexes into v1 internally
Matt Mackall <mpm@selenic.com>
parents: 4976
diff changeset
780 nodemap[e[6]] = n
4973
a386a6e4fe46 revlog: simplify the v0 parser
Matt Mackall <mpm@selenic.com>
parents: 4972
diff changeset
781 n += 1
4972
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
782
13265
04b302ce2781 revlog: always add the magic nullid/nullrev entry in parseindex
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 13264
diff changeset
783 # add the magic null revision at -1
04b302ce2781 revlog: always add the magic nullid/nullrev entry in parseindex
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 13264
diff changeset
784 index.append((0, 0, 0, -1, -1, -1, -1, nullid))
04b302ce2781 revlog: always add the magic nullid/nullrev entry in parseindex
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 13264
diff changeset
785
4983
4dbcfc6e359e revlog: pull chunkcache back into revlog
Matt Mackall <mpm@selenic.com>
parents: 4982
diff changeset
786 return index, nodemap, None
4972
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
787
5338
f87685355c9c revlog: fix revlogio.packentry corner case
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5325
diff changeset
788 def packentry(self, entry, node, version, rev):
10395
ea52a2d4f42c revlog: don't silently discard revlog flags on revlogv0
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 10329
diff changeset
789 if gettype(entry[0]):
32393
d47b62368f3a revlog: remove some revlogNG terminology
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32392
diff changeset
790 raise RevlogError(_('index entry flags need revlog version 1'))
4986
58cc017ec7e0 revlog: abstract out index entry packing
Matt Mackall <mpm@selenic.com>
parents: 4985
diff changeset
791 e2 = (getoffset(entry[0]), entry[1], entry[3], entry[4],
58cc017ec7e0 revlog: abstract out index entry packing
Matt Mackall <mpm@selenic.com>
parents: 4985
diff changeset
792 node(entry[5]), node(entry[6]), entry[7])
33392
ac6446611ad2 revlog: use struct.Struct instances for slight performance wins
Alex Gaynor <agaynor@mozilla.com>
parents: 33391
diff changeset
793 return indexformatv0_pack(*e2)
4986
58cc017ec7e0 revlog: abstract out index entry packing
Matt Mackall <mpm@selenic.com>
parents: 4985
diff changeset
794
4987
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
795 # index ng:
11323
d65b74106113 revlog: fix inconsistent comment formatting
Martin Geisler <mg@aragost.com>
parents: 11155
diff changeset
796 # 6 bytes: offset
d65b74106113 revlog: fix inconsistent comment formatting
Martin Geisler <mg@aragost.com>
parents: 11155
diff changeset
797 # 2 bytes: flags
d65b74106113 revlog: fix inconsistent comment formatting
Martin Geisler <mg@aragost.com>
parents: 11155
diff changeset
798 # 4 bytes: compressed length
d65b74106113 revlog: fix inconsistent comment formatting
Martin Geisler <mg@aragost.com>
parents: 11155
diff changeset
799 # 4 bytes: uncompressed length
d65b74106113 revlog: fix inconsistent comment formatting
Martin Geisler <mg@aragost.com>
parents: 11155
diff changeset
800 # 4 bytes: base rev
d65b74106113 revlog: fix inconsistent comment formatting
Martin Geisler <mg@aragost.com>
parents: 11155
diff changeset
801 # 4 bytes: link rev
d65b74106113 revlog: fix inconsistent comment formatting
Martin Geisler <mg@aragost.com>
parents: 11155
diff changeset
802 # 4 bytes: parent 1 rev
d65b74106113 revlog: fix inconsistent comment formatting
Martin Geisler <mg@aragost.com>
parents: 11155
diff changeset
803 # 4 bytes: parent 2 rev
4987
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
804 # 32 bytes: nodeid
33392
ac6446611ad2 revlog: use struct.Struct instances for slight performance wins
Alex Gaynor <agaynor@mozilla.com>
parents: 33391
diff changeset
805 indexformatng = struct.Struct(">Qiiiiii20s12x")
ac6446611ad2 revlog: use struct.Struct instances for slight performance wins
Alex Gaynor <agaynor@mozilla.com>
parents: 33391
diff changeset
806 indexformatng_pack = indexformatng.pack
ac6446611ad2 revlog: use struct.Struct instances for slight performance wins
Alex Gaynor <agaynor@mozilla.com>
parents: 33391
diff changeset
807 versionformat = struct.Struct(">I")
ac6446611ad2 revlog: use struct.Struct instances for slight performance wins
Alex Gaynor <agaynor@mozilla.com>
parents: 33391
diff changeset
808 versionformat_pack = versionformat.pack
ac6446611ad2 revlog: use struct.Struct instances for slight performance wins
Alex Gaynor <agaynor@mozilla.com>
parents: 33391
diff changeset
809 versionformat_unpack = versionformat.unpack
4987
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
810
25410
eee88912db0a revlog: raise an exception earlier if an entry is too large (issue4675)
Jordi Gutiérrez Hermoso <jordigh@octave.org>
parents: 25361
diff changeset
811 # corresponds to uncompressed length of indexformatng (2 gigs, 4-byte
eee88912db0a revlog: raise an exception earlier if an entry is too large (issue4675)
Jordi Gutiérrez Hermoso <jordigh@octave.org>
parents: 25361
diff changeset
812 # signed integer)
eee88912db0a revlog: raise an exception earlier if an entry is too large (issue4675)
Jordi Gutiérrez Hermoso <jordigh@octave.org>
parents: 25361
diff changeset
813 _maxentrysize = 0x7fffffff
eee88912db0a revlog: raise an exception earlier if an entry is too large (issue4675)
Jordi Gutiérrez Hermoso <jordigh@octave.org>
parents: 25361
diff changeset
814
4972
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
815 class revlogio(object):
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
816 def __init__(self):
33392
ac6446611ad2 revlog: use struct.Struct instances for slight performance wins
Alex Gaynor <agaynor@mozilla.com>
parents: 33391
diff changeset
817 self.size = indexformatng.size
4972
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
818
13264
8439526fb407 revlog/parseindex: no need to pass the file around
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 13259
diff changeset
819 def parseindex(self, data, inline):
7109
528b7fc1216c use the new parseindex implementation C in parsers
Bernhard Leiner <bleiner@gmail.com>
parents: 7089
diff changeset
820 # call the C implementation to parse the index data
13254
5ef5eb1f3515 revlog: only build the nodemap on demand
Matt Mackall <mpm@selenic.com>
parents: 13253
diff changeset
821 index, cache = parsers.parse_index2(data, inline)
16414
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16375
diff changeset
822 return index, getattr(index, 'nodemap', None), cache
4972
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
823
5338
f87685355c9c revlog: fix revlogio.packentry corner case
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5325
diff changeset
824 def packentry(self, entry, node, version, rev):
33392
ac6446611ad2 revlog: use struct.Struct instances for slight performance wins
Alex Gaynor <agaynor@mozilla.com>
parents: 33391
diff changeset
825 p = indexformatng_pack(*entry)
5338
f87685355c9c revlog: fix revlogio.packentry corner case
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5325
diff changeset
826 if rev == 0:
33392
ac6446611ad2 revlog: use struct.Struct instances for slight performance wins
Alex Gaynor <agaynor@mozilla.com>
parents: 33391
diff changeset
827 p = versionformat_pack(version) + p[4:]
4986
58cc017ec7e0 revlog: abstract out index entry packing
Matt Mackall <mpm@selenic.com>
parents: 4985
diff changeset
828 return p
58cc017ec7e0 revlog: abstract out index entry packing
Matt Mackall <mpm@selenic.com>
parents: 4985
diff changeset
829
1559
59b3639df0a9 Convert all classes to new-style classes by deriving them from object.
Eric Hopper <hopper@omnifarious.org>
parents: 1551
diff changeset
830 class revlog(object):
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
831 """
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
832 the underlying revision storage object
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
833
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
834 A revlog consists of two parts, an index and the revision data.
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
835
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
836 The index is a file with a fixed record size containing
6912
b92baef99ebf Fixed docstring typos
Martin Geisler <mg@daimi.au.dk>
parents: 6891
diff changeset
837 information on each revision, including its nodeid (hash), the
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
838 nodeids of its parents, the position and offset of its data within
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
839 the data file, and the revision it's based on. Finally, each entry
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
840 contains a linkrev entry that can serve as a pointer to external
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
841 data.
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
842
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
843 The revision data itself is a linear collection of data chunks.
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
844 Each chunk represents a revision and is usually represented as a
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
845 delta against the previous chunk. To bound lookup time, runs of
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
846 deltas are limited to about 2 times the length of the original
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
847 version data. This makes retrieval of a version proportional to
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
848 its size, or O(1) relative to the number of revisions.
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
849
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
850 Both pieces of the revlog are written to in an append-only
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
851 fashion, which means we never need to rewrite a file to insert or
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
852 remove data, and can use some simple techniques to avoid the need
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
853 for locking while reading.
29997
b5e5ddf48bd2 revlog: specify checkambig at writing to avoid file stat ambiguity
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29830
diff changeset
854
b5e5ddf48bd2 revlog: specify checkambig at writing to avoid file stat ambiguity
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29830
diff changeset
855 If checkambig, indexfile is opened with checkambig=True at
b5e5ddf48bd2 revlog: specify checkambig at writing to avoid file stat ambiguity
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29830
diff changeset
856 writing, to avoid file stat ambiguity.
34296
3c9691728237 revlog: add option to mmap revlog index
Mark Thomas <mbthomas@fb.com>
parents: 34291
diff changeset
857
3c9691728237 revlog: add option to mmap revlog index
Mark Thomas <mbthomas@fb.com>
parents: 34291
diff changeset
858 If mmaplargeindex is True, and an mmapindexthreshold is set, the
3c9691728237 revlog: add option to mmap revlog index
Mark Thomas <mbthomas@fb.com>
parents: 34291
diff changeset
859 index will be mmapped rather than read if it is larger than the
3c9691728237 revlog: add option to mmap revlog index
Mark Thomas <mbthomas@fb.com>
parents: 34291
diff changeset
860 configured threshold.
37443
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
861
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
862 If censorable is True, the revlog can have censored revisions.
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
863 """
34296
3c9691728237 revlog: add option to mmap revlog index
Mark Thomas <mbthomas@fb.com>
parents: 34291
diff changeset
864 def __init__(self, opener, indexfile, datafile=None, checkambig=False,
37443
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
865 mmaplargeindex=False, censorable=False):
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
866 """
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
867 create a revlog object
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
868
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
869 opener is a function that abstracts the file opening operation
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
870 and can be used to implement COW semantics or the like.
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
871 """
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
872 self.indexfile = indexfile
32307
3caec778774b changelog: make sure datafile is 00changelog.d (API)
Jun Wu <quark@fb.com>
parents: 32291
diff changeset
873 self.datafile = datafile or (indexfile[:-2] + ".d")
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
874 self.opener = opener
29997
b5e5ddf48bd2 revlog: specify checkambig at writing to avoid file stat ambiguity
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29830
diff changeset
875 # When True, indexfile is opened with checkambig=True at writing, to
b5e5ddf48bd2 revlog: specify checkambig at writing to avoid file stat ambiguity
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29830
diff changeset
876 # avoid file stat ambiguity.
b5e5ddf48bd2 revlog: specify checkambig at writing to avoid file stat ambiguity
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29830
diff changeset
877 self._checkambig = checkambig
37443
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
878 self._censorable = censorable
27070
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
879 # 3-tuple of (node, rev, text) for a raw revision.
4984
b4066fcbd6ba revlog: mark cache private
Matt Mackall <mpm@selenic.com>
parents: 4983
diff changeset
880 self._cache = None
29830
92ac2baaea86 revlog: use an LRU cache for delta chain bases
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29829
diff changeset
881 # Maps rev to chain base rev.
92ac2baaea86 revlog: use an LRU cache for delta chain bases
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29829
diff changeset
882 self._chainbasecache = util.lrucachedict(100)
27070
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
883 # 2-tuple of (offset, data) of raw data from the revlog at an offset.
8316
d593922cf480 revlog: clean up the chunk caching code
Matt Mackall <mpm@selenic.com>
parents: 8315
diff changeset
884 self._chunkcache = (0, '')
27070
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
885 # How much data to read and cache into the raw revlog data cache.
20180
969148b49fc6 revlog: allow tuning of the chunk cache size (via format.chunkcachesize)
Brodie Rao <brodie@sf.io>
parents: 20179
diff changeset
886 self._chunkcachesize = 65536
23255
76effa770ff9 revlog: add config variable for limiting delta-chain length
Mateusz Kwapich <mitrandir@fb.com>
parents: 23254
diff changeset
887 self._maxchainlen = None
38602
44f5acfb9ad2 aggressivemergedeltas: enabled the option by default
Paul Morelle <paul.morelle@octobus.net>
parents: 38514
diff changeset
888 self._aggressivemergedeltas = True
4985
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
889 self.index = []
27070
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
890 # Mapping of partial identifiers to full nodes.
13258
c2661863f16f revlog: introduce a cache for partial lookups
Matt Mackall <mpm@selenic.com>
parents: 13254
diff changeset
891 self._pcache = {}
27070
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
892 # Mapping of revision integer to full node.
13275
68da048b4c88 revlog: incrementally build node cache with linear searches
Matt Mackall <mpm@selenic.com>
parents: 13268
diff changeset
893 self._nodecache = {nullid: nullrev}
68da048b4c88 revlog: incrementally build node cache with linear searches
Matt Mackall <mpm@selenic.com>
parents: 13268
diff changeset
894 self._nodepos = None
30818
4c0a5a256ae8 localrepo: experimental support for non-zlib revlog compression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30817
diff changeset
895 self._compengine = 'zlib'
33207
895ecec31c70 revlog: add an experimental option to mitigated delta issues (issue5480)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 33171
diff changeset
896 self._maxdeltachainspan = -1
34824
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
897 self._withsparseread = False
38632
eb8504715071 sparse-read: target density of 50% instead of 25%
Paul Morelle <paul.morelle@octobus.net>
parents: 38602
diff changeset
898 self._srdensitythreshold = 0.50
34881
8c9b08a0c48c sparse-read: skip gaps too small to be worth splitting
Paul Morelle <paul.morelle@octobus.net>
parents: 34880
diff changeset
899 self._srmingapsize = 262144
4985
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
900
34296
3c9691728237 revlog: add option to mmap revlog index
Mark Thomas <mbthomas@fb.com>
parents: 34291
diff changeset
901 mmapindexthreshold = None
4985
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
902 v = REVLOG_DEFAULT_VERSION
14960
497819817307 revlog: use getattr instead of hasattr
Augie Fackler <durin42@gmail.com>
parents: 14549
diff changeset
903 opts = getattr(opener, 'options', None)
497819817307 revlog: use getattr instead of hasattr
Augie Fackler <durin42@gmail.com>
parents: 14549
diff changeset
904 if opts is not None:
32697
19b9fc40cc51 revlog: skeleton support for version 2 revlogs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32684
diff changeset
905 if 'revlogv2' in opts:
19b9fc40cc51 revlog: skeleton support for version 2 revlogs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32684
diff changeset
906 # version 2 revlogs always use generaldelta.
19b9fc40cc51 revlog: skeleton support for version 2 revlogs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32684
diff changeset
907 v = REVLOGV2 | FLAG_GENERALDELTA | FLAG_INLINE_DATA
19b9fc40cc51 revlog: skeleton support for version 2 revlogs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32684
diff changeset
908 elif 'revlogv1' in opts:
14960
497819817307 revlog: use getattr instead of hasattr
Augie Fackler <durin42@gmail.com>
parents: 14549
diff changeset
909 if 'generaldelta' in opts:
32315
67026d65a4fc revlog: rename constants (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32307
diff changeset
910 v |= FLAG_GENERALDELTA
14333
31a5973fcf96 revlog: get rid of defversion
Sune Foldager <cryo@cyanite.org>
parents: 14325
diff changeset
911 else:
31a5973fcf96 revlog: get rid of defversion
Sune Foldager <cryo@cyanite.org>
parents: 14325
diff changeset
912 v = 0
20180
969148b49fc6 revlog: allow tuning of the chunk cache size (via format.chunkcachesize)
Brodie Rao <brodie@sf.io>
parents: 20179
diff changeset
913 if 'chunkcachesize' in opts:
969148b49fc6 revlog: allow tuning of the chunk cache size (via format.chunkcachesize)
Brodie Rao <brodie@sf.io>
parents: 20179
diff changeset
914 self._chunkcachesize = opts['chunkcachesize']
23255
76effa770ff9 revlog: add config variable for limiting delta-chain length
Mateusz Kwapich <mitrandir@fb.com>
parents: 23254
diff changeset
915 if 'maxchainlen' in opts:
76effa770ff9 revlog: add config variable for limiting delta-chain length
Mateusz Kwapich <mitrandir@fb.com>
parents: 23254
diff changeset
916 self._maxchainlen = opts['maxchainlen']
26118
049005de325e revlog: add an aggressivemergedelta option
Durham Goode <durham@fb.com>
parents: 26117
diff changeset
917 if 'aggressivemergedeltas' in opts:
049005de325e revlog: add an aggressivemergedelta option
Durham Goode <durham@fb.com>
parents: 26117
diff changeset
918 self._aggressivemergedeltas = opts['aggressivemergedeltas']
26907
dfab6edb98e3 format: introduce 'format.usegeneraldelta`
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26705
diff changeset
919 self._lazydeltabase = bool(opts.get('lazydeltabase', False))
30818
4c0a5a256ae8 localrepo: experimental support for non-zlib revlog compression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30817
diff changeset
920 if 'compengine' in opts:
4c0a5a256ae8 localrepo: experimental support for non-zlib revlog compression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30817
diff changeset
921 self._compengine = opts['compengine']
33207
895ecec31c70 revlog: add an experimental option to mitigated delta issues (issue5480)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 33171
diff changeset
922 if 'maxdeltachainspan' in opts:
895ecec31c70 revlog: add an experimental option to mitigated delta issues (issue5480)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 33171
diff changeset
923 self._maxdeltachainspan = opts['maxdeltachainspan']
34296
3c9691728237 revlog: add option to mmap revlog index
Mark Thomas <mbthomas@fb.com>
parents: 34291
diff changeset
924 if mmaplargeindex and 'mmapindexthreshold' in opts:
3c9691728237 revlog: add option to mmap revlog index
Mark Thomas <mbthomas@fb.com>
parents: 34291
diff changeset
925 mmapindexthreshold = opts['mmapindexthreshold']
34824
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
926 self._withsparseread = bool(opts.get('with-sparse-read', False))
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
927 if 'sparse-read-density-threshold' in opts:
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
928 self._srdensitythreshold = opts['sparse-read-density-threshold']
34881
8c9b08a0c48c sparse-read: skip gaps too small to be worth splitting
Paul Morelle <paul.morelle@octobus.net>
parents: 34880
diff changeset
929 if 'sparse-read-min-gap-size' in opts:
8c9b08a0c48c sparse-read: skip gaps too small to be worth splitting
Paul Morelle <paul.morelle@octobus.net>
parents: 34880
diff changeset
930 self._srmingapsize = opts['sparse-read-min-gap-size']
20180
969148b49fc6 revlog: allow tuning of the chunk cache size (via format.chunkcachesize)
Brodie Rao <brodie@sf.io>
parents: 20179
diff changeset
931
969148b49fc6 revlog: allow tuning of the chunk cache size (via format.chunkcachesize)
Brodie Rao <brodie@sf.io>
parents: 20179
diff changeset
932 if self._chunkcachesize <= 0:
969148b49fc6 revlog: allow tuning of the chunk cache size (via format.chunkcachesize)
Brodie Rao <brodie@sf.io>
parents: 20179
diff changeset
933 raise RevlogError(_('revlog chunk cache size %r is not greater '
969148b49fc6 revlog: allow tuning of the chunk cache size (via format.chunkcachesize)
Brodie Rao <brodie@sf.io>
parents: 20179
diff changeset
934 'than 0') % self._chunkcachesize)
969148b49fc6 revlog: allow tuning of the chunk cache size (via format.chunkcachesize)
Brodie Rao <brodie@sf.io>
parents: 20179
diff changeset
935 elif self._chunkcachesize & (self._chunkcachesize - 1):
969148b49fc6 revlog: allow tuning of the chunk cache size (via format.chunkcachesize)
Brodie Rao <brodie@sf.io>
parents: 20179
diff changeset
936 raise RevlogError(_('revlog chunk cache size %r is not a power '
969148b49fc6 revlog: allow tuning of the chunk cache size (via format.chunkcachesize)
Brodie Rao <brodie@sf.io>
parents: 20179
diff changeset
937 'of 2') % self._chunkcachesize)
11928
b69899dbad40 revlog: parentdelta flags for revlog index
Pradeepkumar Gayam <in3xes@gmail.com>
parents: 11759
diff changeset
938
26241
eb97d49768cc revlog: rename generic "i" variable to "indexdata"
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26118
diff changeset
939 indexdata = ''
14334
85c82ebc96a3 changelog: don't use generaldelta
Sune Foldager <cryo@cyanite.org>
parents: 14333
diff changeset
940 self._initempty = True
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
941 try:
35969
1f2b8a644dd4 revlog: use context manager for index file life time in __init__
Boris Feld <boris.feld@octobus.net>
parents: 35968
diff changeset
942 with self._indexfp() as f:
1f2b8a644dd4 revlog: use context manager for index file life time in __init__
Boris Feld <boris.feld@octobus.net>
parents: 35968
diff changeset
943 if (mmapindexthreshold is not None and
1f2b8a644dd4 revlog: use context manager for index file life time in __init__
Boris Feld <boris.feld@octobus.net>
parents: 35968
diff changeset
944 self.opener.fstat(f).st_size >= mmapindexthreshold):
1f2b8a644dd4 revlog: use context manager for index file life time in __init__
Boris Feld <boris.feld@octobus.net>
parents: 35968
diff changeset
945 indexdata = util.buffer(util.mmapread(f))
1f2b8a644dd4 revlog: use context manager for index file life time in __init__
Boris Feld <boris.feld@octobus.net>
parents: 35968
diff changeset
946 else:
1f2b8a644dd4 revlog: use context manager for index file life time in __init__
Boris Feld <boris.feld@octobus.net>
parents: 35968
diff changeset
947 indexdata = f.read()
26241
eb97d49768cc revlog: rename generic "i" variable to "indexdata"
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26118
diff changeset
948 if len(indexdata) > 0:
33392
ac6446611ad2 revlog: use struct.Struct instances for slight performance wins
Alex Gaynor <agaynor@mozilla.com>
parents: 33391
diff changeset
949 v = versionformat_unpack(indexdata[:4])[0]
14334
85c82ebc96a3 changelog: don't use generaldelta
Sune Foldager <cryo@cyanite.org>
parents: 14333
diff changeset
950 self._initempty = False
25660
328739ea70c3 global: mass rewrite to use modern exception syntax
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25459
diff changeset
951 except IOError as inst:
1322
b3d44e9b3092 Make revlog constructor more discerning in its treatment of errors.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1232
diff changeset
952 if inst.errno != errno.ENOENT:
b3d44e9b3092 Make revlog constructor more discerning in its treatment of errors.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1232
diff changeset
953 raise
4985
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
954
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
955 self.version = v
32315
67026d65a4fc revlog: rename constants (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32307
diff changeset
956 self._inline = v & FLAG_INLINE_DATA
67026d65a4fc revlog: rename constants (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32307
diff changeset
957 self._generaldelta = v & FLAG_GENERALDELTA
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
958 flags = v & ~0xFFFF
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
959 fmt = v & 0xFFFF
32392
36d3559c69a6 revlog: tweak wording and logic for flags validation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32372
diff changeset
960 if fmt == REVLOGV0:
36d3559c69a6 revlog: tweak wording and logic for flags validation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32372
diff changeset
961 if flags:
36d3559c69a6 revlog: tweak wording and logic for flags validation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32372
diff changeset
962 raise RevlogError(_('unknown flags (%#04x) in version %d '
36d3559c69a6 revlog: tweak wording and logic for flags validation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32372
diff changeset
963 'revlog %s') %
36d3559c69a6 revlog: tweak wording and logic for flags validation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32372
diff changeset
964 (flags >> 16, fmt, self.indexfile))
36d3559c69a6 revlog: tweak wording and logic for flags validation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32372
diff changeset
965 elif fmt == REVLOGV1:
36d3559c69a6 revlog: tweak wording and logic for flags validation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32372
diff changeset
966 if flags & ~REVLOGV1_FLAGS:
36d3559c69a6 revlog: tweak wording and logic for flags validation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32372
diff changeset
967 raise RevlogError(_('unknown flags (%#04x) in version %d '
36d3559c69a6 revlog: tweak wording and logic for flags validation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32372
diff changeset
968 'revlog %s') %
36d3559c69a6 revlog: tweak wording and logic for flags validation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32372
diff changeset
969 (flags >> 16, fmt, self.indexfile))
32697
19b9fc40cc51 revlog: skeleton support for version 2 revlogs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32684
diff changeset
970 elif fmt == REVLOGV2:
19b9fc40cc51 revlog: skeleton support for version 2 revlogs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32684
diff changeset
971 if flags & ~REVLOGV2_FLAGS:
19b9fc40cc51 revlog: skeleton support for version 2 revlogs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32684
diff changeset
972 raise RevlogError(_('unknown flags (%#04x) in version %d '
19b9fc40cc51 revlog: skeleton support for version 2 revlogs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32684
diff changeset
973 'revlog %s') %
19b9fc40cc51 revlog: skeleton support for version 2 revlogs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32684
diff changeset
974 (flags >> 16, fmt, self.indexfile))
32392
36d3559c69a6 revlog: tweak wording and logic for flags validation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32372
diff changeset
975 else:
36d3559c69a6 revlog: tweak wording and logic for flags validation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32372
diff changeset
976 raise RevlogError(_('unknown version (%d) in revlog %s') %
36d3559c69a6 revlog: tweak wording and logic for flags validation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32372
diff changeset
977 (fmt, self.indexfile))
4985
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
978
30210
5e4f16874a9f revlog: make 'storedeltachains' a "public" attribute
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 30154
diff changeset
979 self.storedeltachains = True
30154
5e72129d75ed revlog: add instance variable controlling delta chain use
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30012
diff changeset
980
4972
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
981 self._io = revlogio()
4971
3e6dae278c99 revlog: regroup parsing code
Matt Mackall <mpm@selenic.com>
parents: 4920
diff changeset
982 if self.version == REVLOGV0:
4972
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
983 self._io = revlogoldio()
13265
04b302ce2781 revlog: always add the magic nullid/nullrev entry in parseindex
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 13264
diff changeset
984 try:
26241
eb97d49768cc revlog: rename generic "i" variable to "indexdata"
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26118
diff changeset
985 d = self._io.parseindex(indexdata, self._inline)
13265
04b302ce2781 revlog: always add the magic nullid/nullrev entry in parseindex
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 13264
diff changeset
986 except (ValueError, IndexError):
04b302ce2781 revlog: always add the magic nullid/nullrev entry in parseindex
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 13264
diff changeset
987 raise RevlogError(_("index %s is corrupted") % (self.indexfile))
13268
fff12b3d953a revlog: explicit test and explicit variable names
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 13267
diff changeset
988 self.index, nodemap, self._chunkcache = d
fff12b3d953a revlog: explicit test and explicit variable names
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 13267
diff changeset
989 if nodemap is not None:
13275
68da048b4c88 revlog: incrementally build node cache with linear searches
Matt Mackall <mpm@selenic.com>
parents: 13268
diff changeset
990 self.nodemap = self._nodecache = nodemap
13265
04b302ce2781 revlog: always add the magic nullid/nullrev entry in parseindex
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 13264
diff changeset
991 if not self._chunkcache:
04b302ce2781 revlog: always add the magic nullid/nullrev entry in parseindex
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 13264
diff changeset
992 self._chunkclear()
23306
f7a42f8e82bd revlog: cache chain info after calculating it for a rev (issue4452)
Siddharth Agarwal <sid0@fb.com>
parents: 23288
diff changeset
993 # revnum -> (chain-length, sum-delta-length)
f7a42f8e82bd revlog: cache chain info after calculating it for a rev (issue4452)
Siddharth Agarwal <sid0@fb.com>
parents: 23288
diff changeset
994 self._chaininfocache = {}
30817
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
995 # revlog header -> revlog compressor
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
996 self._decompressors = {}
116
e484cd5ec282 Only use lazy indexing for big indices and avoid the overhead of the
mpm@selenic.com
parents: 115
diff changeset
997
30795
78ac56aebab6 revlog: use compression engine API for compression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30793
diff changeset
998 @util.propertycache
78ac56aebab6 revlog: use compression engine API for compression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30793
diff changeset
999 def _compressor(self):
30818
4c0a5a256ae8 localrepo: experimental support for non-zlib revlog compression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30817
diff changeset
1000 return util.compengines[self._compengine].revlogcompressor()
30795
78ac56aebab6 revlog: use compression engine API for compression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30793
diff changeset
1001
35968
91b3b11565e5 revlog: move index file opening in a method
Boris Feld <boris.feld@octobus.net>
parents: 35967
diff changeset
1002 def _indexfp(self, mode='r'):
91b3b11565e5 revlog: move index file opening in a method
Boris Feld <boris.feld@octobus.net>
parents: 35967
diff changeset
1003 """file object for the revlog's index file"""
91b3b11565e5 revlog: move index file opening in a method
Boris Feld <boris.feld@octobus.net>
parents: 35967
diff changeset
1004 args = {r'mode': mode}
91b3b11565e5 revlog: move index file opening in a method
Boris Feld <boris.feld@octobus.net>
parents: 35967
diff changeset
1005 if mode != 'r':
91b3b11565e5 revlog: move index file opening in a method
Boris Feld <boris.feld@octobus.net>
parents: 35967
diff changeset
1006 args[r'checkambig'] = self._checkambig
91b3b11565e5 revlog: move index file opening in a method
Boris Feld <boris.feld@octobus.net>
parents: 35967
diff changeset
1007 if mode == 'w':
91b3b11565e5 revlog: move index file opening in a method
Boris Feld <boris.feld@octobus.net>
parents: 35967
diff changeset
1008 args[r'atomictemp'] = True
91b3b11565e5 revlog: move index file opening in a method
Boris Feld <boris.feld@octobus.net>
parents: 35967
diff changeset
1009 return self.opener(self.indexfile, **args)
91b3b11565e5 revlog: move index file opening in a method
Boris Feld <boris.feld@octobus.net>
parents: 35967
diff changeset
1010
35967
61326dd7cb8d revlog: move datafile opening in a method
Boris Feld <boris.feld@octobus.net>
parents: 35845
diff changeset
1011 def _datafp(self, mode='r'):
61326dd7cb8d revlog: move datafile opening in a method
Boris Feld <boris.feld@octobus.net>
parents: 35845
diff changeset
1012 """file object for the revlog's data file"""
61326dd7cb8d revlog: move datafile opening in a method
Boris Feld <boris.feld@octobus.net>
parents: 35845
diff changeset
1013 return self.opener(self.datafile, mode=mode)
61326dd7cb8d revlog: move datafile opening in a method
Boris Feld <boris.feld@octobus.net>
parents: 35845
diff changeset
1014
35973
4d66993bdcff revlog: add a _datareadfp context manager for data access needs
Boris Feld <boris.feld@octobus.net>
parents: 35972
diff changeset
1015 @contextlib.contextmanager
4d66993bdcff revlog: add a _datareadfp context manager for data access needs
Boris Feld <boris.feld@octobus.net>
parents: 35972
diff changeset
1016 def _datareadfp(self, existingfp=None):
4d66993bdcff revlog: add a _datareadfp context manager for data access needs
Boris Feld <boris.feld@octobus.net>
parents: 35972
diff changeset
1017 """file object suitable to read data"""
4d66993bdcff revlog: add a _datareadfp context manager for data access needs
Boris Feld <boris.feld@octobus.net>
parents: 35972
diff changeset
1018 if existingfp is not None:
4d66993bdcff revlog: add a _datareadfp context manager for data access needs
Boris Feld <boris.feld@octobus.net>
parents: 35972
diff changeset
1019 yield existingfp
4d66993bdcff revlog: add a _datareadfp context manager for data access needs
Boris Feld <boris.feld@octobus.net>
parents: 35972
diff changeset
1020 else:
4d66993bdcff revlog: add a _datareadfp context manager for data access needs
Boris Feld <boris.feld@octobus.net>
parents: 35972
diff changeset
1021 if self._inline:
4d66993bdcff revlog: add a _datareadfp context manager for data access needs
Boris Feld <boris.feld@octobus.net>
parents: 35972
diff changeset
1022 func = self._indexfp
4d66993bdcff revlog: add a _datareadfp context manager for data access needs
Boris Feld <boris.feld@octobus.net>
parents: 35972
diff changeset
1023 else:
4d66993bdcff revlog: add a _datareadfp context manager for data access needs
Boris Feld <boris.feld@octobus.net>
parents: 35972
diff changeset
1024 func = self._datafp
4d66993bdcff revlog: add a _datareadfp context manager for data access needs
Boris Feld <boris.feld@octobus.net>
parents: 35972
diff changeset
1025 with func() as fp:
4d66993bdcff revlog: add a _datareadfp context manager for data access needs
Boris Feld <boris.feld@octobus.net>
parents: 35972
diff changeset
1026 yield fp
4d66993bdcff revlog: add a _datareadfp context manager for data access needs
Boris Feld <boris.feld@octobus.net>
parents: 35972
diff changeset
1027
4980
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
1028 def tip(self):
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
1029 return self.node(len(self.index) - 2)
24030
828dc8db5515 revlog: add __contains__ for fast membership test
Yuya Nishihara <yuya@tcha.org>
parents: 23857
diff changeset
1030 def __contains__(self, rev):
828dc8db5515 revlog: add __contains__ for fast membership test
Yuya Nishihara <yuya@tcha.org>
parents: 23857
diff changeset
1031 return 0 <= rev < len(self)
6750
fb42030d79d6 add __len__ and __iter__ methods to repo and revlog
Matt Mackall <mpm@selenic.com>
parents: 6703
diff changeset
1032 def __len__(self):
4980
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
1033 return len(self.index) - 1
6750
fb42030d79d6 add __len__ and __iter__ methods to repo and revlog
Matt Mackall <mpm@selenic.com>
parents: 6703
diff changeset
1034 def __iter__(self):
17951
6f79c32c0bdf commit: increase perf by avoiding unnecessary filteredrevs check
Durham Goode <durham@fb.com>
parents: 17674
diff changeset
1035 return iter(xrange(len(self)))
17672
474047947b8f clfilter: make the revlog class responsible of all its iteration
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17537
diff changeset
1036 def revs(self, start=0, stop=None):
474047947b8f clfilter: make the revlog class responsible of all its iteration
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17537
diff changeset
1037 """iterate over all rev in this revlog (from start to stop)"""
17975
c56b5b65430d revlog: allow reverse iteration with revlog.revs
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17972
diff changeset
1038 step = 1
c56b5b65430d revlog: allow reverse iteration with revlog.revs
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17972
diff changeset
1039 if stop is not None:
c56b5b65430d revlog: allow reverse iteration with revlog.revs
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17972
diff changeset
1040 if start > stop:
c56b5b65430d revlog: allow reverse iteration with revlog.revs
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17972
diff changeset
1041 step = -1
c56b5b65430d revlog: allow reverse iteration with revlog.revs
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17972
diff changeset
1042 stop += step
17672
474047947b8f clfilter: make the revlog class responsible of all its iteration
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17537
diff changeset
1043 else:
17975
c56b5b65430d revlog: allow reverse iteration with revlog.revs
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17972
diff changeset
1044 stop = len(self)
c56b5b65430d revlog: allow reverse iteration with revlog.revs
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17972
diff changeset
1045 return xrange(start, stop, step)
13275
68da048b4c88 revlog: incrementally build node cache with linear searches
Matt Mackall <mpm@selenic.com>
parents: 13268
diff changeset
1046
68da048b4c88 revlog: incrementally build node cache with linear searches
Matt Mackall <mpm@selenic.com>
parents: 13268
diff changeset
1047 @util.propertycache
68da048b4c88 revlog: incrementally build node cache with linear searches
Matt Mackall <mpm@selenic.com>
parents: 13268
diff changeset
1048 def nodemap(self):
14064
e4bfb9c337f3 remove unused imports and variables
Alexander Solovyov <alexander@solovyov.net>
parents: 13831
diff changeset
1049 self.rev(self.node(0))
13275
68da048b4c88 revlog: incrementally build node cache with linear searches
Matt Mackall <mpm@selenic.com>
parents: 13268
diff changeset
1050 return self._nodecache
13259
3b616dfa4b17 revlog: do revlog node->rev mapping by scanning
Matt Mackall <mpm@selenic.com>
parents: 13258
diff changeset
1051
16374
29c2ff719715 revlog: add hasnode helper method
Matt Mackall <mpm@selenic.com>
parents: 15890
diff changeset
1052 def hasnode(self, node):
29c2ff719715 revlog: add hasnode helper method
Matt Mackall <mpm@selenic.com>
parents: 15890
diff changeset
1053 try:
29c2ff719715 revlog: add hasnode helper method
Matt Mackall <mpm@selenic.com>
parents: 15890
diff changeset
1054 self.rev(node)
29c2ff719715 revlog: add hasnode helper method
Matt Mackall <mpm@selenic.com>
parents: 15890
diff changeset
1055 return True
29c2ff719715 revlog: add hasnode helper method
Matt Mackall <mpm@selenic.com>
parents: 15890
diff changeset
1056 except KeyError:
29c2ff719715 revlog: add hasnode helper method
Matt Mackall <mpm@selenic.com>
parents: 15890
diff changeset
1057 return False
29c2ff719715 revlog: add hasnode helper method
Matt Mackall <mpm@selenic.com>
parents: 15890
diff changeset
1058
36743
d031609b3cb7 changegroup: do not delta lfs revisions
Jun Wu <quark@fb.com>
parents: 35738
diff changeset
1059 def candelta(self, baserev, rev):
d031609b3cb7 changegroup: do not delta lfs revisions
Jun Wu <quark@fb.com>
parents: 35738
diff changeset
1060 """whether two revisions (baserev, rev) can be delta-ed or not"""
d031609b3cb7 changegroup: do not delta lfs revisions
Jun Wu <quark@fb.com>
parents: 35738
diff changeset
1061 # Disable delta if either rev requires a content-changing flag
d031609b3cb7 changegroup: do not delta lfs revisions
Jun Wu <quark@fb.com>
parents: 35738
diff changeset
1062 # processor (ex. LFS). This is because such flag processor can alter
d031609b3cb7 changegroup: do not delta lfs revisions
Jun Wu <quark@fb.com>
parents: 35738
diff changeset
1063 # the rawtext content that the delta will be based on, and two clients
d031609b3cb7 changegroup: do not delta lfs revisions
Jun Wu <quark@fb.com>
parents: 35738
diff changeset
1064 # could have a same revlog node with different flags (i.e. different
d031609b3cb7 changegroup: do not delta lfs revisions
Jun Wu <quark@fb.com>
parents: 35738
diff changeset
1065 # rawtext contents) and the delta could be incompatible.
d031609b3cb7 changegroup: do not delta lfs revisions
Jun Wu <quark@fb.com>
parents: 35738
diff changeset
1066 if ((self.flags(baserev) & REVIDX_RAWTEXT_CHANGING_FLAGS)
d031609b3cb7 changegroup: do not delta lfs revisions
Jun Wu <quark@fb.com>
parents: 35738
diff changeset
1067 or (self.flags(rev) & REVIDX_RAWTEXT_CHANGING_FLAGS)):
d031609b3cb7 changegroup: do not delta lfs revisions
Jun Wu <quark@fb.com>
parents: 35738
diff changeset
1068 return False
d031609b3cb7 changegroup: do not delta lfs revisions
Jun Wu <quark@fb.com>
parents: 35738
diff changeset
1069 return True
d031609b3cb7 changegroup: do not delta lfs revisions
Jun Wu <quark@fb.com>
parents: 35738
diff changeset
1070
16414
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16375
diff changeset
1071 def clearcaches(self):
27465
072a675c51f2 revlog: make clearcaches() more effective
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27449
diff changeset
1072 self._cache = None
29830
92ac2baaea86 revlog: use an LRU cache for delta chain bases
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29829
diff changeset
1073 self._chainbasecache.clear()
27465
072a675c51f2 revlog: make clearcaches() more effective
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27449
diff changeset
1074 self._chunkcache = (0, '')
072a675c51f2 revlog: make clearcaches() more effective
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27449
diff changeset
1075 self._pcache = {}
072a675c51f2 revlog: make clearcaches() more effective
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27449
diff changeset
1076
16414
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16375
diff changeset
1077 try:
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16375
diff changeset
1078 self._nodecache.clearcaches()
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16375
diff changeset
1079 except AttributeError:
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16375
diff changeset
1080 self._nodecache = {nullid: nullrev}
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16375
diff changeset
1081 self._nodepos = None
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16375
diff changeset
1082
13259
3b616dfa4b17 revlog: do revlog node->rev mapping by scanning
Matt Mackall <mpm@selenic.com>
parents: 13258
diff changeset
1083 def rev(self, node):
13275
68da048b4c88 revlog: incrementally build node cache with linear searches
Matt Mackall <mpm@selenic.com>
parents: 13268
diff changeset
1084 try:
68da048b4c88 revlog: incrementally build node cache with linear searches
Matt Mackall <mpm@selenic.com>
parents: 13268
diff changeset
1085 return self._nodecache[node]
22282
4092d12ba18a repoview: fix 0L with pack/unpack for 2.4
Matt Mackall <mpm@selenic.com>
parents: 21752
diff changeset
1086 except TypeError:
4092d12ba18a repoview: fix 0L with pack/unpack for 2.4
Matt Mackall <mpm@selenic.com>
parents: 21752
diff changeset
1087 raise
16414
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16375
diff changeset
1088 except RevlogError:
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16375
diff changeset
1089 # parsers.c radix tree lookup failed
37449
a0d71618074f revlog: detect pseudo file nodeids to raise WdirUnsupported exception
Yuya Nishihara <yuya@tcha.org>
parents: 37443
diff changeset
1090 if node == wdirid or node in wdirfilenodeids:
32659
7b17f9de6d3e revlog: map rev(wdirid) to WdirUnsupported exception
Yuya Nishihara <yuya@tcha.org>
parents: 32443
diff changeset
1091 raise error.WdirUnsupported
16414
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16375
diff changeset
1092 raise LookupError(node, self.indexfile, _('no node'))
13275
68da048b4c88 revlog: incrementally build node cache with linear searches
Matt Mackall <mpm@selenic.com>
parents: 13268
diff changeset
1093 except KeyError:
16414
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16375
diff changeset
1094 # pure python cache lookup failed
13275
68da048b4c88 revlog: incrementally build node cache with linear searches
Matt Mackall <mpm@selenic.com>
parents: 13268
diff changeset
1095 n = self._nodecache
68da048b4c88 revlog: incrementally build node cache with linear searches
Matt Mackall <mpm@selenic.com>
parents: 13268
diff changeset
1096 i = self.index
68da048b4c88 revlog: incrementally build node cache with linear searches
Matt Mackall <mpm@selenic.com>
parents: 13268
diff changeset
1097 p = self._nodepos
68da048b4c88 revlog: incrementally build node cache with linear searches
Matt Mackall <mpm@selenic.com>
parents: 13268
diff changeset
1098 if p is None:
68da048b4c88 revlog: incrementally build node cache with linear searches
Matt Mackall <mpm@selenic.com>
parents: 13268
diff changeset
1099 p = len(i) - 2
37494
1ce7a55b09d1 revlog: reset _nodepos after strip
Joerg Sonnenberger <joerg@bec.de>
parents: 37449
diff changeset
1100 else:
1ce7a55b09d1 revlog: reset _nodepos after strip
Joerg Sonnenberger <joerg@bec.de>
parents: 37449
diff changeset
1101 assert p < len(i)
13275
68da048b4c88 revlog: incrementally build node cache with linear searches
Matt Mackall <mpm@selenic.com>
parents: 13268
diff changeset
1102 for r in xrange(p, -1, -1):
68da048b4c88 revlog: incrementally build node cache with linear searches
Matt Mackall <mpm@selenic.com>
parents: 13268
diff changeset
1103 v = i[r][7]
68da048b4c88 revlog: incrementally build node cache with linear searches
Matt Mackall <mpm@selenic.com>
parents: 13268
diff changeset
1104 n[v] = r
68da048b4c88 revlog: incrementally build node cache with linear searches
Matt Mackall <mpm@selenic.com>
parents: 13268
diff changeset
1105 if v == node:
68da048b4c88 revlog: incrementally build node cache with linear searches
Matt Mackall <mpm@selenic.com>
parents: 13268
diff changeset
1106 self._nodepos = r - 1
68da048b4c88 revlog: incrementally build node cache with linear searches
Matt Mackall <mpm@selenic.com>
parents: 13268
diff changeset
1107 return r
37449
a0d71618074f revlog: detect pseudo file nodeids to raise WdirUnsupported exception
Yuya Nishihara <yuya@tcha.org>
parents: 37443
diff changeset
1108 if node == wdirid or node in wdirfilenodeids:
32659
7b17f9de6d3e revlog: map rev(wdirid) to WdirUnsupported exception
Yuya Nishihara <yuya@tcha.org>
parents: 32443
diff changeset
1109 raise error.WdirUnsupported
13275
68da048b4c88 revlog: incrementally build node cache with linear searches
Matt Mackall <mpm@selenic.com>
parents: 13268
diff changeset
1110 raise LookupError(node, self.indexfile, _('no node'))
68da048b4c88 revlog: incrementally build node cache with linear searches
Matt Mackall <mpm@selenic.com>
parents: 13268
diff changeset
1111
30287
0986f225c149 revlog: reorder index accessors to match data structure order
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30210
diff changeset
1112 # Accessors for index entries.
0986f225c149 revlog: reorder index accessors to match data structure order
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30210
diff changeset
1113
0986f225c149 revlog: reorder index accessors to match data structure order
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30210
diff changeset
1114 # First tuple entry is 8 bytes. First 6 bytes are offset. Last 2 bytes
0986f225c149 revlog: reorder index accessors to match data structure order
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30210
diff changeset
1115 # are flags.
2072
74d3f5336b66 Implement revlogng.
mason@suse.com
parents: 2002
diff changeset
1116 def start(self, rev):
5006
c2febf5420e9 revlog: minor chunk speed-up
Matt Mackall <mpm@selenic.com>
parents: 5005
diff changeset
1117 return int(self.index[rev][0] >> 16)
30287
0986f225c149 revlog: reorder index accessors to match data structure order
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30210
diff changeset
1118
0986f225c149 revlog: reorder index accessors to match data structure order
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30210
diff changeset
1119 def flags(self, rev):
0986f225c149 revlog: reorder index accessors to match data structure order
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30210
diff changeset
1120 return self.index[rev][0] & 0xFFFF
0986f225c149 revlog: reorder index accessors to match data structure order
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30210
diff changeset
1121
4980
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
1122 def length(self, rev):
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
1123 return self.index[rev][1]
30287
0986f225c149 revlog: reorder index accessors to match data structure order
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30210
diff changeset
1124
0986f225c149 revlog: reorder index accessors to match data structure order
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30210
diff changeset
1125 def rawsize(self, rev):
0986f225c149 revlog: reorder index accessors to match data structure order
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30210
diff changeset
1126 """return the length of the uncompressed text for a given revision"""
0986f225c149 revlog: reorder index accessors to match data structure order
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30210
diff changeset
1127 l = self.index[rev][2]
38177
7fa3408f83ef revlog: disallow setting uncompressed length to None
Yuya Nishihara <yuya@tcha.org>
parents: 38169
diff changeset
1128 if l >= 0:
30287
0986f225c149 revlog: reorder index accessors to match data structure order
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30210
diff changeset
1129 return l
0986f225c149 revlog: reorder index accessors to match data structure order
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30210
diff changeset
1130
31801
d22f29abeb42 revlog: use raw revision for rawsize
Jun Wu <quark@fb.com>
parents: 31756
diff changeset
1131 t = self.revision(rev, raw=True)
30287
0986f225c149 revlog: reorder index accessors to match data structure order
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30210
diff changeset
1132 return len(t)
31856
0ab7f469d386 revlog: make "size" diverge from "rawsize"
Jun Wu <quark@fb.com>
parents: 31804
diff changeset
1133
0ab7f469d386 revlog: make "size" diverge from "rawsize"
Jun Wu <quark@fb.com>
parents: 31804
diff changeset
1134 def size(self, rev):
0ab7f469d386 revlog: make "size" diverge from "rawsize"
Jun Wu <quark@fb.com>
parents: 31804
diff changeset
1135 """length of non-raw text (processed by a "read" flag processor)"""
0ab7f469d386 revlog: make "size" diverge from "rawsize"
Jun Wu <quark@fb.com>
parents: 31804
diff changeset
1136 # fast path: if no "read" flag processor could change the content,
0ab7f469d386 revlog: make "size" diverge from "rawsize"
Jun Wu <quark@fb.com>
parents: 31804
diff changeset
1137 # size is rawsize. note: ELLIPSIS is known to not change the content.
0ab7f469d386 revlog: make "size" diverge from "rawsize"
Jun Wu <quark@fb.com>
parents: 31804
diff changeset
1138 flags = self.flags(rev)
0ab7f469d386 revlog: make "size" diverge from "rawsize"
Jun Wu <quark@fb.com>
parents: 31804
diff changeset
1139 if flags & (REVIDX_KNOWN_FLAGS ^ REVIDX_ELLIPSIS) == 0:
0ab7f469d386 revlog: make "size" diverge from "rawsize"
Jun Wu <quark@fb.com>
parents: 31804
diff changeset
1140 return self.rawsize(rev)
0ab7f469d386 revlog: make "size" diverge from "rawsize"
Jun Wu <quark@fb.com>
parents: 31804
diff changeset
1141
0ab7f469d386 revlog: make "size" diverge from "rawsize"
Jun Wu <quark@fb.com>
parents: 31804
diff changeset
1142 return len(self.revision(rev, raw=False))
30287
0986f225c149 revlog: reorder index accessors to match data structure order
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30210
diff changeset
1143
14252
19067884c5f5 revlog: calculate base revisions iteratively
Sune Foldager <cryo@cyanite.org>
parents: 14251
diff changeset
1144 def chainbase(self, rev):
29830
92ac2baaea86 revlog: use an LRU cache for delta chain bases
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29829
diff changeset
1145 base = self._chainbasecache.get(rev)
92ac2baaea86 revlog: use an LRU cache for delta chain bases
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29829
diff changeset
1146 if base is not None:
92ac2baaea86 revlog: use an LRU cache for delta chain bases
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29829
diff changeset
1147 return base
92ac2baaea86 revlog: use an LRU cache for delta chain bases
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29829
diff changeset
1148
14252
19067884c5f5 revlog: calculate base revisions iteratively
Sune Foldager <cryo@cyanite.org>
parents: 14251
diff changeset
1149 index = self.index
38169
fc72beec2a1a revlog: make chainbase cache its result for the correct revision
Paul Morelle <paul.morelle@octobus.net>
parents: 38168
diff changeset
1150 iterrev = rev
fc72beec2a1a revlog: make chainbase cache its result for the correct revision
Paul Morelle <paul.morelle@octobus.net>
parents: 38168
diff changeset
1151 base = index[iterrev][3]
fc72beec2a1a revlog: make chainbase cache its result for the correct revision
Paul Morelle <paul.morelle@octobus.net>
parents: 38168
diff changeset
1152 while base != iterrev:
fc72beec2a1a revlog: make chainbase cache its result for the correct revision
Paul Morelle <paul.morelle@octobus.net>
parents: 38168
diff changeset
1153 iterrev = base
fc72beec2a1a revlog: make chainbase cache its result for the correct revision
Paul Morelle <paul.morelle@octobus.net>
parents: 38168
diff changeset
1154 base = index[iterrev][3]
29830
92ac2baaea86 revlog: use an LRU cache for delta chain bases
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29829
diff changeset
1155
92ac2baaea86 revlog: use an LRU cache for delta chain bases
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29829
diff changeset
1156 self._chainbasecache[rev] = base
14252
19067884c5f5 revlog: calculate base revisions iteratively
Sune Foldager <cryo@cyanite.org>
parents: 14251
diff changeset
1157 return base
30287
0986f225c149 revlog: reorder index accessors to match data structure order
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30210
diff changeset
1158
0986f225c149 revlog: reorder index accessors to match data structure order
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30210
diff changeset
1159 def linkrev(self, rev):
0986f225c149 revlog: reorder index accessors to match data structure order
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30210
diff changeset
1160 return self.index[rev][4]
0986f225c149 revlog: reorder index accessors to match data structure order
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30210
diff changeset
1161
0986f225c149 revlog: reorder index accessors to match data structure order
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30210
diff changeset
1162 def parentrevs(self, rev):
32403
a28c76e1cea9 revlog: raise WdirUnsupported when wdirrev is passed
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32393
diff changeset
1163 try:
35521
a0fab647a8f1 revlog: don't use slicing to return parents
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35345
diff changeset
1164 entry = self.index[rev]
32403
a28c76e1cea9 revlog: raise WdirUnsupported when wdirrev is passed
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32393
diff changeset
1165 except IndexError:
a28c76e1cea9 revlog: raise WdirUnsupported when wdirrev is passed
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32393
diff changeset
1166 if rev == wdirrev:
a28c76e1cea9 revlog: raise WdirUnsupported when wdirrev is passed
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32393
diff changeset
1167 raise error.WdirUnsupported
a28c76e1cea9 revlog: raise WdirUnsupported when wdirrev is passed
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32393
diff changeset
1168 raise
30287
0986f225c149 revlog: reorder index accessors to match data structure order
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30210
diff changeset
1169
35521
a0fab647a8f1 revlog: don't use slicing to return parents
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35345
diff changeset
1170 return entry[5], entry[6]
a0fab647a8f1 revlog: don't use slicing to return parents
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35345
diff changeset
1171
30287
0986f225c149 revlog: reorder index accessors to match data structure order
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30210
diff changeset
1172 def node(self, rev):
32443
34e9b8b94f66 revlog: raise error.WdirUnsupported from revlog.node() if wdirrev is passed
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32403
diff changeset
1173 try:
34e9b8b94f66 revlog: raise error.WdirUnsupported from revlog.node() if wdirrev is passed
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32403
diff changeset
1174 return self.index[rev][7]
34e9b8b94f66 revlog: raise error.WdirUnsupported from revlog.node() if wdirrev is passed
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32403
diff changeset
1175 except IndexError:
34e9b8b94f66 revlog: raise error.WdirUnsupported from revlog.node() if wdirrev is passed
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32403
diff changeset
1176 if rev == wdirrev:
34e9b8b94f66 revlog: raise error.WdirUnsupported from revlog.node() if wdirrev is passed
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32403
diff changeset
1177 raise error.WdirUnsupported
34e9b8b94f66 revlog: raise error.WdirUnsupported from revlog.node() if wdirrev is passed
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32403
diff changeset
1178 raise
30287
0986f225c149 revlog: reorder index accessors to match data structure order
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30210
diff changeset
1179
0986f225c149 revlog: reorder index accessors to match data structure order
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30210
diff changeset
1180 # Derived from index values.
0986f225c149 revlog: reorder index accessors to match data structure order
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30210
diff changeset
1181
0986f225c149 revlog: reorder index accessors to match data structure order
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30210
diff changeset
1182 def end(self, rev):
0986f225c149 revlog: reorder index accessors to match data structure order
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30210
diff changeset
1183 return self.start(rev) + self.length(rev)
0986f225c149 revlog: reorder index accessors to match data structure order
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30210
diff changeset
1184
0986f225c149 revlog: reorder index accessors to match data structure order
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30210
diff changeset
1185 def parents(self, node):
0986f225c149 revlog: reorder index accessors to match data structure order
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30210
diff changeset
1186 i = self.index
0986f225c149 revlog: reorder index accessors to match data structure order
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30210
diff changeset
1187 d = i[self.rev(node)]
0986f225c149 revlog: reorder index accessors to match data structure order
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30210
diff changeset
1188 return i[d[5]][7], i[d[6]][7] # map revisions to nodes inline
0986f225c149 revlog: reorder index accessors to match data structure order
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30210
diff changeset
1189
23254
d23834b871ac debugrevlog: fix computing chain length in debugrevlog -d
Mateusz Kwapich <mitrandir@fb.com>
parents: 22934
diff changeset
1190 def chainlen(self, rev):
23286
40e0067899d4 revlog: compute length of compressed deltas along with chain length
Siddharth Agarwal <sid0@fb.com>
parents: 23285
diff changeset
1191 return self._chaininfo(rev)[0]
23306
f7a42f8e82bd revlog: cache chain info after calculating it for a rev (issue4452)
Siddharth Agarwal <sid0@fb.com>
parents: 23288
diff changeset
1192
23286
40e0067899d4 revlog: compute length of compressed deltas along with chain length
Siddharth Agarwal <sid0@fb.com>
parents: 23285
diff changeset
1193 def _chaininfo(self, rev):
23306
f7a42f8e82bd revlog: cache chain info after calculating it for a rev (issue4452)
Siddharth Agarwal <sid0@fb.com>
parents: 23288
diff changeset
1194 chaininfocache = self._chaininfocache
f7a42f8e82bd revlog: cache chain info after calculating it for a rev (issue4452)
Siddharth Agarwal <sid0@fb.com>
parents: 23288
diff changeset
1195 if rev in chaininfocache:
f7a42f8e82bd revlog: cache chain info after calculating it for a rev (issue4452)
Siddharth Agarwal <sid0@fb.com>
parents: 23288
diff changeset
1196 return chaininfocache[rev]
23254
d23834b871ac debugrevlog: fix computing chain length in debugrevlog -d
Mateusz Kwapich <mitrandir@fb.com>
parents: 22934
diff changeset
1197 index = self.index
d23834b871ac debugrevlog: fix computing chain length in debugrevlog -d
Mateusz Kwapich <mitrandir@fb.com>
parents: 22934
diff changeset
1198 generaldelta = self._generaldelta
d23834b871ac debugrevlog: fix computing chain length in debugrevlog -d
Mateusz Kwapich <mitrandir@fb.com>
parents: 22934
diff changeset
1199 iterrev = rev
d23834b871ac debugrevlog: fix computing chain length in debugrevlog -d
Mateusz Kwapich <mitrandir@fb.com>
parents: 22934
diff changeset
1200 e = index[iterrev]
d23834b871ac debugrevlog: fix computing chain length in debugrevlog -d
Mateusz Kwapich <mitrandir@fb.com>
parents: 22934
diff changeset
1201 clen = 0
23286
40e0067899d4 revlog: compute length of compressed deltas along with chain length
Siddharth Agarwal <sid0@fb.com>
parents: 23285
diff changeset
1202 compresseddeltalen = 0
23254
d23834b871ac debugrevlog: fix computing chain length in debugrevlog -d
Mateusz Kwapich <mitrandir@fb.com>
parents: 22934
diff changeset
1203 while iterrev != e[3]:
d23834b871ac debugrevlog: fix computing chain length in debugrevlog -d
Mateusz Kwapich <mitrandir@fb.com>
parents: 22934
diff changeset
1204 clen += 1
23286
40e0067899d4 revlog: compute length of compressed deltas along with chain length
Siddharth Agarwal <sid0@fb.com>
parents: 23285
diff changeset
1205 compresseddeltalen += e[1]
23254
d23834b871ac debugrevlog: fix computing chain length in debugrevlog -d
Mateusz Kwapich <mitrandir@fb.com>
parents: 22934
diff changeset
1206 if generaldelta:
d23834b871ac debugrevlog: fix computing chain length in debugrevlog -d
Mateusz Kwapich <mitrandir@fb.com>
parents: 22934
diff changeset
1207 iterrev = e[3]
d23834b871ac debugrevlog: fix computing chain length in debugrevlog -d
Mateusz Kwapich <mitrandir@fb.com>
parents: 22934
diff changeset
1208 else:
d23834b871ac debugrevlog: fix computing chain length in debugrevlog -d
Mateusz Kwapich <mitrandir@fb.com>
parents: 22934
diff changeset
1209 iterrev -= 1
23306
f7a42f8e82bd revlog: cache chain info after calculating it for a rev (issue4452)
Siddharth Agarwal <sid0@fb.com>
parents: 23288
diff changeset
1210 if iterrev in chaininfocache:
f7a42f8e82bd revlog: cache chain info after calculating it for a rev (issue4452)
Siddharth Agarwal <sid0@fb.com>
parents: 23288
diff changeset
1211 t = chaininfocache[iterrev]
f7a42f8e82bd revlog: cache chain info after calculating it for a rev (issue4452)
Siddharth Agarwal <sid0@fb.com>
parents: 23288
diff changeset
1212 clen += t[0]
f7a42f8e82bd revlog: cache chain info after calculating it for a rev (issue4452)
Siddharth Agarwal <sid0@fb.com>
parents: 23288
diff changeset
1213 compresseddeltalen += t[1]
f7a42f8e82bd revlog: cache chain info after calculating it for a rev (issue4452)
Siddharth Agarwal <sid0@fb.com>
parents: 23288
diff changeset
1214 break
23254
d23834b871ac debugrevlog: fix computing chain length in debugrevlog -d
Mateusz Kwapich <mitrandir@fb.com>
parents: 22934
diff changeset
1215 e = index[iterrev]
23306
f7a42f8e82bd revlog: cache chain info after calculating it for a rev (issue4452)
Siddharth Agarwal <sid0@fb.com>
parents: 23288
diff changeset
1216 else:
f7a42f8e82bd revlog: cache chain info after calculating it for a rev (issue4452)
Siddharth Agarwal <sid0@fb.com>
parents: 23288
diff changeset
1217 # Add text length of base since decompressing that also takes
f7a42f8e82bd revlog: cache chain info after calculating it for a rev (issue4452)
Siddharth Agarwal <sid0@fb.com>
parents: 23288
diff changeset
1218 # work. For cache hits the length is already included.
f7a42f8e82bd revlog: cache chain info after calculating it for a rev (issue4452)
Siddharth Agarwal <sid0@fb.com>
parents: 23288
diff changeset
1219 compresseddeltalen += e[1]
f7a42f8e82bd revlog: cache chain info after calculating it for a rev (issue4452)
Siddharth Agarwal <sid0@fb.com>
parents: 23288
diff changeset
1220 r = (clen, compresseddeltalen)
f7a42f8e82bd revlog: cache chain info after calculating it for a rev (issue4452)
Siddharth Agarwal <sid0@fb.com>
parents: 23288
diff changeset
1221 chaininfocache[rev] = r
f7a42f8e82bd revlog: cache chain info after calculating it for a rev (issue4452)
Siddharth Agarwal <sid0@fb.com>
parents: 23288
diff changeset
1222 return r
f7a42f8e82bd revlog: cache chain info after calculating it for a rev (issue4452)
Siddharth Agarwal <sid0@fb.com>
parents: 23288
diff changeset
1223
27468
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
1224 def _deltachain(self, rev, stoprev=None):
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
1225 """Obtain the delta chain for a revision.
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
1226
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
1227 ``stoprev`` specifies a revision to stop at. If not specified, we
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
1228 stop at the base of the chain.
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
1229
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
1230 Returns a 2-tuple of (chain, stopped) where ``chain`` is a list of
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
1231 revs in ascending order and ``stopped`` is a bool indicating whether
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
1232 ``stoprev`` was hit.
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
1233 """
33171
6d678ab1b10d revlog: C implementation of delta chain resolution
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32969
diff changeset
1234 # Try C implementation.
6d678ab1b10d revlog: C implementation of delta chain resolution
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32969
diff changeset
1235 try:
6d678ab1b10d revlog: C implementation of delta chain resolution
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32969
diff changeset
1236 return self.index.deltachain(rev, stoprev, self._generaldelta)
6d678ab1b10d revlog: C implementation of delta chain resolution
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32969
diff changeset
1237 except AttributeError:
6d678ab1b10d revlog: C implementation of delta chain resolution
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32969
diff changeset
1238 pass
6d678ab1b10d revlog: C implementation of delta chain resolution
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32969
diff changeset
1239
27468
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
1240 chain = []
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
1241
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
1242 # Alias to prevent attribute lookup in tight loop.
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
1243 index = self.index
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
1244 generaldelta = self._generaldelta
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
1245
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
1246 iterrev = rev
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
1247 e = index[iterrev]
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
1248 while iterrev != e[3] and iterrev != stoprev:
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
1249 chain.append(iterrev)
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
1250 if generaldelta:
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
1251 iterrev = e[3]
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
1252 else:
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
1253 iterrev -= 1
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
1254 e = index[iterrev]
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
1255
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
1256 if iterrev == stoprev:
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
1257 stopped = True
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
1258 else:
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
1259 chain.append(iterrev)
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
1260 stopped = False
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
1261
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
1262 chain.reverse()
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
1263 return chain, stopped
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
1264
18081
f88c60e740a1 revlog.ancestors: add support for including revs
Siddharth Agarwal <sid0@fb.com>
parents: 17975
diff changeset
1265 def ancestors(self, revs, stoprev=0, inclusive=False):
10047
27267b1f68b4 revlog: rewrite several method docstrings
Greg Ward <greg-hg@gerg.ca>
parents: 9679
diff changeset
1266 """Generate the ancestors of 'revs' in reverse topological order.
16868
eb88ed4269c5 revlog: add optional stoprev arg to revlog.ancestors()
Joshua Redstone <joshua.redstone@fb.com>
parents: 16867
diff changeset
1267 Does not generate revs lower than stoprev.
10047
27267b1f68b4 revlog: rewrite several method docstrings
Greg Ward <greg-hg@gerg.ca>
parents: 9679
diff changeset
1268
18090
9abc55ef85b5 revlog: move ancestor generation out to a new class
Siddharth Agarwal <sid0@fb.com>
parents: 18083
diff changeset
1269 See the documentation for ancestor.lazyancestors for more details."""
18081
f88c60e740a1 revlog.ancestors: add support for including revs
Siddharth Agarwal <sid0@fb.com>
parents: 17975
diff changeset
1270
23328
3a7d9c0c57a5 ancestor.lazyancestors: take parentrevs function rather than changelog
Siddharth Agarwal <sid0@fb.com>
parents: 23306
diff changeset
1271 return ancestor.lazyancestors(self.parentrevs, revs, stoprev=stoprev,
18090
9abc55ef85b5 revlog: move ancestor generation out to a new class
Siddharth Agarwal <sid0@fb.com>
parents: 18083
diff changeset
1272 inclusive=inclusive)
6872
c7cc40fd74f6 Add ancestors and descendants to revlog
Stefano Tortarolo <stefano.tortarolo@gmail.com>
parents: 6750
diff changeset
1273
16867
1093ad1e8903 revlog: descendants(*revs) becomes descendants(revs) (API)
Bryan O'Sullivan <bryano@fb.com>
parents: 16866
diff changeset
1274 def descendants(self, revs):
10047
27267b1f68b4 revlog: rewrite several method docstrings
Greg Ward <greg-hg@gerg.ca>
parents: 9679
diff changeset
1275 """Generate the descendants of 'revs' in revision order.
27267b1f68b4 revlog: rewrite several method docstrings
Greg Ward <greg-hg@gerg.ca>
parents: 9679
diff changeset
1276
27267b1f68b4 revlog: rewrite several method docstrings
Greg Ward <greg-hg@gerg.ca>
parents: 9679
diff changeset
1277 Yield a sequence of revision numbers starting with a child of
27267b1f68b4 revlog: rewrite several method docstrings
Greg Ward <greg-hg@gerg.ca>
parents: 9679
diff changeset
1278 some rev in revs, i.e., each revision is *not* considered a
27267b1f68b4 revlog: rewrite several method docstrings
Greg Ward <greg-hg@gerg.ca>
parents: 9679
diff changeset
1279 descendant of itself. Results are ordered by revision number (a
27267b1f68b4 revlog: rewrite several method docstrings
Greg Ward <greg-hg@gerg.ca>
parents: 9679
diff changeset
1280 topological sort)."""
12950
2405b4a5964a revlog: fix descendants() if nullrev is in revs
Nicolas Dumazet <nicdumz.commits@gmail.com>
parents: 12949
diff changeset
1281 first = min(revs)
2405b4a5964a revlog: fix descendants() if nullrev is in revs
Nicolas Dumazet <nicdumz.commits@gmail.com>
parents: 12949
diff changeset
1282 if first == nullrev:
2405b4a5964a revlog: fix descendants() if nullrev is in revs
Nicolas Dumazet <nicdumz.commits@gmail.com>
parents: 12949
diff changeset
1283 for i in self:
2405b4a5964a revlog: fix descendants() if nullrev is in revs
Nicolas Dumazet <nicdumz.commits@gmail.com>
parents: 12949
diff changeset
1284 yield i
2405b4a5964a revlog: fix descendants() if nullrev is in revs
Nicolas Dumazet <nicdumz.commits@gmail.com>
parents: 12949
diff changeset
1285 return
2405b4a5964a revlog: fix descendants() if nullrev is in revs
Nicolas Dumazet <nicdumz.commits@gmail.com>
parents: 12949
diff changeset
1286
8150
bbc24c0753a0 util: use built-in set and frozenset
Martin Geisler <mg@lazybytes.net>
parents: 8073
diff changeset
1287 seen = set(revs)
17672
474047947b8f clfilter: make the revlog class responsible of all its iteration
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17537
diff changeset
1288 for i in self.revs(start=first + 1):
6872
c7cc40fd74f6 Add ancestors and descendants to revlog
Stefano Tortarolo <stefano.tortarolo@gmail.com>
parents: 6750
diff changeset
1289 for x in self.parentrevs(i):
c7cc40fd74f6 Add ancestors and descendants to revlog
Stefano Tortarolo <stefano.tortarolo@gmail.com>
parents: 6750
diff changeset
1290 if x != nullrev and x in seen:
c7cc40fd74f6 Add ancestors and descendants to revlog
Stefano Tortarolo <stefano.tortarolo@gmail.com>
parents: 6750
diff changeset
1291 seen.add(i)
c7cc40fd74f6 Add ancestors and descendants to revlog
Stefano Tortarolo <stefano.tortarolo@gmail.com>
parents: 6750
diff changeset
1292 yield i
c7cc40fd74f6 Add ancestors and descendants to revlog
Stefano Tortarolo <stefano.tortarolo@gmail.com>
parents: 6750
diff changeset
1293 break
c7cc40fd74f6 Add ancestors and descendants to revlog
Stefano Tortarolo <stefano.tortarolo@gmail.com>
parents: 6750
diff changeset
1294
13741
b51bf961b3cb wireproto: add getbundle() function
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13400
diff changeset
1295 def findcommonmissing(self, common=None, heads=None):
b51bf961b3cb wireproto: add getbundle() function
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13400
diff changeset
1296 """Return a tuple of the ancestors of common and the ancestors of heads
15835
fa15869bf95c revlog: improve docstring for findcommonmissing
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 15827
diff changeset
1297 that are not ancestors of common. In revset terminology, we return the
fa15869bf95c revlog: improve docstring for findcommonmissing
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 15827
diff changeset
1298 tuple:
10047
27267b1f68b4 revlog: rewrite several method docstrings
Greg Ward <greg-hg@gerg.ca>
parents: 9679
diff changeset
1299
15835
fa15869bf95c revlog: improve docstring for findcommonmissing
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 15827
diff changeset
1300 ::common, (::heads) - (::common)
7233
9f0e52e1df77 fix pull racing with push/commit (issue1320)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 7109
diff changeset
1301
10047
27267b1f68b4 revlog: rewrite several method docstrings
Greg Ward <greg-hg@gerg.ca>
parents: 9679
diff changeset
1302 The list is sorted by revision number, meaning it is
27267b1f68b4 revlog: rewrite several method docstrings
Greg Ward <greg-hg@gerg.ca>
parents: 9679
diff changeset
1303 topologically sorted.
27267b1f68b4 revlog: rewrite several method docstrings
Greg Ward <greg-hg@gerg.ca>
parents: 9679
diff changeset
1304
27267b1f68b4 revlog: rewrite several method docstrings
Greg Ward <greg-hg@gerg.ca>
parents: 9679
diff changeset
1305 'heads' and 'common' are both lists of node IDs. If heads is
27267b1f68b4 revlog: rewrite several method docstrings
Greg Ward <greg-hg@gerg.ca>
parents: 9679
diff changeset
1306 not supplied, uses all of the revlog's heads. If common is not
27267b1f68b4 revlog: rewrite several method docstrings
Greg Ward <greg-hg@gerg.ca>
parents: 9679
diff changeset
1307 supplied, uses nullid."""
7233
9f0e52e1df77 fix pull racing with push/commit (issue1320)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 7109
diff changeset
1308 if common is None:
9f0e52e1df77 fix pull racing with push/commit (issue1320)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 7109
diff changeset
1309 common = [nullid]
9f0e52e1df77 fix pull racing with push/commit (issue1320)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 7109
diff changeset
1310 if heads is None:
9f0e52e1df77 fix pull racing with push/commit (issue1320)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 7109
diff changeset
1311 heads = self.heads()
9f0e52e1df77 fix pull racing with push/commit (issue1320)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 7109
diff changeset
1312
9f0e52e1df77 fix pull racing with push/commit (issue1320)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 7109
diff changeset
1313 common = [self.rev(n) for n in common]
9f0e52e1df77 fix pull racing with push/commit (issue1320)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 7109
diff changeset
1314 heads = [self.rev(n) for n in heads]
9f0e52e1df77 fix pull racing with push/commit (issue1320)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 7109
diff changeset
1315
9f0e52e1df77 fix pull racing with push/commit (issue1320)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 7109
diff changeset
1316 # we want the ancestors, but inclusive
20073
eeba4eaf0716 revlog: return lazy set from findcommonmissing
Durham Goode <durham@fb.com>
parents: 19776
diff changeset
1317 class lazyset(object):
eeba4eaf0716 revlog: return lazy set from findcommonmissing
Durham Goode <durham@fb.com>
parents: 19776
diff changeset
1318 def __init__(self, lazyvalues):
eeba4eaf0716 revlog: return lazy set from findcommonmissing
Durham Goode <durham@fb.com>
parents: 19776
diff changeset
1319 self.addedvalues = set()
eeba4eaf0716 revlog: return lazy set from findcommonmissing
Durham Goode <durham@fb.com>
parents: 19776
diff changeset
1320 self.lazyvalues = lazyvalues
eeba4eaf0716 revlog: return lazy set from findcommonmissing
Durham Goode <durham@fb.com>
parents: 19776
diff changeset
1321
eeba4eaf0716 revlog: return lazy set from findcommonmissing
Durham Goode <durham@fb.com>
parents: 19776
diff changeset
1322 def __contains__(self, value):
eeba4eaf0716 revlog: return lazy set from findcommonmissing
Durham Goode <durham@fb.com>
parents: 19776
diff changeset
1323 return value in self.addedvalues or value in self.lazyvalues
eeba4eaf0716 revlog: return lazy set from findcommonmissing
Durham Goode <durham@fb.com>
parents: 19776
diff changeset
1324
eeba4eaf0716 revlog: return lazy set from findcommonmissing
Durham Goode <durham@fb.com>
parents: 19776
diff changeset
1325 def __iter__(self):
eeba4eaf0716 revlog: return lazy set from findcommonmissing
Durham Goode <durham@fb.com>
parents: 19776
diff changeset
1326 added = self.addedvalues
eeba4eaf0716 revlog: return lazy set from findcommonmissing
Durham Goode <durham@fb.com>
parents: 19776
diff changeset
1327 for r in added:
eeba4eaf0716 revlog: return lazy set from findcommonmissing
Durham Goode <durham@fb.com>
parents: 19776
diff changeset
1328 yield r
eeba4eaf0716 revlog: return lazy set from findcommonmissing
Durham Goode <durham@fb.com>
parents: 19776
diff changeset
1329 for r in self.lazyvalues:
eeba4eaf0716 revlog: return lazy set from findcommonmissing
Durham Goode <durham@fb.com>
parents: 19776
diff changeset
1330 if not r in added:
eeba4eaf0716 revlog: return lazy set from findcommonmissing
Durham Goode <durham@fb.com>
parents: 19776
diff changeset
1331 yield r
eeba4eaf0716 revlog: return lazy set from findcommonmissing
Durham Goode <durham@fb.com>
parents: 19776
diff changeset
1332
eeba4eaf0716 revlog: return lazy set from findcommonmissing
Durham Goode <durham@fb.com>
parents: 19776
diff changeset
1333 def add(self, value):
eeba4eaf0716 revlog: return lazy set from findcommonmissing
Durham Goode <durham@fb.com>
parents: 19776
diff changeset
1334 self.addedvalues.add(value)
eeba4eaf0716 revlog: return lazy set from findcommonmissing
Durham Goode <durham@fb.com>
parents: 19776
diff changeset
1335
eeba4eaf0716 revlog: return lazy set from findcommonmissing
Durham Goode <durham@fb.com>
parents: 19776
diff changeset
1336 def update(self, values):
eeba4eaf0716 revlog: return lazy set from findcommonmissing
Durham Goode <durham@fb.com>
parents: 19776
diff changeset
1337 self.addedvalues.update(values)
eeba4eaf0716 revlog: return lazy set from findcommonmissing
Durham Goode <durham@fb.com>
parents: 19776
diff changeset
1338
eeba4eaf0716 revlog: return lazy set from findcommonmissing
Durham Goode <durham@fb.com>
parents: 19776
diff changeset
1339 has = lazyset(self.ancestors(common))
8152
08e1baf924ca replace set-like dictionaries with real sets
Martin Geisler <mg@lazybytes.net>
parents: 8150
diff changeset
1340 has.add(nullrev)
08e1baf924ca replace set-like dictionaries with real sets
Martin Geisler <mg@lazybytes.net>
parents: 8150
diff changeset
1341 has.update(common)
7233
9f0e52e1df77 fix pull racing with push/commit (issue1320)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 7109
diff changeset
1342
9f0e52e1df77 fix pull racing with push/commit (issue1320)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 7109
diff changeset
1343 # take all ancestors from heads that aren't in has
8453
d1ca637b0773 revlog.missing(): use sets instead of a dict
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 8391
diff changeset
1344 missing = set()
25113
0ca8410ea345 util: drop alias for collections.deque
Martin von Zweigbergk <martinvonz@google.com>
parents: 24454
diff changeset
1345 visit = collections.deque(r for r in heads if r not in has)
7233
9f0e52e1df77 fix pull racing with push/commit (issue1320)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 7109
diff changeset
1346 while visit:
16803
107a3270a24a cleanup: use the deque type where appropriate
Bryan O'Sullivan <bryano@fb.com>
parents: 16786
diff changeset
1347 r = visit.popleft()
7233
9f0e52e1df77 fix pull racing with push/commit (issue1320)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 7109
diff changeset
1348 if r in missing:
9f0e52e1df77 fix pull racing with push/commit (issue1320)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 7109
diff changeset
1349 continue
9f0e52e1df77 fix pull racing with push/commit (issue1320)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 7109
diff changeset
1350 else:
8453
d1ca637b0773 revlog.missing(): use sets instead of a dict
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 8391
diff changeset
1351 missing.add(r)
7233
9f0e52e1df77 fix pull racing with push/commit (issue1320)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 7109
diff changeset
1352 for p in self.parentrevs(r):
9f0e52e1df77 fix pull racing with push/commit (issue1320)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 7109
diff changeset
1353 if p not in has:
9f0e52e1df77 fix pull racing with push/commit (issue1320)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 7109
diff changeset
1354 visit.append(p)
8453
d1ca637b0773 revlog.missing(): use sets instead of a dict
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 8391
diff changeset
1355 missing = list(missing)
7233
9f0e52e1df77 fix pull racing with push/commit (issue1320)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 7109
diff changeset
1356 missing.sort()
30391
2ded17b64f09 revlog: avoid shadowing several variables using list comprehensions
Augie Fackler <augie@google.com>
parents: 30289
diff changeset
1357 return has, [self.node(miss) for miss in missing]
13741
b51bf961b3cb wireproto: add getbundle() function
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13400
diff changeset
1358
23337
3a8a763f4197 revlog: add a method to get missing revs incrementally
Siddharth Agarwal <sid0@fb.com>
parents: 23328
diff changeset
1359 def incrementalmissingrevs(self, common=None):
3a8a763f4197 revlog: add a method to get missing revs incrementally
Siddharth Agarwal <sid0@fb.com>
parents: 23328
diff changeset
1360 """Return an object that can be used to incrementally compute the
3a8a763f4197 revlog: add a method to get missing revs incrementally
Siddharth Agarwal <sid0@fb.com>
parents: 23328
diff changeset
1361 revision numbers of the ancestors of arbitrary sets that are not
3a8a763f4197 revlog: add a method to get missing revs incrementally
Siddharth Agarwal <sid0@fb.com>
parents: 23328
diff changeset
1362 ancestors of common. This is an ancestor.incrementalmissingancestors
3a8a763f4197 revlog: add a method to get missing revs incrementally
Siddharth Agarwal <sid0@fb.com>
parents: 23328
diff changeset
1363 object.
3a8a763f4197 revlog: add a method to get missing revs incrementally
Siddharth Agarwal <sid0@fb.com>
parents: 23328
diff changeset
1364
3a8a763f4197 revlog: add a method to get missing revs incrementally
Siddharth Agarwal <sid0@fb.com>
parents: 23328
diff changeset
1365 'common' is a list of revision numbers. If common is not supplied, uses
3a8a763f4197 revlog: add a method to get missing revs incrementally
Siddharth Agarwal <sid0@fb.com>
parents: 23328
diff changeset
1366 nullrev.
3a8a763f4197 revlog: add a method to get missing revs incrementally
Siddharth Agarwal <sid0@fb.com>
parents: 23328
diff changeset
1367 """
3a8a763f4197 revlog: add a method to get missing revs incrementally
Siddharth Agarwal <sid0@fb.com>
parents: 23328
diff changeset
1368 if common is None:
3a8a763f4197 revlog: add a method to get missing revs incrementally
Siddharth Agarwal <sid0@fb.com>
parents: 23328
diff changeset
1369 common = [nullrev]
3a8a763f4197 revlog: add a method to get missing revs incrementally
Siddharth Agarwal <sid0@fb.com>
parents: 23328
diff changeset
1370
3a8a763f4197 revlog: add a method to get missing revs incrementally
Siddharth Agarwal <sid0@fb.com>
parents: 23328
diff changeset
1371 return ancestor.incrementalmissingancestors(self.parentrevs, common)
3a8a763f4197 revlog: add a method to get missing revs incrementally
Siddharth Agarwal <sid0@fb.com>
parents: 23328
diff changeset
1372
17972
7ef00d09ef35 revlog: add rev-specific variant of findmissing
Siddharth Agarwal <sid0@fb.com>
parents: 17971
diff changeset
1373 def findmissingrevs(self, common=None, heads=None):
7ef00d09ef35 revlog: add rev-specific variant of findmissing
Siddharth Agarwal <sid0@fb.com>
parents: 17971
diff changeset
1374 """Return the revision numbers of the ancestors of heads that
7ef00d09ef35 revlog: add rev-specific variant of findmissing
Siddharth Agarwal <sid0@fb.com>
parents: 17971
diff changeset
1375 are not ancestors of common.
7ef00d09ef35 revlog: add rev-specific variant of findmissing
Siddharth Agarwal <sid0@fb.com>
parents: 17971
diff changeset
1376
7ef00d09ef35 revlog: add rev-specific variant of findmissing
Siddharth Agarwal <sid0@fb.com>
parents: 17971
diff changeset
1377 More specifically, return a list of revision numbers corresponding to
7ef00d09ef35 revlog: add rev-specific variant of findmissing
Siddharth Agarwal <sid0@fb.com>
parents: 17971
diff changeset
1378 nodes N such that every N satisfies the following constraints:
7ef00d09ef35 revlog: add rev-specific variant of findmissing
Siddharth Agarwal <sid0@fb.com>
parents: 17971
diff changeset
1379
7ef00d09ef35 revlog: add rev-specific variant of findmissing
Siddharth Agarwal <sid0@fb.com>
parents: 17971
diff changeset
1380 1. N is an ancestor of some node in 'heads'
7ef00d09ef35 revlog: add rev-specific variant of findmissing
Siddharth Agarwal <sid0@fb.com>
parents: 17971
diff changeset
1381 2. N is not an ancestor of any node in 'common'
7ef00d09ef35 revlog: add rev-specific variant of findmissing
Siddharth Agarwal <sid0@fb.com>
parents: 17971
diff changeset
1382
7ef00d09ef35 revlog: add rev-specific variant of findmissing
Siddharth Agarwal <sid0@fb.com>
parents: 17971
diff changeset
1383 The list is sorted by revision number, meaning it is
7ef00d09ef35 revlog: add rev-specific variant of findmissing
Siddharth Agarwal <sid0@fb.com>
parents: 17971
diff changeset
1384 topologically sorted.
7ef00d09ef35 revlog: add rev-specific variant of findmissing
Siddharth Agarwal <sid0@fb.com>
parents: 17971
diff changeset
1385
7ef00d09ef35 revlog: add rev-specific variant of findmissing
Siddharth Agarwal <sid0@fb.com>
parents: 17971
diff changeset
1386 'heads' and 'common' are both lists of revision numbers. If heads is
7ef00d09ef35 revlog: add rev-specific variant of findmissing
Siddharth Agarwal <sid0@fb.com>
parents: 17971
diff changeset
1387 not supplied, uses all of the revlog's heads. If common is not
7ef00d09ef35 revlog: add rev-specific variant of findmissing
Siddharth Agarwal <sid0@fb.com>
parents: 17971
diff changeset
1388 supplied, uses nullid."""
7ef00d09ef35 revlog: add rev-specific variant of findmissing
Siddharth Agarwal <sid0@fb.com>
parents: 17971
diff changeset
1389 if common is None:
7ef00d09ef35 revlog: add rev-specific variant of findmissing
Siddharth Agarwal <sid0@fb.com>
parents: 17971
diff changeset
1390 common = [nullrev]
7ef00d09ef35 revlog: add rev-specific variant of findmissing
Siddharth Agarwal <sid0@fb.com>
parents: 17971
diff changeset
1391 if heads is None:
7ef00d09ef35 revlog: add rev-specific variant of findmissing
Siddharth Agarwal <sid0@fb.com>
parents: 17971
diff changeset
1392 heads = self.headrevs()
7ef00d09ef35 revlog: add rev-specific variant of findmissing
Siddharth Agarwal <sid0@fb.com>
parents: 17971
diff changeset
1393
23338
d8f5b2f50f41 revlog: switch findmissing* methods to incrementalmissingrevs
Siddharth Agarwal <sid0@fb.com>
parents: 23337
diff changeset
1394 inc = self.incrementalmissingrevs(common=common)
d8f5b2f50f41 revlog: switch findmissing* methods to incrementalmissingrevs
Siddharth Agarwal <sid0@fb.com>
parents: 23337
diff changeset
1395 return inc.missingancestors(heads)
17972
7ef00d09ef35 revlog: add rev-specific variant of findmissing
Siddharth Agarwal <sid0@fb.com>
parents: 17971
diff changeset
1396
13741
b51bf961b3cb wireproto: add getbundle() function
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13400
diff changeset
1397 def findmissing(self, common=None, heads=None):
b51bf961b3cb wireproto: add getbundle() function
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13400
diff changeset
1398 """Return the ancestors of heads that are not ancestors of common.
b51bf961b3cb wireproto: add getbundle() function
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13400
diff changeset
1399
b51bf961b3cb wireproto: add getbundle() function
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13400
diff changeset
1400 More specifically, return a list of nodes N such that every N
b51bf961b3cb wireproto: add getbundle() function
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13400
diff changeset
1401 satisfies the following constraints:
b51bf961b3cb wireproto: add getbundle() function
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13400
diff changeset
1402
b51bf961b3cb wireproto: add getbundle() function
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13400
diff changeset
1403 1. N is an ancestor of some node in 'heads'
b51bf961b3cb wireproto: add getbundle() function
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13400
diff changeset
1404 2. N is not an ancestor of any node in 'common'
b51bf961b3cb wireproto: add getbundle() function
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13400
diff changeset
1405
b51bf961b3cb wireproto: add getbundle() function
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13400
diff changeset
1406 The list is sorted by revision number, meaning it is
b51bf961b3cb wireproto: add getbundle() function
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13400
diff changeset
1407 topologically sorted.
b51bf961b3cb wireproto: add getbundle() function
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13400
diff changeset
1408
b51bf961b3cb wireproto: add getbundle() function
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13400
diff changeset
1409 'heads' and 'common' are both lists of node IDs. If heads is
b51bf961b3cb wireproto: add getbundle() function
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13400
diff changeset
1410 not supplied, uses all of the revlog's heads. If common is not
b51bf961b3cb wireproto: add getbundle() function
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13400
diff changeset
1411 supplied, uses nullid."""
17971
e1b9a78a7aed revlog: switch findmissing to use ancestor.missingancestors
Siddharth Agarwal <sid0@fb.com>
parents: 17951
diff changeset
1412 if common is None:
e1b9a78a7aed revlog: switch findmissing to use ancestor.missingancestors
Siddharth Agarwal <sid0@fb.com>
parents: 17951
diff changeset
1413 common = [nullid]
e1b9a78a7aed revlog: switch findmissing to use ancestor.missingancestors
Siddharth Agarwal <sid0@fb.com>
parents: 17951
diff changeset
1414 if heads is None:
e1b9a78a7aed revlog: switch findmissing to use ancestor.missingancestors
Siddharth Agarwal <sid0@fb.com>
parents: 17951
diff changeset
1415 heads = self.heads()
e1b9a78a7aed revlog: switch findmissing to use ancestor.missingancestors
Siddharth Agarwal <sid0@fb.com>
parents: 17951
diff changeset
1416
e1b9a78a7aed revlog: switch findmissing to use ancestor.missingancestors
Siddharth Agarwal <sid0@fb.com>
parents: 17951
diff changeset
1417 common = [self.rev(n) for n in common]
e1b9a78a7aed revlog: switch findmissing to use ancestor.missingancestors
Siddharth Agarwal <sid0@fb.com>
parents: 17951
diff changeset
1418 heads = [self.rev(n) for n in heads]
e1b9a78a7aed revlog: switch findmissing to use ancestor.missingancestors
Siddharth Agarwal <sid0@fb.com>
parents: 17951
diff changeset
1419
23338
d8f5b2f50f41 revlog: switch findmissing* methods to incrementalmissingrevs
Siddharth Agarwal <sid0@fb.com>
parents: 23337
diff changeset
1420 inc = self.incrementalmissingrevs(common=common)
d8f5b2f50f41 revlog: switch findmissing* methods to incrementalmissingrevs
Siddharth Agarwal <sid0@fb.com>
parents: 23337
diff changeset
1421 return [self.node(r) for r in inc.missingancestors(heads)]
7233
9f0e52e1df77 fix pull racing with push/commit (issue1320)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 7109
diff changeset
1422
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1423 def nodesbetween(self, roots=None, heads=None):
10047
27267b1f68b4 revlog: rewrite several method docstrings
Greg Ward <greg-hg@gerg.ca>
parents: 9679
diff changeset
1424 """Return a topological path from 'roots' to 'heads'.
27267b1f68b4 revlog: rewrite several method docstrings
Greg Ward <greg-hg@gerg.ca>
parents: 9679
diff changeset
1425
27267b1f68b4 revlog: rewrite several method docstrings
Greg Ward <greg-hg@gerg.ca>
parents: 9679
diff changeset
1426 Return a tuple (nodes, outroots, outheads) where 'nodes' is a
27267b1f68b4 revlog: rewrite several method docstrings
Greg Ward <greg-hg@gerg.ca>
parents: 9679
diff changeset
1427 topologically sorted list of all nodes N that satisfy both of
27267b1f68b4 revlog: rewrite several method docstrings
Greg Ward <greg-hg@gerg.ca>
parents: 9679
diff changeset
1428 these constraints:
27267b1f68b4 revlog: rewrite several method docstrings
Greg Ward <greg-hg@gerg.ca>
parents: 9679
diff changeset
1429
27267b1f68b4 revlog: rewrite several method docstrings
Greg Ward <greg-hg@gerg.ca>
parents: 9679
diff changeset
1430 1. N is a descendant of some node in 'roots'
27267b1f68b4 revlog: rewrite several method docstrings
Greg Ward <greg-hg@gerg.ca>
parents: 9679
diff changeset
1431 2. N is an ancestor of some node in 'heads'
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1432
10047
27267b1f68b4 revlog: rewrite several method docstrings
Greg Ward <greg-hg@gerg.ca>
parents: 9679
diff changeset
1433 Every node is considered to be both a descendant and an ancestor
27267b1f68b4 revlog: rewrite several method docstrings
Greg Ward <greg-hg@gerg.ca>
parents: 9679
diff changeset
1434 of itself, so every reachable node in 'roots' and 'heads' will be
27267b1f68b4 revlog: rewrite several method docstrings
Greg Ward <greg-hg@gerg.ca>
parents: 9679
diff changeset
1435 included in 'nodes'.
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1436
10047
27267b1f68b4 revlog: rewrite several method docstrings
Greg Ward <greg-hg@gerg.ca>
parents: 9679
diff changeset
1437 'outroots' is the list of reachable nodes in 'roots', i.e., the
27267b1f68b4 revlog: rewrite several method docstrings
Greg Ward <greg-hg@gerg.ca>
parents: 9679
diff changeset
1438 subset of 'roots' that is returned in 'nodes'. Likewise,
27267b1f68b4 revlog: rewrite several method docstrings
Greg Ward <greg-hg@gerg.ca>
parents: 9679
diff changeset
1439 'outheads' is the subset of 'heads' that is also in 'nodes'.
27267b1f68b4 revlog: rewrite several method docstrings
Greg Ward <greg-hg@gerg.ca>
parents: 9679
diff changeset
1440
27267b1f68b4 revlog: rewrite several method docstrings
Greg Ward <greg-hg@gerg.ca>
parents: 9679
diff changeset
1441 'roots' and 'heads' are both lists of node IDs. If 'roots' is
27267b1f68b4 revlog: rewrite several method docstrings
Greg Ward <greg-hg@gerg.ca>
parents: 9679
diff changeset
1442 unspecified, uses nullid as the only root. If 'heads' is
27267b1f68b4 revlog: rewrite several method docstrings
Greg Ward <greg-hg@gerg.ca>
parents: 9679
diff changeset
1443 unspecified, uses list of all of the revlog's heads."""
1463
26e73acc0cdf Fix to handle case of empty list for roots or heads in nodesbetween.
Eric Hopper <hopper@omnifarious.org>
parents: 1459
diff changeset
1444 nonodes = ([], [], [])
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1445 if roots is not None:
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1446 roots = list(roots)
1463
26e73acc0cdf Fix to handle case of empty list for roots or heads in nodesbetween.
Eric Hopper <hopper@omnifarious.org>
parents: 1459
diff changeset
1447 if not roots:
26e73acc0cdf Fix to handle case of empty list for roots or heads in nodesbetween.
Eric Hopper <hopper@omnifarious.org>
parents: 1459
diff changeset
1448 return nonodes
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1449 lowestrev = min([self.rev(n) for n in roots])
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1450 else:
14549
48ec0763afbb check-code: catch misspellings of descendant
Matt Mackall <mpm@selenic.com>
parents: 14523
diff changeset
1451 roots = [nullid] # Everybody's a descendant of nullid
3578
3b4e00cba57a Define and use nullrev (revision of nullid) instead of -1.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 3508
diff changeset
1452 lowestrev = nullrev
3b4e00cba57a Define and use nullrev (revision of nullid) instead of -1.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 3508
diff changeset
1453 if (lowestrev == nullrev) and (heads is None):
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1454 # We want _all_ the nodes!
6750
fb42030d79d6 add __len__ and __iter__ methods to repo and revlog
Matt Mackall <mpm@selenic.com>
parents: 6703
diff changeset
1455 return ([self.node(r) for r in self], [nullid], list(self.heads()))
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1456 if heads is None:
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1457 # All nodes are ancestors, so the latest ancestor is the last
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1458 # node.
6750
fb42030d79d6 add __len__ and __iter__ methods to repo and revlog
Matt Mackall <mpm@selenic.com>
parents: 6703
diff changeset
1459 highestrev = len(self) - 1
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1460 # Set ancestors to None to signal that every node is an ancestor.
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1461 ancestors = None
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1462 # Set heads to an empty dictionary for later discovery of heads
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1463 heads = {}
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1464 else:
1463
26e73acc0cdf Fix to handle case of empty list for roots or heads in nodesbetween.
Eric Hopper <hopper@omnifarious.org>
parents: 1459
diff changeset
1465 heads = list(heads)
26e73acc0cdf Fix to handle case of empty list for roots or heads in nodesbetween.
Eric Hopper <hopper@omnifarious.org>
parents: 1459
diff changeset
1466 if not heads:
26e73acc0cdf Fix to handle case of empty list for roots or heads in nodesbetween.
Eric Hopper <hopper@omnifarious.org>
parents: 1459
diff changeset
1467 return nonodes
8464
7af92e70bb25 revlog: use set instead of dict
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 8453
diff changeset
1468 ancestors = set()
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1469 # Turn heads into a dictionary so we can remove 'fake' heads.
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1470 # Also, later we will be using it to filter out the heads we can't
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1471 # find from roots.
14219
c33427080671 revlog: use real Booleans instead of 0/1 in nodesbetween
Martin Geisler <mg@aragost.com>
parents: 14208
diff changeset
1472 heads = dict.fromkeys(heads, False)
3360
ef8307585b41 nodesbetween: fix a bug with duplicate heads
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 3335
diff changeset
1473 # Start at the top and keep marking parents until we're done.
8163
62d7287fe6b0 rebase, revlog: use set(x) instead of set(x.keys())
Martin Geisler <mg@lazybytes.net>
parents: 8153
diff changeset
1474 nodestotag = set(heads)
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1475 # Remember where the top was so we can use it as a limit later.
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1476 highestrev = max([self.rev(n) for n in nodestotag])
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1477 while nodestotag:
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1478 # grab a node to tag
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1479 n = nodestotag.pop()
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1480 # Never tag nullid
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1481 if n == nullid:
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1482 continue
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1483 # A node's revision number represents its place in a
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1484 # topologically sorted list of nodes.
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1485 r = self.rev(n)
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1486 if r >= lowestrev:
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1487 if n not in ancestors:
14549
48ec0763afbb check-code: catch misspellings of descendant
Matt Mackall <mpm@selenic.com>
parents: 14523
diff changeset
1488 # If we are possibly a descendant of one of the roots
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1489 # and we haven't already been marked as an ancestor
8464
7af92e70bb25 revlog: use set instead of dict
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 8453
diff changeset
1490 ancestors.add(n) # Mark as ancestor
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1491 # Add non-nullid parents to list of nodes to tag.
8153
616f20e1004a revlog: let nodestotag be a set instead of a list
Martin Geisler <mg@lazybytes.net>
parents: 8152
diff changeset
1492 nodestotag.update([p for p in self.parents(n) if
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1493 p != nullid])
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1494 elif n in heads: # We've seen it before, is it a fake head?
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1495 # So it is, real heads should not be the ancestors of
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1496 # any other heads.
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1497 heads.pop(n)
1459
106fdec8e1fb Fix small bug in nodesbetween if heads is [nullid].
Eric Hopper <hopper@omnifarious.org>
parents: 1458
diff changeset
1498 if not ancestors:
1463
26e73acc0cdf Fix to handle case of empty list for roots or heads in nodesbetween.
Eric Hopper <hopper@omnifarious.org>
parents: 1459
diff changeset
1499 return nonodes
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1500 # Now that we have our set of ancestors, we want to remove any
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1501 # roots that are not ancestors.
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1502
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1503 # If one of the roots was nullid, everything is included anyway.
3578
3b4e00cba57a Define and use nullrev (revision of nullid) instead of -1.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 3508
diff changeset
1504 if lowestrev > nullrev:
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1505 # But, since we weren't, let's recompute the lowest rev to not
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1506 # include roots that aren't ancestors.
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1507
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1508 # Filter out roots that aren't ancestors of heads
30391
2ded17b64f09 revlog: avoid shadowing several variables using list comprehensions
Augie Fackler <augie@google.com>
parents: 30289
diff changeset
1509 roots = [root for root in roots if root in ancestors]
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1510 # Recompute the lowest revision
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1511 if roots:
30391
2ded17b64f09 revlog: avoid shadowing several variables using list comprehensions
Augie Fackler <augie@google.com>
parents: 30289
diff changeset
1512 lowestrev = min([self.rev(root) for root in roots])
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1513 else:
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1514 # No more roots? Return empty list
1463
26e73acc0cdf Fix to handle case of empty list for roots or heads in nodesbetween.
Eric Hopper <hopper@omnifarious.org>
parents: 1459
diff changeset
1515 return nonodes
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1516 else:
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1517 # We are descending from nullid, and don't need to care about
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1518 # any other roots.
3578
3b4e00cba57a Define and use nullrev (revision of nullid) instead of -1.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 3508
diff changeset
1519 lowestrev = nullrev
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1520 roots = [nullid]
8152
08e1baf924ca replace set-like dictionaries with real sets
Martin Geisler <mg@lazybytes.net>
parents: 8150
diff changeset
1521 # Transform our roots list into a set.
14549
48ec0763afbb check-code: catch misspellings of descendant
Matt Mackall <mpm@selenic.com>
parents: 14523
diff changeset
1522 descendants = set(roots)
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1523 # Also, keep the original roots so we can filter out roots that aren't
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1524 # 'real' roots (i.e. are descended from other roots).
14549
48ec0763afbb check-code: catch misspellings of descendant
Matt Mackall <mpm@selenic.com>
parents: 14523
diff changeset
1525 roots = descendants.copy()
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1526 # Our topologically sorted list of output nodes.
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1527 orderedout = []
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1528 # Don't start at nullid since we don't want nullid in our output list,
17483
fe3b26048140 spelling: descendants
timeless@mozdev.org
parents: 17150
diff changeset
1529 # and if nullid shows up in descendants, empty parents will look like
14549
48ec0763afbb check-code: catch misspellings of descendant
Matt Mackall <mpm@selenic.com>
parents: 14523
diff changeset
1530 # they're descendants.
17672
474047947b8f clfilter: make the revlog class responsible of all its iteration
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17537
diff changeset
1531 for r in self.revs(start=max(lowestrev, 0), stop=highestrev + 1):
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1532 n = self.node(r)
14549
48ec0763afbb check-code: catch misspellings of descendant
Matt Mackall <mpm@selenic.com>
parents: 14523
diff changeset
1533 isdescendant = False
48ec0763afbb check-code: catch misspellings of descendant
Matt Mackall <mpm@selenic.com>
parents: 14523
diff changeset
1534 if lowestrev == nullrev: # Everybody is a descendant of nullid
48ec0763afbb check-code: catch misspellings of descendant
Matt Mackall <mpm@selenic.com>
parents: 14523
diff changeset
1535 isdescendant = True
48ec0763afbb check-code: catch misspellings of descendant
Matt Mackall <mpm@selenic.com>
parents: 14523
diff changeset
1536 elif n in descendants:
48ec0763afbb check-code: catch misspellings of descendant
Matt Mackall <mpm@selenic.com>
parents: 14523
diff changeset
1537 # n is already a descendant
48ec0763afbb check-code: catch misspellings of descendant
Matt Mackall <mpm@selenic.com>
parents: 14523
diff changeset
1538 isdescendant = True
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1539 # This check only needs to be done here because all the roots
14549
48ec0763afbb check-code: catch misspellings of descendant
Matt Mackall <mpm@selenic.com>
parents: 14523
diff changeset
1540 # will start being marked is descendants before the loop.
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1541 if n in roots:
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1542 # If n was a root, check if it's a 'real' root.
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1543 p = tuple(self.parents(n))
14549
48ec0763afbb check-code: catch misspellings of descendant
Matt Mackall <mpm@selenic.com>
parents: 14523
diff changeset
1544 # If any of its parents are descendants, it's not a root.
48ec0763afbb check-code: catch misspellings of descendant
Matt Mackall <mpm@selenic.com>
parents: 14523
diff changeset
1545 if (p[0] in descendants) or (p[1] in descendants):
8152
08e1baf924ca replace set-like dictionaries with real sets
Martin Geisler <mg@lazybytes.net>
parents: 8150
diff changeset
1546 roots.remove(n)
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1547 else:
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1548 p = tuple(self.parents(n))
14549
48ec0763afbb check-code: catch misspellings of descendant
Matt Mackall <mpm@selenic.com>
parents: 14523
diff changeset
1549 # A node is a descendant if either of its parents are
48ec0763afbb check-code: catch misspellings of descendant
Matt Mackall <mpm@selenic.com>
parents: 14523
diff changeset
1550 # descendants. (We seeded the dependents list with the roots
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1551 # up there, remember?)
14549
48ec0763afbb check-code: catch misspellings of descendant
Matt Mackall <mpm@selenic.com>
parents: 14523
diff changeset
1552 if (p[0] in descendants) or (p[1] in descendants):
48ec0763afbb check-code: catch misspellings of descendant
Matt Mackall <mpm@selenic.com>
parents: 14523
diff changeset
1553 descendants.add(n)
48ec0763afbb check-code: catch misspellings of descendant
Matt Mackall <mpm@selenic.com>
parents: 14523
diff changeset
1554 isdescendant = True
48ec0763afbb check-code: catch misspellings of descendant
Matt Mackall <mpm@selenic.com>
parents: 14523
diff changeset
1555 if isdescendant and ((ancestors is None) or (n in ancestors)):
48ec0763afbb check-code: catch misspellings of descendant
Matt Mackall <mpm@selenic.com>
parents: 14523
diff changeset
1556 # Only include nodes that are both descendants and ancestors.
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1557 orderedout.append(n)
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1558 if (ancestors is not None) and (n in heads):
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1559 # We're trying to figure out which heads are reachable
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1560 # from roots.
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1561 # Mark this head as having been reached
14219
c33427080671 revlog: use real Booleans instead of 0/1 in nodesbetween
Martin Geisler <mg@aragost.com>
parents: 14208
diff changeset
1562 heads[n] = True
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1563 elif ancestors is None:
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1564 # Otherwise, we're trying to discover the heads.
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1565 # Assume this is a head because if it isn't, the next step
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1566 # will eventually remove it.
14219
c33427080671 revlog: use real Booleans instead of 0/1 in nodesbetween
Martin Geisler <mg@aragost.com>
parents: 14208
diff changeset
1567 heads[n] = True
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1568 # But, obviously its parents aren't.
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1569 for p in self.parents(n):
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1570 heads.pop(p, None)
30391
2ded17b64f09 revlog: avoid shadowing several variables using list comprehensions
Augie Fackler <augie@google.com>
parents: 30289
diff changeset
1571 heads = [head for head, flag in heads.iteritems() if flag]
8152
08e1baf924ca replace set-like dictionaries with real sets
Martin Geisler <mg@lazybytes.net>
parents: 8150
diff changeset
1572 roots = list(roots)
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1573 assert orderedout
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1574 assert roots
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1575 assert heads
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1576 return (orderedout, roots, heads)
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
1577
14164
cb98fed52495 discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14144
diff changeset
1578 def headrevs(self):
16786
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16762
diff changeset
1579 try:
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16762
diff changeset
1580 return self.index.headrevs()
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16762
diff changeset
1581 except AttributeError:
17674
e69274f8d444 clfilter: split `revlog.headrevs` C call from python code
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17673
diff changeset
1582 return self._headrevs()
e69274f8d444 clfilter: split `revlog.headrevs` C call from python code
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17673
diff changeset
1583
24444
27e3ba73fbb1 phase: default to C implementation for phase computation
Laurent Charignon <lcharignon@fb.com>
parents: 24255
diff changeset
1584 def computephases(self, roots):
25361
1635579f9baf phases: fix bug where native phase computation wasn't called
Laurent Charignon <lcharignon@fb.com>
parents: 25113
diff changeset
1585 return self.index.computephasesmapsets(roots)
24444
27e3ba73fbb1 phase: default to C implementation for phase computation
Laurent Charignon <lcharignon@fb.com>
parents: 24255
diff changeset
1586
17674
e69274f8d444 clfilter: split `revlog.headrevs` C call from python code
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17673
diff changeset
1587 def _headrevs(self):
14164
cb98fed52495 discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14144
diff changeset
1588 count = len(self)
cb98fed52495 discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14144
diff changeset
1589 if not count:
cb98fed52495 discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14144
diff changeset
1590 return [nullrev]
17673
d686c6876ef6 clfilter: handle non contiguous iteration in `revlov.headrevs`
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17672
diff changeset
1591 # we won't iter over filtered rev so nobody is a head at start
d686c6876ef6 clfilter: handle non contiguous iteration in `revlov.headrevs`
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17672
diff changeset
1592 ishead = [0] * (count + 1)
14164
cb98fed52495 discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14144
diff changeset
1593 index = self.index
17672
474047947b8f clfilter: make the revlog class responsible of all its iteration
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17537
diff changeset
1594 for r in self:
17673
d686c6876ef6 clfilter: handle non contiguous iteration in `revlov.headrevs`
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17672
diff changeset
1595 ishead[r] = 1 # I may be an head
14164
cb98fed52495 discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14144
diff changeset
1596 e = index[r]
17673
d686c6876ef6 clfilter: handle non contiguous iteration in `revlov.headrevs`
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17672
diff changeset
1597 ishead[e[5]] = ishead[e[6]] = 0 # my parent are not
d686c6876ef6 clfilter: handle non contiguous iteration in `revlov.headrevs`
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17672
diff changeset
1598 return [r for r, val in enumerate(ishead) if val]
14164
cb98fed52495 discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14144
diff changeset
1599
3923
27230c29bfec fix calculation of new heads added during push with -r
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 3755
diff changeset
1600 def heads(self, start=None, stop=None):
1550
ccb9b62de892 add a -r/--rev option to heads to show only heads descendant from rev
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1535
diff changeset
1601 """return the list of all nodes that have no children
1551
e793cbc8be00 Fixes to "hg heads -r FOO":
Thomas Arendsen Hein <thomas@intevation.de>
parents: 1550
diff changeset
1602
e793cbc8be00 Fixes to "hg heads -r FOO":
Thomas Arendsen Hein <thomas@intevation.de>
parents: 1550
diff changeset
1603 if start is specified, only heads that are descendants of
e793cbc8be00 Fixes to "hg heads -r FOO":
Thomas Arendsen Hein <thomas@intevation.de>
parents: 1550
diff changeset
1604 start will be returned
3923
27230c29bfec fix calculation of new heads added during push with -r
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 3755
diff changeset
1605 if stop is specified, it will consider all the revs from stop
27230c29bfec fix calculation of new heads added during push with -r
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 3755
diff changeset
1606 as if they had no children
1551
e793cbc8be00 Fixes to "hg heads -r FOO":
Thomas Arendsen Hein <thomas@intevation.de>
parents: 1550
diff changeset
1607 """
4991
9c8c42bcf17a revlog: implement a fast path for heads
Matt Mackall <mpm@selenic.com>
parents: 4990
diff changeset
1608 if start is None and stop is None:
14164
cb98fed52495 discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14144
diff changeset
1609 if not len(self):
4991
9c8c42bcf17a revlog: implement a fast path for heads
Matt Mackall <mpm@selenic.com>
parents: 4990
diff changeset
1610 return [nullid]
14164
cb98fed52495 discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14144
diff changeset
1611 return [self.node(r) for r in self.headrevs()]
4991
9c8c42bcf17a revlog: implement a fast path for heads
Matt Mackall <mpm@selenic.com>
parents: 4990
diff changeset
1612
1551
e793cbc8be00 Fixes to "hg heads -r FOO":
Thomas Arendsen Hein <thomas@intevation.de>
parents: 1550
diff changeset
1613 if start is None:
e793cbc8be00 Fixes to "hg heads -r FOO":
Thomas Arendsen Hein <thomas@intevation.de>
parents: 1550
diff changeset
1614 start = nullid
3923
27230c29bfec fix calculation of new heads added during push with -r
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 3755
diff changeset
1615 if stop is None:
27230c29bfec fix calculation of new heads added during push with -r
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 3755
diff changeset
1616 stop = []
8152
08e1baf924ca replace set-like dictionaries with real sets
Martin Geisler <mg@lazybytes.net>
parents: 8150
diff changeset
1617 stoprevs = set([self.rev(n) for n in stop])
1550
ccb9b62de892 add a -r/--rev option to heads to show only heads descendant from rev
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1535
diff changeset
1618 startrev = self.rev(start)
32291
bd872f64a8ba cleanup: use set literals
Martin von Zweigbergk <martinvonz@google.com>
parents: 32286
diff changeset
1619 reachable = {startrev}
bd872f64a8ba cleanup: use set literals
Martin von Zweigbergk <martinvonz@google.com>
parents: 32286
diff changeset
1620 heads = {startrev}
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
1621
2490
6ff82ec1f4b8 Change revlog.heads to walk the revision graph using revision numbers
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 2489
diff changeset
1622 parentrevs = self.parentrevs
17672
474047947b8f clfilter: make the revlog class responsible of all its iteration
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17537
diff changeset
1623 for r in self.revs(start=startrev + 1):
2490
6ff82ec1f4b8 Change revlog.heads to walk the revision graph using revision numbers
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 2489
diff changeset
1624 for p in parentrevs(r):
6ff82ec1f4b8 Change revlog.heads to walk the revision graph using revision numbers
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 2489
diff changeset
1625 if p in reachable:
3923
27230c29bfec fix calculation of new heads added during push with -r
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 3755
diff changeset
1626 if r not in stoprevs:
8464
7af92e70bb25 revlog: use set instead of dict
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 8453
diff changeset
1627 reachable.add(r)
7af92e70bb25 revlog: use set instead of dict
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 8453
diff changeset
1628 heads.add(r)
3923
27230c29bfec fix calculation of new heads added during push with -r
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 3755
diff changeset
1629 if p in heads and p not in stoprevs:
8464
7af92e70bb25 revlog: use set instead of dict
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 8453
diff changeset
1630 heads.remove(p)
3923
27230c29bfec fix calculation of new heads added during push with -r
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 3755
diff changeset
1631
2490
6ff82ec1f4b8 Change revlog.heads to walk the revision graph using revision numbers
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 2489
diff changeset
1632 return [self.node(r) for r in heads]
370
c90385d82aec revlog: add a children function
mpm@selenic.com
parents: 330
diff changeset
1633
c90385d82aec revlog: add a children function
mpm@selenic.com
parents: 330
diff changeset
1634 def children(self, node):
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
1635 """find the children of a given node"""
370
c90385d82aec revlog: add a children function
mpm@selenic.com
parents: 330
diff changeset
1636 c = []
c90385d82aec revlog: add a children function
mpm@selenic.com
parents: 330
diff changeset
1637 p = self.rev(node)
17672
474047947b8f clfilter: make the revlog class responsible of all its iteration
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17537
diff changeset
1638 for r in self.revs(start=p + 1):
4746
62c56d8f368b Fix revlog.children so the real children of the null revision can be calculated.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4635
diff changeset
1639 prevs = [pr for pr in self.parentrevs(r) if pr != nullrev]
62c56d8f368b Fix revlog.children so the real children of the null revision can be calculated.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4635
diff changeset
1640 if prevs:
62c56d8f368b Fix revlog.children so the real children of the null revision can be calculated.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4635
diff changeset
1641 for pr in prevs:
62c56d8f368b Fix revlog.children so the real children of the null revision can be calculated.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4635
diff changeset
1642 if pr == p:
62c56d8f368b Fix revlog.children so the real children of the null revision can be calculated.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4635
diff changeset
1643 c.append(self.node(r))
62c56d8f368b Fix revlog.children so the real children of the null revision can be calculated.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4635
diff changeset
1644 elif p == nullrev:
62c56d8f368b Fix revlog.children so the real children of the null revision can be calculated.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4635
diff changeset
1645 c.append(self.node(r))
370
c90385d82aec revlog: add a children function
mpm@selenic.com
parents: 330
diff changeset
1646 return c
515
03f27b1381f9 Whitespace cleanups
mpm@selenic.com
parents: 484
diff changeset
1647
21104
40ace21cb3a1 revlog: introduce commonancestorsheads method
Mads Kiilerich <madski@unity3d.com>
parents: 20965
diff changeset
1648 def commonancestorsheads(self, a, b):
40ace21cb3a1 revlog: introduce commonancestorsheads method
Mads Kiilerich <madski@unity3d.com>
parents: 20965
diff changeset
1649 """calculate all the heads of the common ancestors of nodes a and b"""
40ace21cb3a1 revlog: introduce commonancestorsheads method
Mads Kiilerich <madski@unity3d.com>
parents: 20965
diff changeset
1650 a, b = self.rev(a), self.rev(b)
38512
99f864b34451 revlog: refactor out the rev-oriented part of commonancestorheads
Boris Feld <boris.feld@octobus.net>
parents: 38511
diff changeset
1651 ancs = self._commonancestorsheads(a, b)
99f864b34451 revlog: refactor out the rev-oriented part of commonancestorheads
Boris Feld <boris.feld@octobus.net>
parents: 38511
diff changeset
1652 return pycompat.maplist(self.node, ancs)
99f864b34451 revlog: refactor out the rev-oriented part of commonancestorheads
Boris Feld <boris.feld@octobus.net>
parents: 38511
diff changeset
1653
99f864b34451 revlog: refactor out the rev-oriented part of commonancestorheads
Boris Feld <boris.feld@octobus.net>
parents: 38511
diff changeset
1654 def _commonancestorsheads(self, *revs):
99f864b34451 revlog: refactor out the rev-oriented part of commonancestorheads
Boris Feld <boris.feld@octobus.net>
parents: 38511
diff changeset
1655 """calculate all the heads of the common ancestors of revs"""
21104
40ace21cb3a1 revlog: introduce commonancestorsheads method
Mads Kiilerich <madski@unity3d.com>
parents: 20965
diff changeset
1656 try:
38512
99f864b34451 revlog: refactor out the rev-oriented part of commonancestorheads
Boris Feld <boris.feld@octobus.net>
parents: 38511
diff changeset
1657 ancs = self.index.commonancestorsheads(*revs)
21104
40ace21cb3a1 revlog: introduce commonancestorsheads method
Mads Kiilerich <madski@unity3d.com>
parents: 20965
diff changeset
1658 except (AttributeError, OverflowError): # C implementation failed
38512
99f864b34451 revlog: refactor out the rev-oriented part of commonancestorheads
Boris Feld <boris.feld@octobus.net>
parents: 38511
diff changeset
1659 ancs = ancestor.commonancestorsheads(self.parentrevs, *revs)
99f864b34451 revlog: refactor out the rev-oriented part of commonancestorheads
Boris Feld <boris.feld@octobus.net>
parents: 38511
diff changeset
1660 return ancs
21104
40ace21cb3a1 revlog: introduce commonancestorsheads method
Mads Kiilerich <madski@unity3d.com>
parents: 20965
diff changeset
1661
22381
392ae5cb8d62 revlog: introduce isancestor method for efficiently determining node lineage
Mads Kiilerich <madski@unity3d.com>
parents: 22282
diff changeset
1662 def isancestor(self, a, b):
38664
160da69ba1bf revlog: replace descendant(b, a) by isdescendantrev(a, b) (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 38663
diff changeset
1663 """return True if node a is an ancestor of node b
160da69ba1bf revlog: replace descendant(b, a) by isdescendantrev(a, b) (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 38663
diff changeset
1664
160da69ba1bf revlog: replace descendant(b, a) by isdescendantrev(a, b) (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 38663
diff changeset
1665 A revision is considered an ancestor of itself."""
38514
cc3543c87de5 revlog: reuse 'descendant' implemention in 'isancestor'
Boris Feld <boris.feld@octobus.net>
parents: 38513
diff changeset
1666 a, b = self.rev(a), self.rev(b)
38666
a06b2b032557 revlog: introduce a isancestorrev() and use it in rebase
Martin von Zweigbergk <martinvonz@google.com>
parents: 38665
diff changeset
1667 return self.isancestorrev(a, b)
a06b2b032557 revlog: introduce a isancestorrev() and use it in rebase
Martin von Zweigbergk <martinvonz@google.com>
parents: 38665
diff changeset
1668
38710
4ac3c2078567 revlog: reintroduce `revlog.descendant` as deprecated
Boris Feld <boris.feld@octobus.net>
parents: 38668
diff changeset
1669 def descendant(self, a, b):
4ac3c2078567 revlog: reintroduce `revlog.descendant` as deprecated
Boris Feld <boris.feld@octobus.net>
parents: 38668
diff changeset
1670 msg = (b'revlog.descendant is deprecated, use revlog.isancestorrev')
4ac3c2078567 revlog: reintroduce `revlog.descendant` as deprecated
Boris Feld <boris.feld@octobus.net>
parents: 38668
diff changeset
1671 self._repo.ui.deprecwarn(msg, b'4.7')
4ac3c2078567 revlog: reintroduce `revlog.descendant` as deprecated
Boris Feld <boris.feld@octobus.net>
parents: 38668
diff changeset
1672 return self.isancestorrev(a, b)
4ac3c2078567 revlog: reintroduce `revlog.descendant` as deprecated
Boris Feld <boris.feld@octobus.net>
parents: 38668
diff changeset
1673
38666
a06b2b032557 revlog: introduce a isancestorrev() and use it in rebase
Martin von Zweigbergk <martinvonz@google.com>
parents: 38665
diff changeset
1674 def isancestorrev(self, a, b):
a06b2b032557 revlog: introduce a isancestorrev() and use it in rebase
Martin von Zweigbergk <martinvonz@google.com>
parents: 38665
diff changeset
1675 """return True if revision a is an ancestor of revision b
a06b2b032557 revlog: introduce a isancestorrev() and use it in rebase
Martin von Zweigbergk <martinvonz@google.com>
parents: 38665
diff changeset
1676
38668
21846c94e605 revlog: delete isdescendantrev() in favor of isancestorrev()
Martin von Zweigbergk <martinvonz@google.com>
parents: 38666
diff changeset
1677 A revision is considered an ancestor of itself.
21846c94e605 revlog: delete isdescendantrev() in favor of isancestorrev()
Martin von Zweigbergk <martinvonz@google.com>
parents: 38666
diff changeset
1678
21846c94e605 revlog: delete isdescendantrev() in favor of isancestorrev()
Martin von Zweigbergk <martinvonz@google.com>
parents: 38666
diff changeset
1679 The implementation of this is trivial but the use of
21846c94e605 revlog: delete isdescendantrev() in favor of isancestorrev()
Martin von Zweigbergk <martinvonz@google.com>
parents: 38666
diff changeset
1680 commonancestorsheads is not."""
21846c94e605 revlog: delete isdescendantrev() in favor of isancestorrev()
Martin von Zweigbergk <martinvonz@google.com>
parents: 38666
diff changeset
1681 if a == nullrev:
21846c94e605 revlog: delete isdescendantrev() in favor of isancestorrev()
Martin von Zweigbergk <martinvonz@google.com>
parents: 38666
diff changeset
1682 return True
21846c94e605 revlog: delete isdescendantrev() in favor of isancestorrev()
Martin von Zweigbergk <martinvonz@google.com>
parents: 38666
diff changeset
1683 elif a == b:
21846c94e605 revlog: delete isdescendantrev() in favor of isancestorrev()
Martin von Zweigbergk <martinvonz@google.com>
parents: 38666
diff changeset
1684 return True
21846c94e605 revlog: delete isdescendantrev() in favor of isancestorrev()
Martin von Zweigbergk <martinvonz@google.com>
parents: 38666
diff changeset
1685 elif a > b:
21846c94e605 revlog: delete isdescendantrev() in favor of isancestorrev()
Martin von Zweigbergk <martinvonz@google.com>
parents: 38666
diff changeset
1686 return False
21846c94e605 revlog: delete isdescendantrev() in favor of isancestorrev()
Martin von Zweigbergk <martinvonz@google.com>
parents: 38666
diff changeset
1687 return a in self._commonancestorsheads(a, b)
22381
392ae5cb8d62 revlog: introduce isancestor method for efficiently determining node lineage
Mads Kiilerich <madski@unity3d.com>
parents: 22282
diff changeset
1688
21107
4a6c8b6b10d3 revlog: backout 514d32de6646 - commonancestors
Mads Kiilerich <madski@unity3d.com>
parents: 21104
diff changeset
1689 def ancestor(self, a, b):
22389
94f77624dbb5 comments: describe ancestor consistently - avoid 'least common ancestor'
Mads Kiilerich <madski@unity3d.com>
parents: 22381
diff changeset
1690 """calculate the "best" common ancestor of nodes a and b"""
21107
4a6c8b6b10d3 revlog: backout 514d32de6646 - commonancestors
Mads Kiilerich <madski@unity3d.com>
parents: 21104
diff changeset
1691
10897
adb6a291bbdb revlog: put graph related functions together
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 10404
diff changeset
1692 a, b = self.rev(a), self.rev(b)
18988
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18987
diff changeset
1693 try:
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18987
diff changeset
1694 ancs = self.index.ancestors(a, b)
21107
4a6c8b6b10d3 revlog: backout 514d32de6646 - commonancestors
Mads Kiilerich <madski@unity3d.com>
parents: 21104
diff changeset
1695 except (AttributeError, OverflowError):
18988
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18987
diff changeset
1696 ancs = ancestor.ancestors(self.parentrevs, a, b)
18987
3605d4e7e618 revlog: choose a consistent ancestor when there's a tie
Bryan O'Sullivan <bryano@fb.com>
parents: 18986
diff changeset
1697 if ancs:
3605d4e7e618 revlog: choose a consistent ancestor when there's a tie
Bryan O'Sullivan <bryano@fb.com>
parents: 18986
diff changeset
1698 # choose a consistent winner when there's a tie
21107
4a6c8b6b10d3 revlog: backout 514d32de6646 - commonancestors
Mads Kiilerich <madski@unity3d.com>
parents: 21104
diff changeset
1699 return min(map(self.node, ancs))
18987
3605d4e7e618 revlog: choose a consistent ancestor when there's a tie
Bryan O'Sullivan <bryano@fb.com>
parents: 18986
diff changeset
1700 return nullid
10897
adb6a291bbdb revlog: put graph related functions together
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 10404
diff changeset
1701
3453
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
1702 def _match(self, id):
16762
93f8b9565257 revlog: don't handle long for revision matching
Matt Mackall <mpm@selenic.com>
parents: 16686
diff changeset
1703 if isinstance(id, int):
3156
d01e4cb2f5f2 cleanups in revlog.lookup
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 3139
diff changeset
1704 # rev
2641
156fb1feab62 lookup should allow -1 to represent nullid (if passed an int as arg)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 2600
diff changeset
1705 return self.node(id)
3438
b17f9d3eda74 revlog.lookup tweaks
Matt Mackall <mpm@selenic.com>
parents: 3390
diff changeset
1706 if len(id) == 20:
b17f9d3eda74 revlog.lookup tweaks
Matt Mackall <mpm@selenic.com>
parents: 3390
diff changeset
1707 # possibly a binary node
b17f9d3eda74 revlog.lookup tweaks
Matt Mackall <mpm@selenic.com>
parents: 3390
diff changeset
1708 # odds of a binary node being all hex in ASCII are 1 in 10**25
b17f9d3eda74 revlog.lookup tweaks
Matt Mackall <mpm@selenic.com>
parents: 3390
diff changeset
1709 try:
b17f9d3eda74 revlog.lookup tweaks
Matt Mackall <mpm@selenic.com>
parents: 3390
diff changeset
1710 node = id
7874
d812029cda85 cleanup: drop variables for unused return values
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 7873
diff changeset
1711 self.rev(node) # quick search the index
3438
b17f9d3eda74 revlog.lookup tweaks
Matt Mackall <mpm@selenic.com>
parents: 3390
diff changeset
1712 return node
3930
01d98d68d697 Add revlog.LookupError exception, and use it instead of RevlogError.
Brendan Cully <brendan@kublai.com>
parents: 3928
diff changeset
1713 except LookupError:
3438
b17f9d3eda74 revlog.lookup tweaks
Matt Mackall <mpm@selenic.com>
parents: 3390
diff changeset
1714 pass # may be partial hex id
36
da28286bf6b7 Add smart node lookup by substring or by rev number
mpm@selenic.com
parents: 26
diff changeset
1715 try:
3156
d01e4cb2f5f2 cleanups in revlog.lookup
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 3139
diff changeset
1716 # str(rev)
36
da28286bf6b7 Add smart node lookup by substring or by rev number
mpm@selenic.com
parents: 26
diff changeset
1717 rev = int(id)
36721
981f328d6d16 py3: use b"%d" instead of str() to convert integers to bytes
Pulkit Goyal <7895pulkit@gmail.com>
parents: 36702
diff changeset
1718 if "%d" % rev != id:
4980
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
1719 raise ValueError
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
1720 if rev < 0:
6750
fb42030d79d6 add __len__ and __iter__ methods to repo and revlog
Matt Mackall <mpm@selenic.com>
parents: 6703
diff changeset
1721 rev = len(self) + rev
fb42030d79d6 add __len__ and __iter__ methods to repo and revlog
Matt Mackall <mpm@selenic.com>
parents: 6703
diff changeset
1722 if rev < 0 or rev >= len(self):
4980
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
1723 raise ValueError
36
da28286bf6b7 Add smart node lookup by substring or by rev number
mpm@selenic.com
parents: 26
diff changeset
1724 return self.node(rev)
469
e205194ca7ef Various node id lookup tweaks
mpm@selenic.com
parents: 451
diff changeset
1725 except (ValueError, OverflowError):
3156
d01e4cb2f5f2 cleanups in revlog.lookup
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 3139
diff changeset
1726 pass
3453
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
1727 if len(id) == 40:
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
1728 try:
3438
b17f9d3eda74 revlog.lookup tweaks
Matt Mackall <mpm@selenic.com>
parents: 3390
diff changeset
1729 # a full hex nodeid?
b17f9d3eda74 revlog.lookup tweaks
Matt Mackall <mpm@selenic.com>
parents: 3390
diff changeset
1730 node = bin(id)
7874
d812029cda85 cleanup: drop variables for unused return values
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 7873
diff changeset
1731 self.rev(node)
3157
4fe41a9e4591 optimize revlog.lookup when passed hex(node)[:...]
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 3156
diff changeset
1732 return node
7062
efc579fdaf69 provide nicer feedback when an unknown node id is passed to a command
Sune Foldager <cryo@cyanite.org>
parents: 6912
diff changeset
1733 except (TypeError, LookupError):
3453
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
1734 pass
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
1735
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
1736 def _partialmatch(self, id):
37449
a0d71618074f revlog: detect pseudo file nodeids to raise WdirUnsupported exception
Yuya Nishihara <yuya@tcha.org>
parents: 37443
diff changeset
1737 # we don't care wdirfilenodeids as they should be always full hash
32684
af854b1b36f8 revlog: add support for partial matching of wdir node id
Yuya Nishihara <yuya@tcha.org>
parents: 32659
diff changeset
1738 maybewdir = wdirhex.startswith(id)
16665
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16533
diff changeset
1739 try:
30391
2ded17b64f09 revlog: avoid shadowing several variables using list comprehensions
Augie Fackler <augie@google.com>
parents: 30289
diff changeset
1740 partial = self.index.partialmatch(id)
2ded17b64f09 revlog: avoid shadowing several variables using list comprehensions
Augie Fackler <augie@google.com>
parents: 30289
diff changeset
1741 if partial and self.hasnode(partial):
32684
af854b1b36f8 revlog: add support for partial matching of wdir node id
Yuya Nishihara <yuya@tcha.org>
parents: 32659
diff changeset
1742 if maybewdir:
af854b1b36f8 revlog: add support for partial matching of wdir node id
Yuya Nishihara <yuya@tcha.org>
parents: 32659
diff changeset
1743 # single 'ff...' match in radix tree, ambiguous with wdir
af854b1b36f8 revlog: add support for partial matching of wdir node id
Yuya Nishihara <yuya@tcha.org>
parents: 32659
diff changeset
1744 raise RevlogError
30391
2ded17b64f09 revlog: avoid shadowing several variables using list comprehensions
Augie Fackler <augie@google.com>
parents: 30289
diff changeset
1745 return partial
32684
af854b1b36f8 revlog: add support for partial matching of wdir node id
Yuya Nishihara <yuya@tcha.org>
parents: 32659
diff changeset
1746 if maybewdir:
af854b1b36f8 revlog: add support for partial matching of wdir node id
Yuya Nishihara <yuya@tcha.org>
parents: 32659
diff changeset
1747 # no 'ff...' match in radix tree, wdir identified
af854b1b36f8 revlog: add support for partial matching of wdir node id
Yuya Nishihara <yuya@tcha.org>
parents: 32659
diff changeset
1748 raise error.WdirUnsupported
19471
fd1bb7c1be78 revlog: handle hidden revs in _partialmatch (issue3979)
Matt Mackall <mpm@selenic.com>
parents: 19326
diff changeset
1749 return None
16665
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16533
diff changeset
1750 except RevlogError:
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16533
diff changeset
1751 # parsers.c radix tree lookup gave multiple matches
29396
d0ae5b8f80dc revlog: add a fast path for "ambiguous identifier"
Jun Wu <quark@fb.com>
parents: 29339
diff changeset
1752 # fast path: for unfiltered changelog, radix tree is accurate
d0ae5b8f80dc revlog: add a fast path for "ambiguous identifier"
Jun Wu <quark@fb.com>
parents: 29339
diff changeset
1753 if not getattr(self, 'filteredrevs', None):
d0ae5b8f80dc revlog: add a fast path for "ambiguous identifier"
Jun Wu <quark@fb.com>
parents: 29339
diff changeset
1754 raise LookupError(id, self.indexfile,
d0ae5b8f80dc revlog: add a fast path for "ambiguous identifier"
Jun Wu <quark@fb.com>
parents: 29339
diff changeset
1755 _('ambiguous identifier'))
19471
fd1bb7c1be78 revlog: handle hidden revs in _partialmatch (issue3979)
Matt Mackall <mpm@selenic.com>
parents: 19326
diff changeset
1756 # fall through to slow path that filters hidden revisions
16665
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16533
diff changeset
1757 except (AttributeError, ValueError):
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16533
diff changeset
1758 # we are pure python, or key was too short to search radix tree
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16533
diff changeset
1759 pass
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16533
diff changeset
1760
13258
c2661863f16f revlog: introduce a cache for partial lookups
Matt Mackall <mpm@selenic.com>
parents: 13254
diff changeset
1761 if id in self._pcache:
c2661863f16f revlog: introduce a cache for partial lookups
Matt Mackall <mpm@selenic.com>
parents: 13254
diff changeset
1762 return self._pcache[id]
c2661863f16f revlog: introduce a cache for partial lookups
Matt Mackall <mpm@selenic.com>
parents: 13254
diff changeset
1763
37819
ee3d58b4a47f revlog: make pure version of _partialmatch() support 40-byte hex nodeids
Martin von Zweigbergk <martinvonz@google.com>
parents: 37767
diff changeset
1764 if len(id) <= 40:
3453
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
1765 try:
3438
b17f9d3eda74 revlog.lookup tweaks
Matt Mackall <mpm@selenic.com>
parents: 3390
diff changeset
1766 # hex(node)[:...]
9029
0001e49f1c11 compat: use // for integer division
Alejandro Santos <alejolp@alejolp.com>
parents: 8658
diff changeset
1767 l = len(id) // 2 # grab an even number of digits
13259
3b616dfa4b17 revlog: do revlog node->rev mapping by scanning
Matt Mackall <mpm@selenic.com>
parents: 13258
diff changeset
1768 prefix = bin(id[:l * 2])
3b616dfa4b17 revlog: do revlog node->rev mapping by scanning
Matt Mackall <mpm@selenic.com>
parents: 13258
diff changeset
1769 nl = [e[7] for e in self.index if e[7].startswith(prefix)]
19471
fd1bb7c1be78 revlog: handle hidden revs in _partialmatch (issue3979)
Matt Mackall <mpm@selenic.com>
parents: 19326
diff changeset
1770 nl = [n for n in nl if hex(n).startswith(id) and
fd1bb7c1be78 revlog: handle hidden revs in _partialmatch (issue3979)
Matt Mackall <mpm@selenic.com>
parents: 19326
diff changeset
1771 self.hasnode(n)]
7365
ec3aafa84d44 lookup: speed up partial lookup
Matt Mackall <mpm@selenic.com>
parents: 7363
diff changeset
1772 if len(nl) > 0:
32684
af854b1b36f8 revlog: add support for partial matching of wdir node id
Yuya Nishihara <yuya@tcha.org>
parents: 32659
diff changeset
1773 if len(nl) == 1 and not maybewdir:
13258
c2661863f16f revlog: introduce a cache for partial lookups
Matt Mackall <mpm@selenic.com>
parents: 13254
diff changeset
1774 self._pcache[id] = nl[0]
7365
ec3aafa84d44 lookup: speed up partial lookup
Matt Mackall <mpm@selenic.com>
parents: 7363
diff changeset
1775 return nl[0]
ec3aafa84d44 lookup: speed up partial lookup
Matt Mackall <mpm@selenic.com>
parents: 7363
diff changeset
1776 raise LookupError(id, self.indexfile,
ec3aafa84d44 lookup: speed up partial lookup
Matt Mackall <mpm@selenic.com>
parents: 7363
diff changeset
1777 _('ambiguous identifier'))
32684
af854b1b36f8 revlog: add support for partial matching of wdir node id
Yuya Nishihara <yuya@tcha.org>
parents: 32659
diff changeset
1778 if maybewdir:
af854b1b36f8 revlog: add support for partial matching of wdir node id
Yuya Nishihara <yuya@tcha.org>
parents: 32659
diff changeset
1779 raise error.WdirUnsupported
7365
ec3aafa84d44 lookup: speed up partial lookup
Matt Mackall <mpm@selenic.com>
parents: 7363
diff changeset
1780 return None
36238
f574cc00831a node: make bin() be a wrapper instead of just an alias
Augie Fackler <augie@google.com>
parents: 35974
diff changeset
1781 except TypeError:
3453
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
1782 pass
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
1783
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
1784 def lookup(self, id):
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
1785 """locate a node based on:
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
1786 - revision number or str(revision number)
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
1787 - nodeid or subset of hex nodeid
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
1788 """
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
1789 n = self._match(id)
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
1790 if n is not None:
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
1791 return n
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
1792 n = self._partialmatch(id)
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
1793 if n:
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
1794 return n
515
03f27b1381f9 Whitespace cleanups
mpm@selenic.com
parents: 484
diff changeset
1795
6228
c0c4c7b1e8d3 revlog: report node and file when lookup fails
Matt Mackall <mpm@selenic.com>
parents: 6212
diff changeset
1796 raise LookupError(id, self.indexfile, _('no match found'))
36
da28286bf6b7 Add smart node lookup by substring or by rev number
mpm@selenic.com
parents: 26
diff changeset
1797
37767
44d1959acb3b revlog: make shortest() take a full binary nodeid (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 37494
diff changeset
1798 def shortest(self, node, minlength=1):
44d1959acb3b revlog: make shortest() take a full binary nodeid (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 37494
diff changeset
1799 """Find the shortest unambiguous prefix that matches node."""
37863
6921d3ecadc1 shortest: rename "test" variable to "prefix"
Martin von Zweigbergk <martinvonz@google.com>
parents: 37862
diff changeset
1800 def isvalid(prefix):
34250
448725a2ef73 templater: extract shortest() logic from template function
Martin von Zweigbergk <martinvonz@google.com>
parents: 34148
diff changeset
1801 try:
37865
da083d9fafab shortest: don't keep checking for longer prefix if node doesn't exist (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 37864
diff changeset
1802 node = self._partialmatch(prefix)
34250
448725a2ef73 templater: extract shortest() logic from template function
Martin von Zweigbergk <martinvonz@google.com>
parents: 34148
diff changeset
1803 except error.RevlogError:
448725a2ef73 templater: extract shortest() logic from template function
Martin von Zweigbergk <martinvonz@google.com>
parents: 34148
diff changeset
1804 return False
448725a2ef73 templater: extract shortest() logic from template function
Martin von Zweigbergk <martinvonz@google.com>
parents: 34148
diff changeset
1805 except error.WdirUnsupported:
448725a2ef73 templater: extract shortest() logic from template function
Martin von Zweigbergk <martinvonz@google.com>
parents: 34148
diff changeset
1806 # single 'ff...' match
448725a2ef73 templater: extract shortest() logic from template function
Martin von Zweigbergk <martinvonz@google.com>
parents: 34148
diff changeset
1807 return True
37865
da083d9fafab shortest: don't keep checking for longer prefix if node doesn't exist (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 37864
diff changeset
1808 if node is None:
da083d9fafab shortest: don't keep checking for longer prefix if node doesn't exist (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 37864
diff changeset
1809 raise LookupError(node, self.indexfile, _('no node'))
37970
76e933e0ccc9 shortest: remove unnecessary check for revnum in isvalid()
Martin von Zweigbergk <martinvonz@google.com>
parents: 37969
diff changeset
1810 return True
34250
448725a2ef73 templater: extract shortest() logic from template function
Martin von Zweigbergk <martinvonz@google.com>
parents: 34148
diff changeset
1811
37968
0304f22497fa revlog: use node tree (native code) for shortest() calculation
Martin von Zweigbergk <martinvonz@google.com>
parents: 37865
diff changeset
1812 def maybewdir(prefix):
0304f22497fa revlog: use node tree (native code) for shortest() calculation
Martin von Zweigbergk <martinvonz@google.com>
parents: 37865
diff changeset
1813 return all(c == 'f' for c in prefix)
0304f22497fa revlog: use node tree (native code) for shortest() calculation
Martin von Zweigbergk <martinvonz@google.com>
parents: 37865
diff changeset
1814
37767
44d1959acb3b revlog: make shortest() take a full binary nodeid (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 37494
diff changeset
1815 hexnode = hex(node)
37968
0304f22497fa revlog: use node tree (native code) for shortest() calculation
Martin von Zweigbergk <martinvonz@google.com>
parents: 37865
diff changeset
1816
0304f22497fa revlog: use node tree (native code) for shortest() calculation
Martin von Zweigbergk <martinvonz@google.com>
parents: 37865
diff changeset
1817 def disambiguate(hexnode, minlength):
37971
3ac950cd5978 shortest: move revnum-disambiguation out of revlog
Martin von Zweigbergk <martinvonz@google.com>
parents: 37970
diff changeset
1818 """Disambiguate against wdirid."""
37968
0304f22497fa revlog: use node tree (native code) for shortest() calculation
Martin von Zweigbergk <martinvonz@google.com>
parents: 37865
diff changeset
1819 for length in range(minlength, 41):
0304f22497fa revlog: use node tree (native code) for shortest() calculation
Martin von Zweigbergk <martinvonz@google.com>
parents: 37865
diff changeset
1820 prefix = hexnode[:length]
37971
3ac950cd5978 shortest: move revnum-disambiguation out of revlog
Martin von Zweigbergk <martinvonz@google.com>
parents: 37970
diff changeset
1821 if not maybewdir(prefix):
37968
0304f22497fa revlog: use node tree (native code) for shortest() calculation
Martin von Zweigbergk <martinvonz@google.com>
parents: 37865
diff changeset
1822 return prefix
0304f22497fa revlog: use node tree (native code) for shortest() calculation
Martin von Zweigbergk <martinvonz@google.com>
parents: 37865
diff changeset
1823
0304f22497fa revlog: use node tree (native code) for shortest() calculation
Martin von Zweigbergk <martinvonz@google.com>
parents: 37865
diff changeset
1824 if not getattr(self, 'filteredrevs', None):
0304f22497fa revlog: use node tree (native code) for shortest() calculation
Martin von Zweigbergk <martinvonz@google.com>
parents: 37865
diff changeset
1825 try:
0304f22497fa revlog: use node tree (native code) for shortest() calculation
Martin von Zweigbergk <martinvonz@google.com>
parents: 37865
diff changeset
1826 length = max(self.index.shortest(node), minlength)
0304f22497fa revlog: use node tree (native code) for shortest() calculation
Martin von Zweigbergk <martinvonz@google.com>
parents: 37865
diff changeset
1827 return disambiguate(hexnode, length)
0304f22497fa revlog: use node tree (native code) for shortest() calculation
Martin von Zweigbergk <martinvonz@google.com>
parents: 37865
diff changeset
1828 except RevlogError:
37969
0db7fe7c34d3 shortest: make pure code also disambigute against revnums at end
Martin von Zweigbergk <martinvonz@google.com>
parents: 37968
diff changeset
1829 if node != wdirid:
37968
0304f22497fa revlog: use node tree (native code) for shortest() calculation
Martin von Zweigbergk <martinvonz@google.com>
parents: 37865
diff changeset
1830 raise LookupError(node, self.indexfile, _('no node'))
0304f22497fa revlog: use node tree (native code) for shortest() calculation
Martin von Zweigbergk <martinvonz@google.com>
parents: 37865
diff changeset
1831 except AttributeError:
0304f22497fa revlog: use node tree (native code) for shortest() calculation
Martin von Zweigbergk <martinvonz@google.com>
parents: 37865
diff changeset
1832 # Fall through to pure code
0304f22497fa revlog: use node tree (native code) for shortest() calculation
Martin von Zweigbergk <martinvonz@google.com>
parents: 37865
diff changeset
1833 pass
0304f22497fa revlog: use node tree (native code) for shortest() calculation
Martin von Zweigbergk <martinvonz@google.com>
parents: 37865
diff changeset
1834
37969
0db7fe7c34d3 shortest: make pure code also disambigute against revnums at end
Martin von Zweigbergk <martinvonz@google.com>
parents: 37968
diff changeset
1835 if node == wdirid:
0db7fe7c34d3 shortest: make pure code also disambigute against revnums at end
Martin von Zweigbergk <martinvonz@google.com>
parents: 37968
diff changeset
1836 for length in range(minlength, 41):
0db7fe7c34d3 shortest: make pure code also disambigute against revnums at end
Martin von Zweigbergk <martinvonz@google.com>
parents: 37968
diff changeset
1837 prefix = hexnode[:length]
0db7fe7c34d3 shortest: make pure code also disambigute against revnums at end
Martin von Zweigbergk <martinvonz@google.com>
parents: 37968
diff changeset
1838 if isvalid(prefix):
0db7fe7c34d3 shortest: make pure code also disambigute against revnums at end
Martin von Zweigbergk <martinvonz@google.com>
parents: 37968
diff changeset
1839 return prefix
0db7fe7c34d3 shortest: make pure code also disambigute against revnums at end
Martin von Zweigbergk <martinvonz@google.com>
parents: 37968
diff changeset
1840
0db7fe7c34d3 shortest: make pure code also disambigute against revnums at end
Martin von Zweigbergk <martinvonz@google.com>
parents: 37968
diff changeset
1841 for length in range(minlength, 41):
37863
6921d3ecadc1 shortest: rename "test" variable to "prefix"
Martin von Zweigbergk <martinvonz@google.com>
parents: 37862
diff changeset
1842 prefix = hexnode[:length]
6921d3ecadc1 shortest: rename "test" variable to "prefix"
Martin von Zweigbergk <martinvonz@google.com>
parents: 37862
diff changeset
1843 if isvalid(prefix):
37969
0db7fe7c34d3 shortest: make pure code also disambigute against revnums at end
Martin von Zweigbergk <martinvonz@google.com>
parents: 37968
diff changeset
1844 return disambiguate(hexnode, length)
34250
448725a2ef73 templater: extract shortest() logic from template function
Martin von Zweigbergk <martinvonz@google.com>
parents: 34148
diff changeset
1845
2890
5df3e5cf16bc Move cmp bits from filelog to revlog
Matt Mackall <mpm@selenic.com>
parents: 2859
diff changeset
1846 def cmp(self, node, text):
11539
a463e3c50212 cmp: document the fact that we return True if content is different
Nicolas Dumazet <nicdumz.commits@gmail.com>
parents: 11323
diff changeset
1847 """compare text with a given file revision
a463e3c50212 cmp: document the fact that we return True if content is different
Nicolas Dumazet <nicdumz.commits@gmail.com>
parents: 11323
diff changeset
1848
a463e3c50212 cmp: document the fact that we return True if content is different
Nicolas Dumazet <nicdumz.commits@gmail.com>
parents: 11323
diff changeset
1849 returns True if text is different than what is stored.
a463e3c50212 cmp: document the fact that we return True if content is different
Nicolas Dumazet <nicdumz.commits@gmail.com>
parents: 11323
diff changeset
1850 """
2890
5df3e5cf16bc Move cmp bits from filelog to revlog
Matt Mackall <mpm@selenic.com>
parents: 2859
diff changeset
1851 p1, p2 = self.parents(node)
5df3e5cf16bc Move cmp bits from filelog to revlog
Matt Mackall <mpm@selenic.com>
parents: 2859
diff changeset
1852 return hash(text, p1, p2) != node
5df3e5cf16bc Move cmp bits from filelog to revlog
Matt Mackall <mpm@selenic.com>
parents: 2859
diff changeset
1853
32227
1395f843ece4 revlog: rename internal functions containing "chunk" to use "segment"
Gregory Szorc <gregory.szorc@gmail.com>
parents: 31856
diff changeset
1854 def _cachesegment(self, offset, data):
27070
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
1855 """Add a segment to the revlog cache.
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
1856
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
1857 Accepts an absolute offset and the data that is at that location.
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
1858 """
8316
d593922cf480 revlog: clean up the chunk caching code
Matt Mackall <mpm@selenic.com>
parents: 8315
diff changeset
1859 o, d = self._chunkcache
d593922cf480 revlog: clean up the chunk caching code
Matt Mackall <mpm@selenic.com>
parents: 8315
diff changeset
1860 # try to add to existing cache
13253
61c9bc3da402 revlog: remove lazy index
Matt Mackall <mpm@selenic.com>
parents: 13239
diff changeset
1861 if o + len(d) == offset and len(d) + len(data) < _chunksize:
8316
d593922cf480 revlog: clean up the chunk caching code
Matt Mackall <mpm@selenic.com>
parents: 8315
diff changeset
1862 self._chunkcache = o, d + data
d593922cf480 revlog: clean up the chunk caching code
Matt Mackall <mpm@selenic.com>
parents: 8315
diff changeset
1863 else:
d593922cf480 revlog: clean up the chunk caching code
Matt Mackall <mpm@selenic.com>
parents: 8315
diff changeset
1864 self._chunkcache = offset, data
d593922cf480 revlog: clean up the chunk caching code
Matt Mackall <mpm@selenic.com>
parents: 8315
diff changeset
1865
32227
1395f843ece4 revlog: rename internal functions containing "chunk" to use "segment"
Gregory Szorc <gregory.szorc@gmail.com>
parents: 31856
diff changeset
1866 def _readsegment(self, offset, length, df=None):
27070
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
1867 """Load a segment of raw data from the revlog.
26377
dfef0d3be65e revlog: support using an existing file handle when reading revlogs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26376
diff changeset
1868
27070
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
1869 Accepts an absolute offset, length to read, and an optional existing
26377
dfef0d3be65e revlog: support using an existing file handle when reading revlogs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26376
diff changeset
1870 file handle to read from.
dfef0d3be65e revlog: support using an existing file handle when reading revlogs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26376
diff changeset
1871
dfef0d3be65e revlog: support using an existing file handle when reading revlogs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26376
diff changeset
1872 If an existing file handle is passed, it will be seeked and the
dfef0d3be65e revlog: support using an existing file handle when reading revlogs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26376
diff changeset
1873 original seek position will NOT be restored.
27070
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
1874
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
1875 Returns a str or buffer of raw byte data.
26377
dfef0d3be65e revlog: support using an existing file handle when reading revlogs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26376
diff changeset
1876 """
20179
5bb3826bdac4 revlog: read/cache chunks in fixed windows of 64 KB
Brodie Rao <brodie@sf.io>
parents: 20074
diff changeset
1877 # Cache data both forward and backward around the requested
5bb3826bdac4 revlog: read/cache chunks in fixed windows of 64 KB
Brodie Rao <brodie@sf.io>
parents: 20074
diff changeset
1878 # data, in a fixed size window. This helps speed up operations
5bb3826bdac4 revlog: read/cache chunks in fixed windows of 64 KB
Brodie Rao <brodie@sf.io>
parents: 20074
diff changeset
1879 # involving reading the revlog backwards.
20180
969148b49fc6 revlog: allow tuning of the chunk cache size (via format.chunkcachesize)
Brodie Rao <brodie@sf.io>
parents: 20179
diff changeset
1880 cachesize = self._chunkcachesize
969148b49fc6 revlog: allow tuning of the chunk cache size (via format.chunkcachesize)
Brodie Rao <brodie@sf.io>
parents: 20179
diff changeset
1881 realoffset = offset & ~(cachesize - 1)
969148b49fc6 revlog: allow tuning of the chunk cache size (via format.chunkcachesize)
Brodie Rao <brodie@sf.io>
parents: 20179
diff changeset
1882 reallength = (((offset + length + cachesize) & ~(cachesize - 1))
969148b49fc6 revlog: allow tuning of the chunk cache size (via format.chunkcachesize)
Brodie Rao <brodie@sf.io>
parents: 20179
diff changeset
1883 - realoffset)
35973
4d66993bdcff revlog: add a _datareadfp context manager for data access needs
Boris Feld <boris.feld@octobus.net>
parents: 35972
diff changeset
1884 with self._datareadfp(df) as df:
4d66993bdcff revlog: add a _datareadfp context manager for data access needs
Boris Feld <boris.feld@octobus.net>
parents: 35972
diff changeset
1885 df.seek(realoffset)
4d66993bdcff revlog: add a _datareadfp context manager for data access needs
Boris Feld <boris.feld@octobus.net>
parents: 35972
diff changeset
1886 d = df.read(reallength)
32227
1395f843ece4 revlog: rename internal functions containing "chunk" to use "segment"
Gregory Szorc <gregory.szorc@gmail.com>
parents: 31856
diff changeset
1887 self._cachesegment(realoffset, d)
20179
5bb3826bdac4 revlog: read/cache chunks in fixed windows of 64 KB
Brodie Rao <brodie@sf.io>
parents: 20074
diff changeset
1888 if offset != realoffset or reallength != length:
5bb3826bdac4 revlog: read/cache chunks in fixed windows of 64 KB
Brodie Rao <brodie@sf.io>
parents: 20074
diff changeset
1889 return util.buffer(d, offset - realoffset, length)
8316
d593922cf480 revlog: clean up the chunk caching code
Matt Mackall <mpm@selenic.com>
parents: 8315
diff changeset
1890 return d
d593922cf480 revlog: clean up the chunk caching code
Matt Mackall <mpm@selenic.com>
parents: 8315
diff changeset
1891
32227
1395f843ece4 revlog: rename internal functions containing "chunk" to use "segment"
Gregory Szorc <gregory.szorc@gmail.com>
parents: 31856
diff changeset
1892 def _getsegment(self, offset, length, df=None):
27070
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
1893 """Obtain a segment of raw data from the revlog.
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
1894
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
1895 Accepts an absolute offset, length of bytes to obtain, and an
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
1896 optional file handle to the already-opened revlog. If the file
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
1897 handle is used, it's original seek position will not be preserved.
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
1898
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
1899 Requests for data may be returned from a cache.
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
1900
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
1901 Returns a str or a buffer instance of raw byte data.
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
1902 """
8316
d593922cf480 revlog: clean up the chunk caching code
Matt Mackall <mpm@selenic.com>
parents: 8315
diff changeset
1903 o, d = self._chunkcache
d593922cf480 revlog: clean up the chunk caching code
Matt Mackall <mpm@selenic.com>
parents: 8315
diff changeset
1904 l = len(d)
d593922cf480 revlog: clean up the chunk caching code
Matt Mackall <mpm@selenic.com>
parents: 8315
diff changeset
1905
d593922cf480 revlog: clean up the chunk caching code
Matt Mackall <mpm@selenic.com>
parents: 8315
diff changeset
1906 # is it in the cache?
d593922cf480 revlog: clean up the chunk caching code
Matt Mackall <mpm@selenic.com>
parents: 8315
diff changeset
1907 cachestart = offset - o
d593922cf480 revlog: clean up the chunk caching code
Matt Mackall <mpm@selenic.com>
parents: 8315
diff changeset
1908 cacheend = cachestart + length
d593922cf480 revlog: clean up the chunk caching code
Matt Mackall <mpm@selenic.com>
parents: 8315
diff changeset
1909 if cachestart >= 0 and cacheend <= l:
d593922cf480 revlog: clean up the chunk caching code
Matt Mackall <mpm@selenic.com>
parents: 8315
diff changeset
1910 if cachestart == 0 and cacheend == l:
d593922cf480 revlog: clean up the chunk caching code
Matt Mackall <mpm@selenic.com>
parents: 8315
diff changeset
1911 return d # avoid a copy
16423
a150923b49ba revlog: avoid an expensive string copy
Bryan O'Sullivan <bryano@fb.com>
parents: 16418
diff changeset
1912 return util.buffer(d, cachestart, cacheend - cachestart)
8316
d593922cf480 revlog: clean up the chunk caching code
Matt Mackall <mpm@selenic.com>
parents: 8315
diff changeset
1913
32227
1395f843ece4 revlog: rename internal functions containing "chunk" to use "segment"
Gregory Szorc <gregory.szorc@gmail.com>
parents: 31856
diff changeset
1914 return self._readsegment(offset, length, df=df)
8316
d593922cf480 revlog: clean up the chunk caching code
Matt Mackall <mpm@selenic.com>
parents: 8315
diff changeset
1915
32229
75e93d95aae6 revlog: rename _chunkraw to _getsegmentforrevs()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32227
diff changeset
1916 def _getsegmentforrevs(self, startrev, endrev, df=None):
27070
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
1917 """Obtain a segment of raw data corresponding to a range of revisions.
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
1918
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
1919 Accepts the start and end revisions and an optional already-open
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
1920 file handle to be used for reading. If the file handle is read, its
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
1921 seek position will not be preserved.
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
1922
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
1923 Requests for data may be satisfied by a cache.
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
1924
27649
6446e9b37c8b revlog: return offset from _chunkraw()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27637
diff changeset
1925 Returns a 2-tuple of (offset, data) for the requested range of
6446e9b37c8b revlog: return offset from _chunkraw()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27637
diff changeset
1926 revisions. Offset is the integer offset from the beginning of the
6446e9b37c8b revlog: return offset from _chunkraw()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27637
diff changeset
1927 revlog and data is a str or buffer of the raw byte data.
6446e9b37c8b revlog: return offset from _chunkraw()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27637
diff changeset
1928
6446e9b37c8b revlog: return offset from _chunkraw()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27637
diff changeset
1929 Callers will need to call ``self.start(rev)`` and ``self.length(rev)``
6446e9b37c8b revlog: return offset from _chunkraw()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27637
diff changeset
1930 to determine where each revision's data begins and ends.
27070
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
1931 """
30288
ceddc3d94d74 revlog: inline start() and end() for perf reasons
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30287
diff changeset
1932 # Inlined self.start(startrev) & self.end(endrev) for perf reasons
ceddc3d94d74 revlog: inline start() and end() for perf reasons
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30287
diff changeset
1933 # (functions are expensive).
ceddc3d94d74 revlog: inline start() and end() for perf reasons
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30287
diff changeset
1934 index = self.index
ceddc3d94d74 revlog: inline start() and end() for perf reasons
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30287
diff changeset
1935 istart = index[startrev]
ceddc3d94d74 revlog: inline start() and end() for perf reasons
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30287
diff changeset
1936 start = int(istart[0] >> 16)
30289
1f92056c4066 revlog: optimize _chunkraw when startrev==endrev
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30288
diff changeset
1937 if startrev == endrev:
1f92056c4066 revlog: optimize _chunkraw when startrev==endrev
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30288
diff changeset
1938 end = start + istart[1]
1f92056c4066 revlog: optimize _chunkraw when startrev==endrev
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30288
diff changeset
1939 else:
1f92056c4066 revlog: optimize _chunkraw when startrev==endrev
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30288
diff changeset
1940 iend = index[endrev]
1f92056c4066 revlog: optimize _chunkraw when startrev==endrev
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30288
diff changeset
1941 end = int(iend[0] >> 16) + iend[1]
30288
ceddc3d94d74 revlog: inline start() and end() for perf reasons
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30287
diff changeset
1942
8318
6b8513f8274a revlog: add cache priming for reconstructing delta chains
Matt Mackall <mpm@selenic.com>
parents: 8317
diff changeset
1943 if self._inline:
6b8513f8274a revlog: add cache priming for reconstructing delta chains
Matt Mackall <mpm@selenic.com>
parents: 8317
diff changeset
1944 start += (startrev + 1) * self._io.size
19714
0e07c0b5fb1c revlog.revision: fix cache preload for inline revlogs
Siddharth Agarwal <sid0@fb.com>
parents: 19713
diff changeset
1945 end += (endrev + 1) * self._io.size
0e07c0b5fb1c revlog.revision: fix cache preload for inline revlogs
Siddharth Agarwal <sid0@fb.com>
parents: 19713
diff changeset
1946 length = end - start
27649
6446e9b37c8b revlog: return offset from _chunkraw()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27637
diff changeset
1947
32227
1395f843ece4 revlog: rename internal functions containing "chunk" to use "segment"
Gregory Szorc <gregory.szorc@gmail.com>
parents: 31856
diff changeset
1948 return start, self._getsegment(start, length, df=df)
8318
6b8513f8274a revlog: add cache priming for reconstructing delta chains
Matt Mackall <mpm@selenic.com>
parents: 8317
diff changeset
1949
26377
dfef0d3be65e revlog: support using an existing file handle when reading revlogs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26376
diff changeset
1950 def _chunk(self, rev, df=None):
27070
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
1951 """Obtain a single decompressed chunk for a revision.
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
1952
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
1953 Accepts an integer revision and an optional already-open file handle
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
1954 to be used for reading. If used, the seek position of the file will not
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
1955 be preserved.
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
1956
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
1957 Returns a str holding uncompressed data for the requested revision.
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
1958 """
32229
75e93d95aae6 revlog: rename _chunkraw to _getsegmentforrevs()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32227
diff changeset
1959 return self.decompress(self._getsegmentforrevs(rev, rev, df=df)[1])
8650
ef393d6ec030 revlog: refactor chunk cache interface again
Matt Mackall <mpm@selenic.com>
parents: 8643
diff changeset
1960
38644
43d0619cec90 revlog: enforce chunk slicing down to a certain size
Boris Feld <boris.feld@octobus.net>
parents: 38643
diff changeset
1961 def _chunks(self, revs, df=None, targetsize=None):
27070
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
1962 """Obtain decompressed chunks for the specified revisions.
19713
c2e27e57d250 revlog: add a fast method for getting a list of chunks
Siddharth Agarwal <sid0@fb.com>
parents: 19625
diff changeset
1963
27070
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
1964 Accepts an iterable of numeric revisions that are assumed to be in
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
1965 ascending order. Also accepts an optional already-open file handle
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
1966 to be used for reading. If used, the seek position of the file will
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
1967 not be preserved.
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
1968
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
1969 This function is similar to calling ``self._chunk()`` multiple times,
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
1970 but is faster.
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
1971
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
1972 Returns a list with decompressed data for each requested revision.
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
1973 """
19716
e17976978ee4 revlog: move chunk cache preload from revision to _chunks
Siddharth Agarwal <sid0@fb.com>
parents: 19715
diff changeset
1974 if not revs:
e17976978ee4 revlog: move chunk cache preload from revision to _chunks
Siddharth Agarwal <sid0@fb.com>
parents: 19715
diff changeset
1975 return []
19713
c2e27e57d250 revlog: add a fast method for getting a list of chunks
Siddharth Agarwal <sid0@fb.com>
parents: 19625
diff changeset
1976 start = self.start
c2e27e57d250 revlog: add a fast method for getting a list of chunks
Siddharth Agarwal <sid0@fb.com>
parents: 19625
diff changeset
1977 length = self.length
c2e27e57d250 revlog: add a fast method for getting a list of chunks
Siddharth Agarwal <sid0@fb.com>
parents: 19625
diff changeset
1978 inline = self._inline
c2e27e57d250 revlog: add a fast method for getting a list of chunks
Siddharth Agarwal <sid0@fb.com>
parents: 19625
diff changeset
1979 iosize = self._io.size
19715
1aab406be57c revlog._chunks: inline getchunk
Siddharth Agarwal <sid0@fb.com>
parents: 19714
diff changeset
1980 buffer = util.buffer
19713
c2e27e57d250 revlog: add a fast method for getting a list of chunks
Siddharth Agarwal <sid0@fb.com>
parents: 19625
diff changeset
1981
c2e27e57d250 revlog: add a fast method for getting a list of chunks
Siddharth Agarwal <sid0@fb.com>
parents: 19625
diff changeset
1982 l = []
c2e27e57d250 revlog: add a fast method for getting a list of chunks
Siddharth Agarwal <sid0@fb.com>
parents: 19625
diff changeset
1983 ladd = l.append
c2e27e57d250 revlog: add a fast method for getting a list of chunks
Siddharth Agarwal <sid0@fb.com>
parents: 19625
diff changeset
1984
34824
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
1985 if not self._withsparseread:
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
1986 slicedchunks = (revs,)
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
1987 else:
38644
43d0619cec90 revlog: enforce chunk slicing down to a certain size
Boris Feld <boris.feld@octobus.net>
parents: 38643
diff changeset
1988 slicedchunks = _slicechunk(self, revs, targetsize)
34824
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
1989
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
1990 for revschunk in slicedchunks:
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
1991 firstrev = revschunk[0]
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
1992 # Skip trailing revisions with empty diff
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
1993 for lastrev in revschunk[::-1]:
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
1994 if length(lastrev) != 0:
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
1995 break
34823
7891d243d821 revlog: ignore empty trailing chunks when reading segments
Paul Morelle <paul.morelle@octobus.net>
parents: 34296
diff changeset
1996
34824
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
1997 try:
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
1998 offset, data = self._getsegmentforrevs(firstrev, lastrev, df=df)
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
1999 except OverflowError:
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
2000 # issue4215 - we can't cache a run of chunks greater than
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
2001 # 2G on Windows
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
2002 return [self._chunk(rev, df=df) for rev in revschunk]
19715
1aab406be57c revlog._chunks: inline getchunk
Siddharth Agarwal <sid0@fb.com>
parents: 19714
diff changeset
2003
34824
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
2004 decomp = self.decompress
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
2005 for rev in revschunk:
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
2006 chunkstart = start(rev)
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
2007 if inline:
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
2008 chunkstart += (rev + 1) * iosize
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
2009 chunklength = length(rev)
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
2010 ladd(decomp(buffer(data, chunkstart - offset, chunklength)))
19713
c2e27e57d250 revlog: add a fast method for getting a list of chunks
Siddharth Agarwal <sid0@fb.com>
parents: 19625
diff changeset
2011
c2e27e57d250 revlog: add a fast method for getting a list of chunks
Siddharth Agarwal <sid0@fb.com>
parents: 19625
diff changeset
2012 return l
14075
bc101902a68d revlog: introduce _chunkbase to allow filelog to override
Sune Foldager <cryo@cyanite.org>
parents: 14064
diff changeset
2013
8650
ef393d6ec030 revlog: refactor chunk cache interface again
Matt Mackall <mpm@selenic.com>
parents: 8643
diff changeset
2014 def _chunkclear(self):
27070
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
2015 """Clear the raw chunk cache."""
8650
ef393d6ec030 revlog: refactor chunk cache interface again
Matt Mackall <mpm@selenic.com>
parents: 8643
diff changeset
2016 self._chunkcache = (0, '')
1598
14d1f1868bf6 cleanup of revlog.group when repository is local
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1559
diff changeset
2017
11929
1839a7518b0d revlog: deltachain() returns chain of revs need to construct a revision
Pradeepkumar Gayam <in3xes@gmail.com>
parents: 11928
diff changeset
2018 def deltaparent(self, rev):
14195
0013d3eeb826 revlog: remove support for parentdelta
Sune Foldager <cryo@cyanite.org>
parents: 14164
diff changeset
2019 """return deltaparent of the given revision"""
14253
c28d5200374c revlog: support reading generaldelta revlogs
Sune Foldager <cryo@cyanite.org>
parents: 14252
diff changeset
2020 base = self.index[rev][3]
c28d5200374c revlog: support reading generaldelta revlogs
Sune Foldager <cryo@cyanite.org>
parents: 14252
diff changeset
2021 if base == rev:
14208
d62d597b8974 revlog: compute correct deltaparent in the deltaparent function
Sune Foldager <cryo@cyanite.org>
parents: 14196
diff changeset
2022 return nullrev
14253
c28d5200374c revlog: support reading generaldelta revlogs
Sune Foldager <cryo@cyanite.org>
parents: 14252
diff changeset
2023 elif self._generaldelta:
c28d5200374c revlog: support reading generaldelta revlogs
Sune Foldager <cryo@cyanite.org>
parents: 14252
diff changeset
2024 return base
14208
d62d597b8974 revlog: compute correct deltaparent in the deltaparent function
Sune Foldager <cryo@cyanite.org>
parents: 14196
diff changeset
2025 else:
d62d597b8974 revlog: compute correct deltaparent in the deltaparent function
Sune Foldager <cryo@cyanite.org>
parents: 14196
diff changeset
2026 return rev - 1
11929
1839a7518b0d revlog: deltachain() returns chain of revs need to construct a revision
Pradeepkumar Gayam <in3xes@gmail.com>
parents: 11928
diff changeset
2027
1941
7518823709a2 revlog.py: factorization and fixes for rev < 0 (nullid)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1853
diff changeset
2028 def revdiff(self, rev1, rev2):
31753
5d11b5edcb0b revlog: use raw revisions in revdiff
Jun Wu <quark@fb.com>
parents: 31752
diff changeset
2029 """return or calculate a delta between two revisions
5d11b5edcb0b revlog: use raw revisions in revdiff
Jun Wu <quark@fb.com>
parents: 31752
diff changeset
2030
5d11b5edcb0b revlog: use raw revisions in revdiff
Jun Wu <quark@fb.com>
parents: 31752
diff changeset
2031 The delta calculated is in binary form and is intended to be written to
5d11b5edcb0b revlog: use raw revisions in revdiff
Jun Wu <quark@fb.com>
parents: 31752
diff changeset
2032 revlog data directly. So this function needs raw revision data.
5d11b5edcb0b revlog: use raw revisions in revdiff
Jun Wu <quark@fb.com>
parents: 31752
diff changeset
2033 """
14208
d62d597b8974 revlog: compute correct deltaparent in the deltaparent function
Sune Foldager <cryo@cyanite.org>
parents: 14196
diff changeset
2034 if rev1 != nullrev and self.deltaparent(rev2) == rev1:
31369
b6f5af372c0c revlog: use bytes() instead of str() to get data from memoryview
Augie Fackler <augie@google.com>
parents: 31357
diff changeset
2035 return bytes(self._chunk(rev2))
5005
72082bfced9a revlog: minor revdiff reorganization
Matt Mackall <mpm@selenic.com>
parents: 5004
diff changeset
2036
31753
5d11b5edcb0b revlog: use raw revisions in revdiff
Jun Wu <quark@fb.com>
parents: 31752
diff changeset
2037 return mdiff.textdiff(self.revision(rev1, raw=True),
5d11b5edcb0b revlog: use raw revisions in revdiff
Jun Wu <quark@fb.com>
parents: 31752
diff changeset
2038 self.revision(rev2, raw=True))
119
c7a66f9752a4 Add code to retrieve or construct a revlog delta
mpm@selenic.com
parents: 117
diff changeset
2039
30743
2df983125d37 revlog: add 'raw' argument to revision and _addrevision
Remi Chaintron <remi@fb.com>
parents: 30584
diff changeset
2040 def revision(self, nodeorrev, _df=None, raw=False):
16435
df347129305d revlog: fix partial revision() docstring (from d7d64b89a65c)
Patrick Mezard <patrick@mezard.eu>
parents: 16424
diff changeset
2041 """return an uncompressed revision of a given node or revision
df347129305d revlog: fix partial revision() docstring (from d7d64b89a65c)
Patrick Mezard <patrick@mezard.eu>
parents: 16424
diff changeset
2042 number.
26377
dfef0d3be65e revlog: support using an existing file handle when reading revlogs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26376
diff changeset
2043
30743
2df983125d37 revlog: add 'raw' argument to revision and _addrevision
Remi Chaintron <remi@fb.com>
parents: 30584
diff changeset
2044 _df - an existing file handle to read from. (internal-only)
2df983125d37 revlog: add 'raw' argument to revision and _addrevision
Remi Chaintron <remi@fb.com>
parents: 30584
diff changeset
2045 raw - an optional argument specifying if the revision data is to be
2df983125d37 revlog: add 'raw' argument to revision and _addrevision
Remi Chaintron <remi@fb.com>
parents: 30584
diff changeset
2046 treated as raw data when applying flag transforms. 'raw' should be set
2df983125d37 revlog: add 'raw' argument to revision and _addrevision
Remi Chaintron <remi@fb.com>
parents: 30584
diff changeset
2047 to True when generating changegroups or in debug commands.
16435
df347129305d revlog: fix partial revision() docstring (from d7d64b89a65c)
Patrick Mezard <patrick@mezard.eu>
parents: 16424
diff changeset
2048 """
16375
d7d64b89a65c revlog: allow retrieving contents by revision number
Matt Mackall <mpm@selenic.com>
parents: 16374
diff changeset
2049 if isinstance(nodeorrev, int):
d7d64b89a65c revlog: allow retrieving contents by revision number
Matt Mackall <mpm@selenic.com>
parents: 16374
diff changeset
2050 rev = nodeorrev
d7d64b89a65c revlog: allow retrieving contents by revision number
Matt Mackall <mpm@selenic.com>
parents: 16374
diff changeset
2051 node = self.node(rev)
d7d64b89a65c revlog: allow retrieving contents by revision number
Matt Mackall <mpm@selenic.com>
parents: 16374
diff changeset
2052 else:
d7d64b89a65c revlog: allow retrieving contents by revision number
Matt Mackall <mpm@selenic.com>
parents: 16374
diff changeset
2053 node = nodeorrev
d7d64b89a65c revlog: allow retrieving contents by revision number
Matt Mackall <mpm@selenic.com>
parents: 16374
diff changeset
2054 rev = None
d7d64b89a65c revlog: allow retrieving contents by revision number
Matt Mackall <mpm@selenic.com>
parents: 16374
diff changeset
2055
11996
3195cf01dfb9 revlog.revision(): don't use nullrev as the default value for the cache
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 11995
diff changeset
2056 cachedrev = None
31802
ac9a5e89113a revlog: avoid calculating "flags" twice in revision()
Jun Wu <quark@fb.com>
parents: 31801
diff changeset
2057 flags = None
31804
726f24123f02 revlog: avoid applying delta chain on cache hit
Jun Wu <quark@fb.com>
parents: 31803
diff changeset
2058 rawtext = None
4980
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
2059 if node == nullid:
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
2060 return ""
26242
d708873f1f33 revlog: drop local assignment of cache variable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26241
diff changeset
2061 if self._cache:
d708873f1f33 revlog: drop local assignment of cache variable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26241
diff changeset
2062 if self._cache[0] == node:
31751
2133437dad17 revlog: fix _cache usage in revision()
Jun Wu <quark@fb.com>
parents: 31750
diff changeset
2063 # _cache only stores rawtext
2133437dad17 revlog: fix _cache usage in revision()
Jun Wu <quark@fb.com>
parents: 31750
diff changeset
2064 if raw:
2133437dad17 revlog: fix _cache usage in revision()
Jun Wu <quark@fb.com>
parents: 31750
diff changeset
2065 return self._cache[2]
31756
9ec03d5af48f revlog: add a fast path for revision(raw=False)
Jun Wu <quark@fb.com>
parents: 31755
diff changeset
2066 # duplicated, but good for perf
9ec03d5af48f revlog: add a fast path for revision(raw=False)
Jun Wu <quark@fb.com>
parents: 31755
diff changeset
2067 if rev is None:
9ec03d5af48f revlog: add a fast path for revision(raw=False)
Jun Wu <quark@fb.com>
parents: 31755
diff changeset
2068 rev = self.rev(node)
31802
ac9a5e89113a revlog: avoid calculating "flags" twice in revision()
Jun Wu <quark@fb.com>
parents: 31801
diff changeset
2069 if flags is None:
ac9a5e89113a revlog: avoid calculating "flags" twice in revision()
Jun Wu <quark@fb.com>
parents: 31801
diff changeset
2070 flags = self.flags(rev)
31756
9ec03d5af48f revlog: add a fast path for revision(raw=False)
Jun Wu <quark@fb.com>
parents: 31755
diff changeset
2071 # no extra flags set, no flag processor runs, text = rawtext
31802
ac9a5e89113a revlog: avoid calculating "flags" twice in revision()
Jun Wu <quark@fb.com>
parents: 31801
diff changeset
2072 if flags == REVIDX_DEFAULT_FLAGS:
31756
9ec03d5af48f revlog: add a fast path for revision(raw=False)
Jun Wu <quark@fb.com>
parents: 31755
diff changeset
2073 return self._cache[2]
31804
726f24123f02 revlog: avoid applying delta chain on cache hit
Jun Wu <quark@fb.com>
parents: 31803
diff changeset
2074 # rawtext is reusable. need to run flag processor
726f24123f02 revlog: avoid applying delta chain on cache hit
Jun Wu <quark@fb.com>
parents: 31803
diff changeset
2075 rawtext = self._cache[2]
31756
9ec03d5af48f revlog: add a fast path for revision(raw=False)
Jun Wu <quark@fb.com>
parents: 31755
diff changeset
2076
26242
d708873f1f33 revlog: drop local assignment of cache variable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26241
diff changeset
2077 cachedrev = self._cache[1]
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
2078
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
2079 # look up what we need to read
31803
2be73f9720a8 revlog: indent block to make review easier
Jun Wu <quark@fb.com>
parents: 31802
diff changeset
2080 if rawtext is None:
2be73f9720a8 revlog: indent block to make review easier
Jun Wu <quark@fb.com>
parents: 31802
diff changeset
2081 if rev is None:
2be73f9720a8 revlog: indent block to make review easier
Jun Wu <quark@fb.com>
parents: 31802
diff changeset
2082 rev = self.rev(node)
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
2083
31803
2be73f9720a8 revlog: indent block to make review easier
Jun Wu <quark@fb.com>
parents: 31802
diff changeset
2084 chain, stopped = self._deltachain(rev, stoprev=cachedrev)
2be73f9720a8 revlog: indent block to make review easier
Jun Wu <quark@fb.com>
parents: 31802
diff changeset
2085 if stopped:
2be73f9720a8 revlog: indent block to make review easier
Jun Wu <quark@fb.com>
parents: 31802
diff changeset
2086 rawtext = self._cache[2]
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
2087
31803
2be73f9720a8 revlog: indent block to make review easier
Jun Wu <quark@fb.com>
parents: 31802
diff changeset
2088 # drop cache to save memory
2be73f9720a8 revlog: indent block to make review easier
Jun Wu <quark@fb.com>
parents: 31802
diff changeset
2089 self._cache = None
11754
6ccd130eab0e revlog: drop cache after use to save memory footprint
Matt Mackall <mpm@selenic.com>
parents: 11539
diff changeset
2090
38644
43d0619cec90 revlog: enforce chunk slicing down to a certain size
Boris Feld <boris.feld@octobus.net>
parents: 38643
diff changeset
2091 targetsize = None
43d0619cec90 revlog: enforce chunk slicing down to a certain size
Boris Feld <boris.feld@octobus.net>
parents: 38643
diff changeset
2092 rawsize = self.index[rev][2]
43d0619cec90 revlog: enforce chunk slicing down to a certain size
Boris Feld <boris.feld@octobus.net>
parents: 38643
diff changeset
2093 if 0 <= rawsize:
43d0619cec90 revlog: enforce chunk slicing down to a certain size
Boris Feld <boris.feld@octobus.net>
parents: 38643
diff changeset
2094 targetsize = 4 * rawsize
43d0619cec90 revlog: enforce chunk slicing down to a certain size
Boris Feld <boris.feld@octobus.net>
parents: 38643
diff changeset
2095
43d0619cec90 revlog: enforce chunk slicing down to a certain size
Boris Feld <boris.feld@octobus.net>
parents: 38643
diff changeset
2096 bins = self._chunks(chain, df=_df, targetsize=targetsize)
31803
2be73f9720a8 revlog: indent block to make review easier
Jun Wu <quark@fb.com>
parents: 31802
diff changeset
2097 if rawtext is None:
2be73f9720a8 revlog: indent block to make review easier
Jun Wu <quark@fb.com>
parents: 31802
diff changeset
2098 rawtext = bytes(bins[0])
2be73f9720a8 revlog: indent block to make review easier
Jun Wu <quark@fb.com>
parents: 31802
diff changeset
2099 bins = bins[1:]
8650
ef393d6ec030 revlog: refactor chunk cache interface again
Matt Mackall <mpm@selenic.com>
parents: 8643
diff changeset
2100
31803
2be73f9720a8 revlog: indent block to make review easier
Jun Wu <quark@fb.com>
parents: 31802
diff changeset
2101 rawtext = mdiff.patches(rawtext, bins)
31804
726f24123f02 revlog: avoid applying delta chain on cache hit
Jun Wu <quark@fb.com>
parents: 31803
diff changeset
2102 self._cache = (node, rev, rawtext)
30745
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2103
31802
ac9a5e89113a revlog: avoid calculating "flags" twice in revision()
Jun Wu <quark@fb.com>
parents: 31801
diff changeset
2104 if flags is None:
31804
726f24123f02 revlog: avoid applying delta chain on cache hit
Jun Wu <quark@fb.com>
parents: 31803
diff changeset
2105 if rev is None:
726f24123f02 revlog: avoid applying delta chain on cache hit
Jun Wu <quark@fb.com>
parents: 31803
diff changeset
2106 rev = self.rev(node)
31802
ac9a5e89113a revlog: avoid calculating "flags" twice in revision()
Jun Wu <quark@fb.com>
parents: 31801
diff changeset
2107 flags = self.flags(rev)
ac9a5e89113a revlog: avoid calculating "flags" twice in revision()
Jun Wu <quark@fb.com>
parents: 31801
diff changeset
2108
ac9a5e89113a revlog: avoid calculating "flags" twice in revision()
Jun Wu <quark@fb.com>
parents: 31801
diff changeset
2109 text, validatehash = self._processflags(rawtext, flags, 'read', raw=raw)
30745
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2110 if validatehash:
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2111 self.checkhash(text, node, rev=rev)
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2112
13239
12ed25f39d0b revlog: break hash checking into subfunction
Matt Mackall <mpm@selenic.com>
parents: 13031
diff changeset
2113 return text
12ed25f39d0b revlog: break hash checking into subfunction
Matt Mackall <mpm@selenic.com>
parents: 13031
diff changeset
2114
22785
abc44fcc9c57 revlog: move references to revlog.hash to inside the revlog class
Augie Fackler <raf@durin42.com>
parents: 22784
diff changeset
2115 def hash(self, text, p1, p2):
abc44fcc9c57 revlog: move references to revlog.hash to inside the revlog class
Augie Fackler <raf@durin42.com>
parents: 22784
diff changeset
2116 """Compute a node hash.
abc44fcc9c57 revlog: move references to revlog.hash to inside the revlog class
Augie Fackler <raf@durin42.com>
parents: 22784
diff changeset
2117
abc44fcc9c57 revlog: move references to revlog.hash to inside the revlog class
Augie Fackler <raf@durin42.com>
parents: 22784
diff changeset
2118 Available as a function so that subclasses can replace the hash
abc44fcc9c57 revlog: move references to revlog.hash to inside the revlog class
Augie Fackler <raf@durin42.com>
parents: 22784
diff changeset
2119 as needed.
abc44fcc9c57 revlog: move references to revlog.hash to inside the revlog class
Augie Fackler <raf@durin42.com>
parents: 22784
diff changeset
2120 """
abc44fcc9c57 revlog: move references to revlog.hash to inside the revlog class
Augie Fackler <raf@durin42.com>
parents: 22784
diff changeset
2121 return hash(text, p1, p2)
abc44fcc9c57 revlog: move references to revlog.hash to inside the revlog class
Augie Fackler <raf@durin42.com>
parents: 22784
diff changeset
2122
30745
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2123 def _processflags(self, text, flags, operation, raw=False):
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2124 """Inspect revision data flags and applies transforms defined by
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2125 registered flag processors.
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2126
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2127 ``text`` - the revision data to process
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2128 ``flags`` - the revision flags
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2129 ``operation`` - the operation being performed (read or write)
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2130 ``raw`` - an optional argument describing if the raw transform should be
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2131 applied.
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2132
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2133 This method processes the flags in the order (or reverse order if
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2134 ``operation`` is 'write') defined by REVIDX_FLAGS_ORDER, applying the
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2135 flag processors registered for present flags. The order of flags defined
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2136 in REVIDX_FLAGS_ORDER needs to be stable to allow non-commutativity.
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2137
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2138 Returns a 2-tuple of ``(text, validatehash)`` where ``text`` is the
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2139 processed text and ``validatehash`` is a bool indicating whether the
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2140 returned text should be checked for hash integrity.
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2141
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2142 Note: If the ``raw`` argument is set, it has precedence over the
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2143 operation and will only update the value of ``validatehash``.
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2144 """
32286
539cbe0f8fa3 flagprocessor: add a fast path when flags is 0
Jun Wu <quark@fb.com>
parents: 32244
diff changeset
2145 # fast path: no flag processors will run
539cbe0f8fa3 flagprocessor: add a fast path when flags is 0
Jun Wu <quark@fb.com>
parents: 32244
diff changeset
2146 if flags == 0:
539cbe0f8fa3 flagprocessor: add a fast path when flags is 0
Jun Wu <quark@fb.com>
parents: 32244
diff changeset
2147 return text, True
30745
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2148 if not operation in ('read', 'write'):
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2149 raise ProgrammingError(_("invalid '%s' operation ") % (operation))
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2150 # Check all flags are known.
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2151 if flags & ~REVIDX_KNOWN_FLAGS:
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2152 raise RevlogError(_("incompatible revision flag '%#x'") %
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2153 (flags & ~REVIDX_KNOWN_FLAGS))
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2154 validatehash = True
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2155 # Depending on the operation (read or write), the order might be
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2156 # reversed due to non-commutative transforms.
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2157 orderedflags = REVIDX_FLAGS_ORDER
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2158 if operation == 'write':
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2159 orderedflags = reversed(orderedflags)
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2160
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2161 for flag in orderedflags:
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2162 # If a flagprocessor has been registered for a known flag, apply the
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2163 # related operation transform and update result tuple.
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2164 if flag & flags:
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2165 vhash = True
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2166
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2167 if flag not in _flagprocessors:
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2168 message = _("missing processor for flag '%#x'") % (flag)
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2169 raise RevlogError(message)
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2170
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2171 processor = _flagprocessors[flag]
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2172 if processor is not None:
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2173 readtransform, writetransform, rawtransform = processor
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2174
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2175 if raw:
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2176 vhash = rawtransform(self, text)
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2177 elif operation == 'read':
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2178 text, vhash = readtransform(self, text)
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2179 else: # write operation
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2180 text, vhash = writetransform(self, text)
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2181 validatehash = validatehash and vhash
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2182
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2183 return text, validatehash
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2184
30584
be5b2098a817 revlog: merge hash checking subfunctions
Remi Chaintron <remi@fb.com>
parents: 30543
diff changeset
2185 def checkhash(self, text, node, p1=None, p2=None, rev=None):
be5b2098a817 revlog: merge hash checking subfunctions
Remi Chaintron <remi@fb.com>
parents: 30543
diff changeset
2186 """Check node hash integrity.
19624
55749cb14d24 revlog: extract 'checkhash' method
Wojciech Lopata <lopek@fb.com>
parents: 19471
diff changeset
2187
30584
be5b2098a817 revlog: merge hash checking subfunctions
Remi Chaintron <remi@fb.com>
parents: 30543
diff changeset
2188 Available as a function so that subclasses can extend hash mismatch
be5b2098a817 revlog: merge hash checking subfunctions
Remi Chaintron <remi@fb.com>
parents: 30543
diff changeset
2189 behaviors as needed.
be5b2098a817 revlog: merge hash checking subfunctions
Remi Chaintron <remi@fb.com>
parents: 30543
diff changeset
2190 """
37443
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
2191 try:
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
2192 if p1 is None and p2 is None:
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
2193 p1, p2 = self.parents(node)
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
2194 if node != self.hash(text, p1, p2):
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
2195 revornode = rev
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
2196 if revornode is None:
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
2197 revornode = templatefilters.short(hex(node))
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
2198 raise RevlogError(_("integrity check failed on %s:%s")
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
2199 % (self.indexfile, pycompat.bytestr(revornode)))
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
2200 except RevlogError:
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
2201 if self._censorable and _censoredtext(text):
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
2202 raise error.CensoredNodeError(self.indexfile, node, text)
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
2203 raise
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
2204
35974
9ba1d0c724e2 revlog: rename 'self.checkinlinesize' into '_enforceinlinesize'
Boris Feld <boris.feld@octobus.net>
parents: 35973
diff changeset
2205 def _enforceinlinesize(self, tr, fp=None):
26376
344a1621674b revlog: add docstring for checkinlinesize()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26243
diff changeset
2206 """Check if the revlog is too big for inline and convert if so.
344a1621674b revlog: add docstring for checkinlinesize()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26243
diff changeset
2207
344a1621674b revlog: add docstring for checkinlinesize()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26243
diff changeset
2208 This should be called after revisions are added to the revlog. If the
344a1621674b revlog: add docstring for checkinlinesize()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26243
diff changeset
2209 revlog has grown too large to be an inline revlog, it will convert it
344a1621674b revlog: add docstring for checkinlinesize()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26243
diff changeset
2210 to use multiple index and data files.
344a1621674b revlog: add docstring for checkinlinesize()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26243
diff changeset
2211 """
10913
f2ecc5733c89 revlog: factor out _maxinline global.
Greg Ward <greg-hg@gerg.ca>
parents: 10404
diff changeset
2212 if not self._inline or (self.start(-2) + self.length(-2)) < _maxinline:
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
2213 return
8315
c8493310ad9b revlog: use index to find index size
Matt Mackall <mpm@selenic.com>
parents: 8314
diff changeset
2214
2084
Chris Mason <mason@suse.com>
parents: 2082
diff changeset
2215 trinfo = tr.find(self.indexfile)
8527
f9a80054dd3c use 'x is None' instead of 'x == None'
Martin Geisler <mg@lazybytes.net>
parents: 8464
diff changeset
2216 if trinfo is None:
3680
69cf255a55a1 Indentation cleanups for 2956948b81f3.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 3679
diff changeset
2217 raise RevlogError(_("%s not found in the transaction")
69cf255a55a1 Indentation cleanups for 2956948b81f3.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 3679
diff changeset
2218 % self.indexfile)
2084
Chris Mason <mason@suse.com>
parents: 2082
diff changeset
2219
Chris Mason <mason@suse.com>
parents: 2082
diff changeset
2220 trindex = trinfo[2]
24454
59904edf0a5e revlog: make converting from inline to non-line work after a strip
Mike Edgar <adgar@google.com>
parents: 24444
diff changeset
2221 if trindex is not None:
59904edf0a5e revlog: make converting from inline to non-line work after a strip
Mike Edgar <adgar@google.com>
parents: 24444
diff changeset
2222 dataoff = self.start(trindex)
59904edf0a5e revlog: make converting from inline to non-line work after a strip
Mike Edgar <adgar@google.com>
parents: 24444
diff changeset
2223 else:
59904edf0a5e revlog: make converting from inline to non-line work after a strip
Mike Edgar <adgar@google.com>
parents: 24444
diff changeset
2224 # revlog was stripped at start of transaction, use all leftover data
59904edf0a5e revlog: make converting from inline to non-line work after a strip
Mike Edgar <adgar@google.com>
parents: 24444
diff changeset
2225 trindex = len(self) - 1
59904edf0a5e revlog: make converting from inline to non-line work after a strip
Mike Edgar <adgar@google.com>
parents: 24444
diff changeset
2226 dataoff = self.end(-2)
2084
Chris Mason <mason@suse.com>
parents: 2082
diff changeset
2227
Chris Mason <mason@suse.com>
parents: 2082
diff changeset
2228 tr.add(self.datafile, dataoff)
8315
c8493310ad9b revlog: use index to find index size
Matt Mackall <mpm@selenic.com>
parents: 8314
diff changeset
2229
8317
5cdf4067857a revlog: use chunk cache to avoid rereading when splitting inline files
Matt Mackall <mpm@selenic.com>
parents: 8316
diff changeset
2230 if fp:
5cdf4067857a revlog: use chunk cache to avoid rereading when splitting inline files
Matt Mackall <mpm@selenic.com>
parents: 8316
diff changeset
2231 fp.flush()
5cdf4067857a revlog: use chunk cache to avoid rereading when splitting inline files
Matt Mackall <mpm@selenic.com>
parents: 8316
diff changeset
2232 fp.close()
8315
c8493310ad9b revlog: use index to find index size
Matt Mackall <mpm@selenic.com>
parents: 8314
diff changeset
2233
35970
69cf2e422490 revlog: use context manager for data file lifetime in checkinlinesize
Boris Feld <boris.feld@octobus.net>
parents: 35969
diff changeset
2234 with self._datafp('w') as df:
6750
fb42030d79d6 add __len__ and __iter__ methods to repo and revlog
Matt Mackall <mpm@selenic.com>
parents: 6703
diff changeset
2235 for r in self:
32229
75e93d95aae6 revlog: rename _chunkraw to _getsegmentforrevs()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32227
diff changeset
2236 df.write(self._getsegmentforrevs(r, r)[1])
6261
7c8101b5ceb1 revlog: make sure the files are closed after an exception happens
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 6228
diff changeset
2237
35971
0f2c51afafb2 revlog: use context manager for index file lifetime in checkinlinesize
Boris Feld <boris.feld@octobus.net>
parents: 35970
diff changeset
2238 with self._indexfp('w') as fp:
0f2c51afafb2 revlog: use context manager for index file lifetime in checkinlinesize
Boris Feld <boris.feld@octobus.net>
parents: 35970
diff changeset
2239 self.version &= ~FLAG_INLINE_DATA
0f2c51afafb2 revlog: use context manager for index file lifetime in checkinlinesize
Boris Feld <boris.feld@octobus.net>
parents: 35970
diff changeset
2240 self._inline = False
0f2c51afafb2 revlog: use context manager for index file lifetime in checkinlinesize
Boris Feld <boris.feld@octobus.net>
parents: 35970
diff changeset
2241 io = self._io
0f2c51afafb2 revlog: use context manager for index file lifetime in checkinlinesize
Boris Feld <boris.feld@octobus.net>
parents: 35970
diff changeset
2242 for i in self:
0f2c51afafb2 revlog: use context manager for index file lifetime in checkinlinesize
Boris Feld <boris.feld@octobus.net>
parents: 35970
diff changeset
2243 e = io.packentry(self.index[i], self.node, self.version, i)
0f2c51afafb2 revlog: use context manager for index file lifetime in checkinlinesize
Boris Feld <boris.feld@octobus.net>
parents: 35970
diff changeset
2244 fp.write(e)
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
2245
35971
0f2c51afafb2 revlog: use context manager for index file lifetime in checkinlinesize
Boris Feld <boris.feld@octobus.net>
parents: 35970
diff changeset
2246 # the temp file replace the real index when we exit the context
0f2c51afafb2 revlog: use context manager for index file lifetime in checkinlinesize
Boris Feld <boris.feld@octobus.net>
parents: 35970
diff changeset
2247 # manager
2084
Chris Mason <mason@suse.com>
parents: 2082
diff changeset
2248
8650
ef393d6ec030 revlog: refactor chunk cache interface again
Matt Mackall <mpm@selenic.com>
parents: 8643
diff changeset
2249 tr.replace(self.indexfile, trindex * self._io.size)
ef393d6ec030 revlog: refactor chunk cache interface again
Matt Mackall <mpm@selenic.com>
parents: 8643
diff changeset
2250 self._chunkclear()
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
2251
19625
6a411a06cb1f revlog: pass node as an argument of addrevision
Wojciech Lopata <lopek@fb.com>
parents: 19624
diff changeset
2252 def addrevision(self, text, transaction, link, p1, p2, cachedelta=None,
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
2253 node=None, flags=REVIDX_DEFAULT_FLAGS, deltacomputer=None):
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
2254 """add a revision to the log
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
2255
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
2256 text - the revision data to add
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
2257 transaction - the transaction object used for rollback
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
2258 link - the linkrev data to add
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
2259 p1, p2 - the parent nodeids of the revision
12012
bade7a9c5c07 revlog: fix docstring
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 12011
diff changeset
2260 cachedelta - an optional precomputed delta
19625
6a411a06cb1f revlog: pass node as an argument of addrevision
Wojciech Lopata <lopek@fb.com>
parents: 19624
diff changeset
2261 node - nodeid of revision; typically node is not specified, and it is
6a411a06cb1f revlog: pass node as an argument of addrevision
Wojciech Lopata <lopek@fb.com>
parents: 19624
diff changeset
2262 computed by default as hash(text, p1, p2), however subclasses might
6a411a06cb1f revlog: pass node as an argument of addrevision
Wojciech Lopata <lopek@fb.com>
parents: 19624
diff changeset
2263 use different hashing method (and override checkhash() in such case)
30744
e12c0fa1f65b revlog: pass revlog flags to addrevision
Remi Chaintron <remi@fb.com>
parents: 30743
diff changeset
2264 flags - the known flags to set on the revision
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
2265 deltacomputer - an optional _deltacomputer instance shared between
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
2266 multiple calls
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
2267 """
19326
7014526d67a8 revlog: add exception when linkrev == nullrev
Durham Goode <durham@fb.com>
parents: 19200
diff changeset
2268 if link == nullrev:
7014526d67a8 revlog: add exception when linkrev == nullrev
Durham Goode <durham@fb.com>
parents: 19200
diff changeset
2269 raise RevlogError(_("attempted to add linkrev -1 to %s")
7014526d67a8 revlog: add exception when linkrev == nullrev
Durham Goode <durham@fb.com>
parents: 19200
diff changeset
2270 % self.indexfile)
25459
0bda5bfaf0b1 revlog: move size limit check to addrevision
Matt Mackall <mpm@selenic.com>
parents: 25410
diff changeset
2271
30745
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2272 if flags:
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2273 node = node or self.hash(text, p1, p2)
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2274
31750
f319981c24c9 revlog: rename some "text"s to "rawtext"
Jun Wu <quark@fb.com>
parents: 31749
diff changeset
2275 rawtext, validatehash = self._processflags(text, flags, 'write')
30745
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2276
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2277 # If the flag processor modifies the revision data, ignore any provided
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2278 # cachedelta.
31750
f319981c24c9 revlog: rename some "text"s to "rawtext"
Jun Wu <quark@fb.com>
parents: 31749
diff changeset
2279 if rawtext != text:
30745
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2280 cachedelta = None
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2281
31750
f319981c24c9 revlog: rename some "text"s to "rawtext"
Jun Wu <quark@fb.com>
parents: 31749
diff changeset
2282 if len(rawtext) > _maxentrysize:
25459
0bda5bfaf0b1 revlog: move size limit check to addrevision
Matt Mackall <mpm@selenic.com>
parents: 25410
diff changeset
2283 raise RevlogError(
0bda5bfaf0b1 revlog: move size limit check to addrevision
Matt Mackall <mpm@selenic.com>
parents: 25410
diff changeset
2284 _("%s: size of %d bytes exceeds maximum revlog storage of 2GiB")
31750
f319981c24c9 revlog: rename some "text"s to "rawtext"
Jun Wu <quark@fb.com>
parents: 31749
diff changeset
2285 % (self.indexfile, len(rawtext)))
25459
0bda5bfaf0b1 revlog: move size limit check to addrevision
Matt Mackall <mpm@selenic.com>
parents: 25410
diff changeset
2286
31750
f319981c24c9 revlog: rename some "text"s to "rawtext"
Jun Wu <quark@fb.com>
parents: 31749
diff changeset
2287 node = node or self.hash(rawtext, p1, p2)
14196
e7483ec3c374 revlog: remove support for punched/shallow
Sune Foldager <cryo@cyanite.org>
parents: 14195
diff changeset
2288 if node in self.nodemap:
12023
44c22dc193a4 revlog.addrevision(): move computation of nodeid in addrevision()
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 12012
diff changeset
2289 return node
44c22dc193a4 revlog.addrevision(): move computation of nodeid in addrevision()
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 12012
diff changeset
2290
30745
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2291 if validatehash:
31750
f319981c24c9 revlog: rename some "text"s to "rawtext"
Jun Wu <quark@fb.com>
parents: 31749
diff changeset
2292 self.checkhash(rawtext, node, p1=p1, p2=p2)
30745
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
2293
32244
3de4c61b5087 revlog: move part of "addrevision" to "addrawrevision"
Jun Wu <quark@fb.com>
parents: 32229
diff changeset
2294 return self.addrawrevision(rawtext, transaction, link, p1, p2, node,
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
2295 flags, cachedelta=cachedelta,
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
2296 deltacomputer=deltacomputer)
32244
3de4c61b5087 revlog: move part of "addrevision" to "addrawrevision"
Jun Wu <quark@fb.com>
parents: 32229
diff changeset
2297
3de4c61b5087 revlog: move part of "addrevision" to "addrawrevision"
Jun Wu <quark@fb.com>
parents: 32229
diff changeset
2298 def addrawrevision(self, rawtext, transaction, link, p1, p2, node, flags,
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
2299 cachedelta=None, deltacomputer=None):
32244
3de4c61b5087 revlog: move part of "addrevision" to "addrawrevision"
Jun Wu <quark@fb.com>
parents: 32229
diff changeset
2300 """add a raw revision with known flags, node and parents
3de4c61b5087 revlog: move part of "addrevision" to "addrawrevision"
Jun Wu <quark@fb.com>
parents: 32229
diff changeset
2301 useful when reusing a revision not stored in this revlog (ex: received
3de4c61b5087 revlog: move part of "addrevision" to "addrawrevision"
Jun Wu <quark@fb.com>
parents: 32229
diff changeset
2302 over wire, or read from an external bundle).
3de4c61b5087 revlog: move part of "addrevision" to "addrawrevision"
Jun Wu <quark@fb.com>
parents: 32229
diff changeset
2303 """
4981
e7131935fbb3 revlog: simplify addrevision
Matt Mackall <mpm@selenic.com>
parents: 4980
diff changeset
2304 dfh = None
4982
9672e3c42b0c revlog: change _inline from a function to a variable
Matt Mackall <mpm@selenic.com>
parents: 4981
diff changeset
2305 if not self._inline:
35967
61326dd7cb8d revlog: move datafile opening in a method
Boris Feld <boris.feld@octobus.net>
parents: 35845
diff changeset
2306 dfh = self._datafp("a+")
35968
91b3b11565e5 revlog: move index file opening in a method
Boris Feld <boris.feld@octobus.net>
parents: 35967
diff changeset
2307 ifh = self._indexfp("a+")
6261
7c8101b5ceb1 revlog: make sure the files are closed after an exception happens
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 6228
diff changeset
2308 try:
31750
f319981c24c9 revlog: rename some "text"s to "rawtext"
Jun Wu <quark@fb.com>
parents: 31749
diff changeset
2309 return self._addrevision(node, rawtext, transaction, link, p1, p2,
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
2310 flags, cachedelta, ifh, dfh,
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
2311 deltacomputer=deltacomputer)
6261
7c8101b5ceb1 revlog: make sure the files are closed after an exception happens
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 6228
diff changeset
2312 finally:
7c8101b5ceb1 revlog: make sure the files are closed after an exception happens
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 6228
diff changeset
2313 if dfh:
7c8101b5ceb1 revlog: make sure the files are closed after an exception happens
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 6228
diff changeset
2314 dfh.close()
7c8101b5ceb1 revlog: make sure the files are closed after an exception happens
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 6228
diff changeset
2315 ifh.close()
3390
a74addddd092 make revlog.addgroup pass its file handles to addrevision
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 3360
diff changeset
2316
30795
78ac56aebab6 revlog: use compression engine API for compression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30793
diff changeset
2317 def compress(self, data):
78ac56aebab6 revlog: use compression engine API for compression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30793
diff changeset
2318 """Generate a possibly-compressed representation of data."""
78ac56aebab6 revlog: use compression engine API for compression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30793
diff changeset
2319 if not data:
78ac56aebab6 revlog: use compression engine API for compression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30793
diff changeset
2320 return '', data
78ac56aebab6 revlog: use compression engine API for compression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30793
diff changeset
2321
78ac56aebab6 revlog: use compression engine API for compression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30793
diff changeset
2322 compressed = self._compressor.compress(data)
78ac56aebab6 revlog: use compression engine API for compression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30793
diff changeset
2323
78ac56aebab6 revlog: use compression engine API for compression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30793
diff changeset
2324 if compressed:
78ac56aebab6 revlog: use compression engine API for compression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30793
diff changeset
2325 # The revlog compressor added the header in the returned data.
78ac56aebab6 revlog: use compression engine API for compression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30793
diff changeset
2326 return '', compressed
78ac56aebab6 revlog: use compression engine API for compression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30793
diff changeset
2327
31643
6ceb3c4c3ab6 py3: fix slicing of byte string in revlog.compress()
Yuya Nishihara <yuya@tcha.org>
parents: 31574
diff changeset
2328 if data[0:1] == '\0':
30795
78ac56aebab6 revlog: use compression engine API for compression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30793
diff changeset
2329 return '', data
78ac56aebab6 revlog: use compression engine API for compression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30793
diff changeset
2330 return 'u', data
17128
1028a1c9077a revlog: make compress a method
Bryan O'Sullivan <bryano@fb.com>
parents: 17009
diff changeset
2331
30793
b6f455a6e4d6 revlog: move decompress() from module to revlog class (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30792
diff changeset
2332 def decompress(self, data):
b6f455a6e4d6 revlog: move decompress() from module to revlog class (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30792
diff changeset
2333 """Decompress a revlog chunk.
b6f455a6e4d6 revlog: move decompress() from module to revlog class (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30792
diff changeset
2334
b6f455a6e4d6 revlog: move decompress() from module to revlog class (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30792
diff changeset
2335 The chunk is expected to begin with a header identifying the
b6f455a6e4d6 revlog: move decompress() from module to revlog class (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30792
diff changeset
2336 format type so it can be routed to an appropriate decompressor.
b6f455a6e4d6 revlog: move decompress() from module to revlog class (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30792
diff changeset
2337 """
b6f455a6e4d6 revlog: move decompress() from module to revlog class (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30792
diff changeset
2338 if not data:
b6f455a6e4d6 revlog: move decompress() from module to revlog class (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30792
diff changeset
2339 return data
30817
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
2340
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
2341 # Revlogs are read much more frequently than they are written and many
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
2342 # chunks only take microseconds to decompress, so performance is
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
2343 # important here.
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
2344 #
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
2345 # We can make a few assumptions about revlogs:
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
2346 #
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
2347 # 1) the majority of chunks will be compressed (as opposed to inline
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
2348 # raw data).
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
2349 # 2) decompressing *any* data will likely by at least 10x slower than
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
2350 # returning raw inline data.
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
2351 # 3) we want to prioritize common and officially supported compression
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
2352 # engines
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
2353 #
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
2354 # It follows that we want to optimize for "decompress compressed data
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
2355 # when encoded with common and officially supported compression engines"
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
2356 # case over "raw data" and "data encoded by less common or non-official
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
2357 # compression engines." That is why we have the inline lookup first
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
2358 # followed by the compengines lookup.
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
2359 #
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
2360 # According to `hg perfrevlogchunks`, this is ~0.5% faster for zlib
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
2361 # compressed chunks. And this matters for changelog and manifest reads.
31356
ef6888172437 revlog: extract first byte of revlog with a slice so it's portable
Augie Fackler <augie@google.com>
parents: 30829
diff changeset
2362 t = data[0:1]
30817
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
2363
30793
b6f455a6e4d6 revlog: move decompress() from module to revlog class (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30792
diff changeset
2364 if t == 'x':
b6f455a6e4d6 revlog: move decompress() from module to revlog class (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30792
diff changeset
2365 try:
30817
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
2366 return _zlibdecompress(data)
30793
b6f455a6e4d6 revlog: move decompress() from module to revlog class (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30792
diff changeset
2367 except zlib.error as e:
36702
f659a407e5ee py3: use util.forcebytestr instead of str to convert error messages
Pulkit Goyal <7895pulkit@gmail.com>
parents: 36238
diff changeset
2368 raise RevlogError(_('revlog decompress error: %s') %
37084
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module
Yuya Nishihara <yuya@tcha.org>
parents: 36760
diff changeset
2369 stringutil.forcebytestr(e))
30817
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
2370 # '\0' is more common than 'u' so it goes first.
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
2371 elif t == '\0':
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
2372 return data
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
2373 elif t == 'u':
30793
b6f455a6e4d6 revlog: move decompress() from module to revlog class (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30792
diff changeset
2374 return util.buffer(data, 1)
30817
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
2375
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
2376 try:
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
2377 compressor = self._decompressors[t]
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
2378 except KeyError:
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
2379 try:
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
2380 engine = util.compengines.forrevlogheader(t)
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
2381 compressor = engine.revlogcompressor()
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
2382 self._decompressors[t] = compressor
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
2383 except KeyError:
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
2384 raise RevlogError(_('unknown compression type %r') % t)
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
2385
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
2386 return compressor.decompress(data)
30793
b6f455a6e4d6 revlog: move decompress() from module to revlog class (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30792
diff changeset
2387
38117
69ec6f98cfa6 revlog: isgooddeltainfo takes the whole revinfo object
Paul Morelle <paul.morelle@octobus.net>
parents: 38116
diff changeset
2388 def _isgooddeltainfo(self, deltainfo, revinfo):
26115
748347e0e8d4 revlog: move delta check to it's own function
Durham Goode <durham@fb.com>
parents: 25892
diff changeset
2389 """Returns True if the given delta is good. Good means that it is within
748347e0e8d4 revlog: move delta check to it's own function
Durham Goode <durham@fb.com>
parents: 25892
diff changeset
2390 the disk span, disk size, and chain length bounds that we know to be
748347e0e8d4 revlog: move delta check to it's own function
Durham Goode <durham@fb.com>
parents: 25892
diff changeset
2391 performant."""
38116
bf59f95583c1 revlog: make variable name 'd' more explicit in _isgooddeltainfo
Paul Morelle <paul.morelle@octobus.net>
parents: 38114
diff changeset
2392 if deltainfo is None:
26115
748347e0e8d4 revlog: move delta check to it's own function
Durham Goode <durham@fb.com>
parents: 25892
diff changeset
2393 return False
748347e0e8d4 revlog: move delta check to it's own function
Durham Goode <durham@fb.com>
parents: 25892
diff changeset
2394
38116
bf59f95583c1 revlog: make variable name 'd' more explicit in _isgooddeltainfo
Paul Morelle <paul.morelle@octobus.net>
parents: 38114
diff changeset
2395 # - 'deltainfo.distance' is the distance from the base revision --
bf59f95583c1 revlog: make variable name 'd' more explicit in _isgooddeltainfo
Paul Morelle <paul.morelle@octobus.net>
parents: 38114
diff changeset
2396 # bounding it limits the amount of I/O we need to do.
bf59f95583c1 revlog: make variable name 'd' more explicit in _isgooddeltainfo
Paul Morelle <paul.morelle@octobus.net>
parents: 38114
diff changeset
2397 # - 'deltainfo.compresseddeltalen' is the sum of the total size of
bf59f95583c1 revlog: make variable name 'd' more explicit in _isgooddeltainfo
Paul Morelle <paul.morelle@octobus.net>
parents: 38114
diff changeset
2398 # deltas we need to apply -- bounding it limits the amount of CPU
bf59f95583c1 revlog: make variable name 'd' more explicit in _isgooddeltainfo
Paul Morelle <paul.morelle@octobus.net>
parents: 38114
diff changeset
2399 # we consume.
33207
895ecec31c70 revlog: add an experimental option to mitigated delta issues (issue5480)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 33171
diff changeset
2400
38716
c67093e81a3e revlog: extract `deltainfo.distance` for future conditional redefinition
Paul Morelle <paul.morelle@octobus.net>
parents: 38710
diff changeset
2401 distance = deltainfo.distance
38117
69ec6f98cfa6 revlog: isgooddeltainfo takes the whole revinfo object
Paul Morelle <paul.morelle@octobus.net>
parents: 38116
diff changeset
2402 textlen = revinfo.textlen
33207
895ecec31c70 revlog: add an experimental option to mitigated delta issues (issue5480)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 33171
diff changeset
2403 defaultmax = textlen * 4
895ecec31c70 revlog: add an experimental option to mitigated delta issues (issue5480)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 33171
diff changeset
2404 maxdist = self._maxdeltachainspan
895ecec31c70 revlog: add an experimental option to mitigated delta issues (issue5480)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 33171
diff changeset
2405 if not maxdist:
38716
c67093e81a3e revlog: extract `deltainfo.distance` for future conditional redefinition
Paul Morelle <paul.morelle@octobus.net>
parents: 38710
diff changeset
2406 maxdist = distance # ensure the conditional pass
33207
895ecec31c70 revlog: add an experimental option to mitigated delta issues (issue5480)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 33171
diff changeset
2407 maxdist = max(maxdist, defaultmax)
38716
c67093e81a3e revlog: extract `deltainfo.distance` for future conditional redefinition
Paul Morelle <paul.morelle@octobus.net>
parents: 38710
diff changeset
2408 if (distance > maxdist or deltainfo.deltalen > textlen or
38116
bf59f95583c1 revlog: make variable name 'd' more explicit in _isgooddeltainfo
Paul Morelle <paul.morelle@octobus.net>
parents: 38114
diff changeset
2409 deltainfo.compresseddeltalen > textlen * 2 or
bf59f95583c1 revlog: make variable name 'd' more explicit in _isgooddeltainfo
Paul Morelle <paul.morelle@octobus.net>
parents: 38114
diff changeset
2410 (self._maxchainlen and deltainfo.chainlen > self._maxchainlen)):
26115
748347e0e8d4 revlog: move delta check to it's own function
Durham Goode <durham@fb.com>
parents: 25892
diff changeset
2411 return False
748347e0e8d4 revlog: move delta check to it's own function
Durham Goode <durham@fb.com>
parents: 25892
diff changeset
2412
748347e0e8d4 revlog: move delta check to it's own function
Durham Goode <durham@fb.com>
parents: 25892
diff changeset
2413 return True
748347e0e8d4 revlog: move delta check to it's own function
Durham Goode <durham@fb.com>
parents: 25892
diff changeset
2414
31755
ec48d57de110 revlog: make _addrevision only accept rawtext
Jun Wu <quark@fb.com>
parents: 31754
diff changeset
2415 def _addrevision(self, node, rawtext, transaction, link, p1, p2, flags,
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
2416 cachedelta, ifh, dfh, alwayscache=False,
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
2417 deltacomputer=None):
14292
c97d8485b5fa revlog: add docstring to _addrevision
Sune Foldager <cryo@cyanite.org>
parents: 14270
diff changeset
2418 """internal function to add revisions to the log
12623
8f97b50a8d10 revlog._addrevision(): allow text argument to be None, build it lazily
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 12336
diff changeset
2419
14292
c97d8485b5fa revlog: add docstring to _addrevision
Sune Foldager <cryo@cyanite.org>
parents: 14270
diff changeset
2420 see addrevision for argument descriptions.
31755
ec48d57de110 revlog: make _addrevision only accept rawtext
Jun Wu <quark@fb.com>
parents: 31754
diff changeset
2421
ec48d57de110 revlog: make _addrevision only accept rawtext
Jun Wu <quark@fb.com>
parents: 31754
diff changeset
2422 note: "addrevision" takes non-raw text, "_addrevision" takes raw text.
ec48d57de110 revlog: make _addrevision only accept rawtext
Jun Wu <quark@fb.com>
parents: 31754
diff changeset
2423
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
2424 if "deltacomputer" is not provided or None, a defaultdeltacomputer will
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
2425 be used.
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
2426
14292
c97d8485b5fa revlog: add docstring to _addrevision
Sune Foldager <cryo@cyanite.org>
parents: 14270
diff changeset
2427 invariants:
31755
ec48d57de110 revlog: make _addrevision only accept rawtext
Jun Wu <quark@fb.com>
parents: 31754
diff changeset
2428 - rawtext is optional (can be None); if not set, cachedelta must be set.
17424
e7cfe3587ea4 fix trivial spelling errors
Mads Kiilerich <mads@kiilerich.com>
parents: 17150
diff changeset
2429 if both are set, they must correspond to each other.
14292
c97d8485b5fa revlog: add docstring to _addrevision
Sune Foldager <cryo@cyanite.org>
parents: 14270
diff changeset
2430 """
33938
9180f8f593f3 revlog: abort on attempt to write null revision
Martin von Zweigbergk <martinvonz@google.com>
parents: 33392
diff changeset
2431 if node == nullid:
9180f8f593f3 revlog: abort on attempt to write null revision
Martin von Zweigbergk <martinvonz@google.com>
parents: 33392
diff changeset
2432 raise RevlogError(_("%s: attempt to add null revision") %
9180f8f593f3 revlog: abort on attempt to write null revision
Martin von Zweigbergk <martinvonz@google.com>
parents: 33392
diff changeset
2433 (self.indexfile))
37449
a0d71618074f revlog: detect pseudo file nodeids to raise WdirUnsupported exception
Yuya Nishihara <yuya@tcha.org>
parents: 37443
diff changeset
2434 if node == wdirid or node in wdirfilenodeids:
34028
bfb38c5cebf4 revlog: move check for wdir from changelog to revlog
Martin von Zweigbergk <martinvonz@google.com>
parents: 34026
diff changeset
2435 raise RevlogError(_("%s: attempt to add wdir revision") %
bfb38c5cebf4 revlog: move check for wdir from changelog to revlog
Martin von Zweigbergk <martinvonz@google.com>
parents: 34026
diff changeset
2436 (self.indexfile))
bfb38c5cebf4 revlog: move check for wdir from changelog to revlog
Martin von Zweigbergk <martinvonz@google.com>
parents: 34026
diff changeset
2437
35635
0b52c0ecbc23 revlog: choose between ifh and dfh once for all
Paul Morelle <paul.morelle@octobus.net>
parents: 35634
diff changeset
2438 if self._inline:
0b52c0ecbc23 revlog: choose between ifh and dfh once for all
Paul Morelle <paul.morelle@octobus.net>
parents: 35634
diff changeset
2439 fh = ifh
0b52c0ecbc23 revlog: choose between ifh and dfh once for all
Paul Morelle <paul.morelle@octobus.net>
parents: 35634
diff changeset
2440 else:
0b52c0ecbc23 revlog: choose between ifh and dfh once for all
Paul Morelle <paul.morelle@octobus.net>
parents: 35634
diff changeset
2441 fh = dfh
0b52c0ecbc23 revlog: choose between ifh and dfh once for all
Paul Morelle <paul.morelle@octobus.net>
parents: 35634
diff changeset
2442
31755
ec48d57de110 revlog: make _addrevision only accept rawtext
Jun Wu <quark@fb.com>
parents: 31754
diff changeset
2443 btext = [rawtext]
12623
8f97b50a8d10 revlog._addrevision(): allow text argument to be None, build it lazily
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 12336
diff changeset
2444
6750
fb42030d79d6 add __len__ and __iter__ methods to repo and revlog
Matt Mackall <mpm@selenic.com>
parents: 6703
diff changeset
2445 curr = len(self)
4981
e7131935fbb3 revlog: simplify addrevision
Matt Mackall <mpm@selenic.com>
parents: 4980
diff changeset
2446 prev = curr - 1
e7131935fbb3 revlog: simplify addrevision
Matt Mackall <mpm@selenic.com>
parents: 4980
diff changeset
2447 offset = self.end(prev)
12889
5482c6b826f4 revlog: precalculate p1 and p2 revisions
Matt Mackall <mpm@selenic.com>
parents: 12888
diff changeset
2448 p1r, p2r = self.rev(p1), self.rev(p2)
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
2449
26116
562cfc99e611 revlog: move textlen calculation to be above delta chooser
Durham Goode <durham@fb.com>
parents: 26115
diff changeset
2450 # full versions are inserted when the needed deltas
562cfc99e611 revlog: move textlen calculation to be above delta chooser
Durham Goode <durham@fb.com>
parents: 26115
diff changeset
2451 # become comparable to the uncompressed text
31755
ec48d57de110 revlog: make _addrevision only accept rawtext
Jun Wu <quark@fb.com>
parents: 31754
diff changeset
2452 if rawtext is None:
36748
369aadf7a326 revlog: resolve lfs rawtext to vanilla rawtext before applying delta
Jun Wu <quark@fb.com>
parents: 36744
diff changeset
2453 # need rawtext size, before changed by flag processors, which is
369aadf7a326 revlog: resolve lfs rawtext to vanilla rawtext before applying delta
Jun Wu <quark@fb.com>
parents: 36744
diff changeset
2454 # the non-raw size. use revlog explicitly to avoid filelog's extra
369aadf7a326 revlog: resolve lfs rawtext to vanilla rawtext before applying delta
Jun Wu <quark@fb.com>
parents: 36744
diff changeset
2455 # logic that might remove metadata size.
369aadf7a326 revlog: resolve lfs rawtext to vanilla rawtext before applying delta
Jun Wu <quark@fb.com>
parents: 36744
diff changeset
2456 textlen = mdiff.patchedsize(revlog.size(self, cachedelta[0]),
26116
562cfc99e611 revlog: move textlen calculation to be above delta chooser
Durham Goode <durham@fb.com>
parents: 26115
diff changeset
2457 cachedelta[1])
562cfc99e611 revlog: move textlen calculation to be above delta chooser
Durham Goode <durham@fb.com>
parents: 26115
diff changeset
2458 else:
31755
ec48d57de110 revlog: make _addrevision only accept rawtext
Jun Wu <quark@fb.com>
parents: 31754
diff changeset
2459 textlen = len(rawtext)
26116
562cfc99e611 revlog: move textlen calculation to be above delta chooser
Durham Goode <durham@fb.com>
parents: 26115
diff changeset
2460
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
2461 if deltacomputer is None:
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
2462 deltacomputer = _deltacomputer(self)
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
2463
35737
d99b07bc69fb revlog: refactor out _finddeltainfo from _addrevision
Paul Morelle <paul.morelle@octobus.net>
parents: 35641
diff changeset
2464 revinfo = _revisioninfo(node, p1, p2, btext, textlen, cachedelta, flags)
36744
33275ab5e837 revlog: do not use delta for lfs revisions
Jun Wu <quark@fb.com>
parents: 36743
diff changeset
2465
33275ab5e837 revlog: do not use delta for lfs revisions
Jun Wu <quark@fb.com>
parents: 36743
diff changeset
2466 # no delta for flag processor revision (see "candelta" for why)
33275ab5e837 revlog: do not use delta for lfs revisions
Jun Wu <quark@fb.com>
parents: 36743
diff changeset
2467 # not calling candelta since only one revision needs test, also to
33275ab5e837 revlog: do not use delta for lfs revisions
Jun Wu <quark@fb.com>
parents: 36743
diff changeset
2468 # avoid overhead fetching flags again.
33275ab5e837 revlog: do not use delta for lfs revisions
Jun Wu <quark@fb.com>
parents: 36743
diff changeset
2469 if flags & REVIDX_RAWTEXT_CHANGING_FLAGS:
33275ab5e837 revlog: do not use delta for lfs revisions
Jun Wu <quark@fb.com>
parents: 36743
diff changeset
2470 deltainfo = None
33275ab5e837 revlog: do not use delta for lfs revisions
Jun Wu <quark@fb.com>
parents: 36743
diff changeset
2471 else:
33275ab5e837 revlog: do not use delta for lfs revisions
Jun Wu <quark@fb.com>
parents: 36743
diff changeset
2472 deltainfo = deltacomputer.finddeltainfo(revinfo, fh)
35634
b43578ec483a revlog: refactor out the selection of candidate revisions
Paul Morelle <paul.morelle@octobus.net>
parents: 35521
diff changeset
2473
35638
edc9330acac1 revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents: 35637
diff changeset
2474 if deltainfo is not None:
edc9330acac1 revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents: 35637
diff changeset
2475 base = deltainfo.base
edc9330acac1 revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents: 35637
diff changeset
2476 chainbase = deltainfo.chainbase
edc9330acac1 revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents: 35637
diff changeset
2477 data = deltainfo.data
edc9330acac1 revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents: 35637
diff changeset
2478 l = deltainfo.deltalen
27250
bff71fe05768 revlog: make calls to _isgooddelta() consistent
Martin von Zweigbergk <martinvonz@google.com>
parents: 27249
diff changeset
2479 else:
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
2480 rawtext = deltacomputer.buildtext(revinfo, fh)
31755
ec48d57de110 revlog: make _addrevision only accept rawtext
Jun Wu <quark@fb.com>
parents: 31754
diff changeset
2481 data = self.compress(rawtext)
1533
3d11f81c9145 Reduce string duplication in compression code
mason@suse.com
parents: 1509
diff changeset
2482 l = len(data[1]) + len(data[0])
14296
62e25c63fb3a revlog: fix bug in chainbase cache
Sune Foldager <cryo@cyanite.org>
parents: 14292
diff changeset
2483 base = chainbase = curr
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
2484
12623
8f97b50a8d10 revlog._addrevision(): allow text argument to be None, build it lazily
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 12336
diff changeset
2485 e = (offset_type(offset, flags), l, textlen,
12889
5482c6b826f4 revlog: precalculate p1 and p2 revisions
Matt Mackall <mpm@selenic.com>
parents: 12888
diff changeset
2486 base, link, p1r, p2r, node)
4979
06abdaf78788 revlog: add a magic null revision to our index
Matt Mackall <mpm@selenic.com>
parents: 4978
diff changeset
2487 self.index.insert(-1, e)
4981
e7131935fbb3 revlog: simplify addrevision
Matt Mackall <mpm@selenic.com>
parents: 4980
diff changeset
2488 self.nodemap[node] = curr
4977
6cb30bc4ca32 revlog: parse revlogv0 indexes into v1 internally
Matt Mackall <mpm@selenic.com>
parents: 4976
diff changeset
2489
5338
f87685355c9c revlog: fix revlogio.packentry corner case
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5325
diff changeset
2490 entry = self._io.packentry(e, self.node, self.version, curr)
20217
33394f2e331e revlog: move file writing to a separate function
Durham Goode <durham@fb.com>
parents: 20180
diff changeset
2491 self._writeentry(transaction, ifh, dfh, entry, data, link, offset)
33394f2e331e revlog: move file writing to a separate function
Durham Goode <durham@fb.com>
parents: 20180
diff changeset
2492
31755
ec48d57de110 revlog: make _addrevision only accept rawtext
Jun Wu <quark@fb.com>
parents: 31754
diff changeset
2493 if alwayscache and rawtext is None:
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
2494 rawtext = deltacomputer._buildtext(revinfo, fh)
26243
836291420d53 revlog: optionally cache the full text when adding revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26242
diff changeset
2495
35845
be923ce44d6a revlog: correct type in check to verify rawtext is immutable
Augie Fackler <augie@google.com>
parents: 35738
diff changeset
2496 if type(rawtext) == bytes: # only accept immutable objects
31755
ec48d57de110 revlog: make _addrevision only accept rawtext
Jun Wu <quark@fb.com>
parents: 31754
diff changeset
2497 self._cache = (node, curr, rawtext)
29830
92ac2baaea86 revlog: use an LRU cache for delta chain bases
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29829
diff changeset
2498 self._chainbasecache[curr] = chainbase
20217
33394f2e331e revlog: move file writing to a separate function
Durham Goode <durham@fb.com>
parents: 20180
diff changeset
2499 return node
33394f2e331e revlog: move file writing to a separate function
Durham Goode <durham@fb.com>
parents: 20180
diff changeset
2500
33394f2e331e revlog: move file writing to a separate function
Durham Goode <durham@fb.com>
parents: 20180
diff changeset
2501 def _writeentry(self, transaction, ifh, dfh, entry, data, link, offset):
27430
e240e914d226 revlog: seek to end of file before writing (issue4943)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26705
diff changeset
2502 # Files opened in a+ mode have inconsistent behavior on various
e240e914d226 revlog: seek to end of file before writing (issue4943)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26705
diff changeset
2503 # platforms. Windows requires that a file positioning call be made
e240e914d226 revlog: seek to end of file before writing (issue4943)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26705
diff changeset
2504 # when the file handle transitions between reads and writes. See
e240e914d226 revlog: seek to end of file before writing (issue4943)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26705
diff changeset
2505 # 3686fa2b8eee and the mixedfilemodewrapper in windows.py. On other
e240e914d226 revlog: seek to end of file before writing (issue4943)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26705
diff changeset
2506 # platforms, Python or the platform itself can be buggy. Some versions
e240e914d226 revlog: seek to end of file before writing (issue4943)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26705
diff changeset
2507 # of Solaris have been observed to not append at the end of the file
e240e914d226 revlog: seek to end of file before writing (issue4943)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26705
diff changeset
2508 # if the file was seeked to before the end. See issue4943 for more.
e240e914d226 revlog: seek to end of file before writing (issue4943)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26705
diff changeset
2509 #
e240e914d226 revlog: seek to end of file before writing (issue4943)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26705
diff changeset
2510 # We work around this issue by inserting a seek() before writing.
e240e914d226 revlog: seek to end of file before writing (issue4943)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26705
diff changeset
2511 # Note: This is likely not necessary on Python 3.
e240e914d226 revlog: seek to end of file before writing (issue4943)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26705
diff changeset
2512 ifh.seek(0, os.SEEK_END)
27441
e47841c8343d revlog: fix bad indentation (replace tab by space)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27430
diff changeset
2513 if dfh:
27430
e240e914d226 revlog: seek to end of file before writing (issue4943)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26705
diff changeset
2514 dfh.seek(0, os.SEEK_END)
e240e914d226 revlog: seek to end of file before writing (issue4943)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26705
diff changeset
2515
20217
33394f2e331e revlog: move file writing to a separate function
Durham Goode <durham@fb.com>
parents: 20180
diff changeset
2516 curr = len(self) - 1
4982
9672e3c42b0c revlog: change _inline from a function to a variable
Matt Mackall <mpm@selenic.com>
parents: 4981
diff changeset
2517 if not self._inline:
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
2518 transaction.add(self.datafile, offset)
4981
e7131935fbb3 revlog: simplify addrevision
Matt Mackall <mpm@selenic.com>
parents: 4980
diff changeset
2519 transaction.add(self.indexfile, curr * len(entry))
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
2520 if data[0]:
3390
a74addddd092 make revlog.addgroup pass its file handles to addrevision
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 3360
diff changeset
2521 dfh.write(data[0])
a74addddd092 make revlog.addgroup pass its file handles to addrevision
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 3360
diff changeset
2522 dfh.write(data[1])
4981
e7131935fbb3 revlog: simplify addrevision
Matt Mackall <mpm@selenic.com>
parents: 4980
diff changeset
2523 ifh.write(entry)
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
2524 else:
4996
a0d37976cd5b revlog: avoid some unnecessary seek/tell syscalls
Matt Mackall <mpm@selenic.com>
parents: 4994
diff changeset
2525 offset += curr * self._io.size
5324
8409a2e3a78d revlog: fix inlined revision transaction extra data (issue 749)
Patrick Mezard <pmezard@gmail.com>
parents: 5007
diff changeset
2526 transaction.add(self.indexfile, offset, curr)
4981
e7131935fbb3 revlog: simplify addrevision
Matt Mackall <mpm@selenic.com>
parents: 4980
diff changeset
2527 ifh.write(entry)
3390
a74addddd092 make revlog.addgroup pass its file handles to addrevision
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 3360
diff changeset
2528 ifh.write(data[0])
a74addddd092 make revlog.addgroup pass its file handles to addrevision
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 3360
diff changeset
2529 ifh.write(data[1])
35974
9ba1d0c724e2 revlog: rename 'self.checkinlinesize' into '_enforceinlinesize'
Boris Feld <boris.feld@octobus.net>
parents: 35973
diff changeset
2530 self._enforceinlinesize(transaction, ifh)
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
2531
34291
1db9abf407c5 revlog: add revmap back to revlog.addgroup
Durham Goode <durham@fb.com>
parents: 34250
diff changeset
2532 def addgroup(self, deltas, linkmapper, transaction, addrevisioncb=None):
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
2533 """
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
2534 add a delta group
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
2535
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
2536 given a set of deltas, add them to the revision log. the
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
2537 first delta is against its parent, which should be in our
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
2538 log, the rest are against the previous delta.
25822
00e3f909907f revlog: add support for a callback whenever revisions are added
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
2539
00e3f909907f revlog: add support for a callback whenever revisions are added
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
2540 If ``addrevisioncb`` is defined, it will be called with arguments of
00e3f909907f revlog: add support for a callback whenever revisions are added
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
2541 this revlog and the node that was added.
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
2542 """
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
2543
32868
ef015ba5ba2e revlog: rename list of nodes from "content" to "nodes"
Martin von Zweigbergk <martinvonz@google.com>
parents: 32867
diff changeset
2544 nodes = []
515
03f27b1381f9 Whitespace cleanups
mpm@selenic.com
parents: 484
diff changeset
2545
12624
557988c691d1 revlog.addgroup(): always use _addrevision() to add new revlog entries
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 12623
diff changeset
2546 r = len(self)
557988c691d1 revlog.addgroup(): always use _addrevision() to add new revlog entries
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 12623
diff changeset
2547 end = 0
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
2548 if r:
12624
557988c691d1 revlog.addgroup(): always use _addrevision() to add new revlog entries
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 12623
diff changeset
2549 end = self.end(r - 1)
35968
91b3b11565e5 revlog: move index file opening in a method
Boris Feld <boris.feld@octobus.net>
parents: 35967
diff changeset
2550 ifh = self._indexfp("a+")
4996
a0d37976cd5b revlog: avoid some unnecessary seek/tell syscalls
Matt Mackall <mpm@selenic.com>
parents: 4994
diff changeset
2551 isize = r * self._io.size
4982
9672e3c42b0c revlog: change _inline from a function to a variable
Matt Mackall <mpm@selenic.com>
parents: 4981
diff changeset
2552 if self._inline:
4996
a0d37976cd5b revlog: avoid some unnecessary seek/tell syscalls
Matt Mackall <mpm@selenic.com>
parents: 4994
diff changeset
2553 transaction.add(self.indexfile, end + isize, r)
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
2554 dfh = None
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
2555 else:
4996
a0d37976cd5b revlog: avoid some unnecessary seek/tell syscalls
Matt Mackall <mpm@selenic.com>
parents: 4994
diff changeset
2556 transaction.add(self.indexfile, isize, r)
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
2557 transaction.add(self.datafile, end)
35967
61326dd7cb8d revlog: move datafile opening in a method
Boris Feld <boris.feld@octobus.net>
parents: 35845
diff changeset
2558 dfh = self._datafp("a+")
24255
4bfe9f2d9761 revlog: addgroup checks if incoming deltas add censored revs, sets flag bit
Mike Edgar <adgar@google.com>
parents: 24123
diff changeset
2559 def flush():
4bfe9f2d9761 revlog: addgroup checks if incoming deltas add censored revs, sets flag bit
Mike Edgar <adgar@google.com>
parents: 24123
diff changeset
2560 if dfh:
4bfe9f2d9761 revlog: addgroup checks if incoming deltas add censored revs, sets flag bit
Mike Edgar <adgar@google.com>
parents: 24123
diff changeset
2561 dfh.flush()
4bfe9f2d9761 revlog: addgroup checks if incoming deltas add censored revs, sets flag bit
Mike Edgar <adgar@google.com>
parents: 24123
diff changeset
2562 ifh.flush()
6261
7c8101b5ceb1 revlog: make sure the files are closed after an exception happens
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 6228
diff changeset
2563 try:
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
2564 deltacomputer = _deltacomputer(self)
6261
7c8101b5ceb1 revlog: make sure the files are closed after an exception happens
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 6228
diff changeset
2565 # loop through our set of deltas
34148
c8b6ed51386b changegroup: remove changegroup dependency from revlog.addgroup
Durham Goode <durham@fb.com>
parents: 34147
diff changeset
2566 for data in deltas:
34291
1db9abf407c5 revlog: add revmap back to revlog.addgroup
Durham Goode <durham@fb.com>
parents: 34250
diff changeset
2567 node, p1, p2, linknode, deltabase, delta, flags = data
1db9abf407c5 revlog: add revmap back to revlog.addgroup
Durham Goode <durham@fb.com>
parents: 34250
diff changeset
2568 link = linkmapper(linknode)
34148
c8b6ed51386b changegroup: remove changegroup dependency from revlog.addgroup
Durham Goode <durham@fb.com>
parents: 34147
diff changeset
2569 flags = flags or REVIDX_DEFAULT_FLAGS
12336
9d234f7d8a77 bundle: move chunk parsing into unbundle class
Matt Mackall <mpm@selenic.com>
parents: 12335
diff changeset
2570
32868
ef015ba5ba2e revlog: rename list of nodes from "content" to "nodes"
Martin von Zweigbergk <martinvonz@google.com>
parents: 32867
diff changeset
2571 nodes.append(node)
15890
e234eda20984 revlog: make addgroup returns a list of node contained in the added source
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 15835
diff changeset
2572
14196
e7483ec3c374 revlog: remove support for punched/shallow
Sune Foldager <cryo@cyanite.org>
parents: 14195
diff changeset
2573 if node in self.nodemap:
6261
7c8101b5ceb1 revlog: make sure the files are closed after an exception happens
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 6228
diff changeset
2574 # this can happen if two branches make the same change
7c8101b5ceb1 revlog: make sure the files are closed after an exception happens
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 6228
diff changeset
2575 continue
192
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
2576
6261
7c8101b5ceb1 revlog: make sure the files are closed after an exception happens
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 6228
diff changeset
2577 for p in (p1, p2):
16686
67964cda8701 cleanup: "not x in y" -> "x not in y"
Brodie Rao <brodie@sf.io>
parents: 16665
diff changeset
2578 if p not in self.nodemap:
14196
e7483ec3c374 revlog: remove support for punched/shallow
Sune Foldager <cryo@cyanite.org>
parents: 14195
diff changeset
2579 raise LookupError(p, self.indexfile,
e7483ec3c374 revlog: remove support for punched/shallow
Sune Foldager <cryo@cyanite.org>
parents: 14195
diff changeset
2580 _('unknown parent'))
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
2581
14141
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14075
diff changeset
2582 if deltabase not in self.nodemap:
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14075
diff changeset
2583 raise LookupError(deltabase, self.indexfile,
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14075
diff changeset
2584 _('unknown delta base'))
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
2585
14141
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14075
diff changeset
2586 baserev = self.rev(deltabase)
24120
a450e0a2ba0a revlog: in addgroup, reject ill-formed deltas based on censored nodes
Mike Edgar <adgar@google.com>
parents: 24118
diff changeset
2587
a450e0a2ba0a revlog: in addgroup, reject ill-formed deltas based on censored nodes
Mike Edgar <adgar@google.com>
parents: 24118
diff changeset
2588 if baserev != nullrev and self.iscensored(baserev):
a450e0a2ba0a revlog: in addgroup, reject ill-formed deltas based on censored nodes
Mike Edgar <adgar@google.com>
parents: 24118
diff changeset
2589 # if base is censored, delta must be full replacement in a
a450e0a2ba0a revlog: in addgroup, reject ill-formed deltas based on censored nodes
Mike Edgar <adgar@google.com>
parents: 24118
diff changeset
2590 # single patch operation
a450e0a2ba0a revlog: in addgroup, reject ill-formed deltas based on censored nodes
Mike Edgar <adgar@google.com>
parents: 24118
diff changeset
2591 hlen = struct.calcsize(">lll")
a450e0a2ba0a revlog: in addgroup, reject ill-formed deltas based on censored nodes
Mike Edgar <adgar@google.com>
parents: 24118
diff changeset
2592 oldlen = self.rawsize(baserev)
a450e0a2ba0a revlog: in addgroup, reject ill-formed deltas based on censored nodes
Mike Edgar <adgar@google.com>
parents: 24118
diff changeset
2593 newlen = len(delta) - hlen
a450e0a2ba0a revlog: in addgroup, reject ill-formed deltas based on censored nodes
Mike Edgar <adgar@google.com>
parents: 24118
diff changeset
2594 if delta[:hlen] != mdiff.replacediffheader(oldlen, newlen):
a450e0a2ba0a revlog: in addgroup, reject ill-formed deltas based on censored nodes
Mike Edgar <adgar@google.com>
parents: 24118
diff changeset
2595 raise error.CensoredBaseError(self.indexfile,
a450e0a2ba0a revlog: in addgroup, reject ill-formed deltas based on censored nodes
Mike Edgar <adgar@google.com>
parents: 24118
diff changeset
2596 self.node(baserev))
a450e0a2ba0a revlog: in addgroup, reject ill-formed deltas based on censored nodes
Mike Edgar <adgar@google.com>
parents: 24118
diff changeset
2597
27433
12f727a5b434 changegroup: add flags field to cg3 delta header
Mike Edgar <adgar@google.com>
parents: 27431
diff changeset
2598 if not flags and self._peek_iscensored(baserev, delta, flush):
24255
4bfe9f2d9761 revlog: addgroup checks if incoming deltas add censored revs, sets flag bit
Mike Edgar <adgar@google.com>
parents: 24123
diff changeset
2599 flags |= REVIDX_ISCENSORED
4bfe9f2d9761 revlog: addgroup checks if incoming deltas add censored revs, sets flag bit
Mike Edgar <adgar@google.com>
parents: 24123
diff changeset
2600
26243
836291420d53 revlog: optionally cache the full text when adding revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26242
diff changeset
2601 # We assume consumers of addrevisioncb will want to retrieve
836291420d53 revlog: optionally cache the full text when adding revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26242
diff changeset
2602 # the added revision, which will require a call to
836291420d53 revlog: optionally cache the full text when adding revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26242
diff changeset
2603 # revision(). revision() will fast path if there is a cache
836291420d53 revlog: optionally cache the full text when adding revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26242
diff changeset
2604 # hit. So, we tell _addrevision() to always cache in this case.
30743
2df983125d37 revlog: add 'raw' argument to revision and _addrevision
Remi Chaintron <remi@fb.com>
parents: 30584
diff changeset
2605 # We're only using addgroup() in the context of changegroup
2df983125d37 revlog: add 'raw' argument to revision and _addrevision
Remi Chaintron <remi@fb.com>
parents: 30584
diff changeset
2606 # generation so the revision data can always be handled as raw
2df983125d37 revlog: add 'raw' argument to revision and _addrevision
Remi Chaintron <remi@fb.com>
parents: 30584
diff changeset
2607 # by the flagprocessor.
34147
b96cfc309ac5 revlog: refactor chain variable
Durham Goode <durham@fb.com>
parents: 34028
diff changeset
2608 self._addrevision(node, None, transaction, link,
b96cfc309ac5 revlog: refactor chain variable
Durham Goode <durham@fb.com>
parents: 34028
diff changeset
2609 p1, p2, flags, (baserev, delta),
b96cfc309ac5 revlog: refactor chain variable
Durham Goode <durham@fb.com>
parents: 34028
diff changeset
2610 ifh, dfh,
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
2611 alwayscache=bool(addrevisioncb),
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
2612 deltacomputer=deltacomputer)
25822
00e3f909907f revlog: add support for a callback whenever revisions are added
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
2613
00e3f909907f revlog: add support for a callback whenever revisions are added
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
2614 if addrevisioncb:
34147
b96cfc309ac5 revlog: refactor chain variable
Durham Goode <durham@fb.com>
parents: 34028
diff changeset
2615 addrevisioncb(self, node)
25822
00e3f909907f revlog: add support for a callback whenever revisions are added
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
2616
12624
557988c691d1 revlog.addgroup(): always use _addrevision() to add new revlog entries
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 12623
diff changeset
2617 if not dfh and not self._inline:
557988c691d1 revlog.addgroup(): always use _addrevision() to add new revlog entries
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 12623
diff changeset
2618 # addrevision switched from inline to conventional
557988c691d1 revlog.addgroup(): always use _addrevision() to add new revlog entries
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 12623
diff changeset
2619 # reopen the index
13400
14f3795a5ed7 explicitly close files
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 13284
diff changeset
2620 ifh.close()
35967
61326dd7cb8d revlog: move datafile opening in a method
Boris Feld <boris.feld@octobus.net>
parents: 35845
diff changeset
2621 dfh = self._datafp("a+")
35968
91b3b11565e5 revlog: move index file opening in a method
Boris Feld <boris.feld@octobus.net>
parents: 35967
diff changeset
2622 ifh = self._indexfp("a+")
6261
7c8101b5ceb1 revlog: make sure the files are closed after an exception happens
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 6228
diff changeset
2623 finally:
7c8101b5ceb1 revlog: make sure the files are closed after an exception happens
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 6228
diff changeset
2624 if dfh:
7c8101b5ceb1 revlog: make sure the files are closed after an exception happens
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 6228
diff changeset
2625 dfh.close()
7c8101b5ceb1 revlog: make sure the files are closed after an exception happens
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 6228
diff changeset
2626 ifh.close()
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
2627
32868
ef015ba5ba2e revlog: rename list of nodes from "content" to "nodes"
Martin von Zweigbergk <martinvonz@google.com>
parents: 32867
diff changeset
2628 return nodes
1493
1a216cb4ee64 verify: add check for mismatch of index and data length
Matt Mackall <mpm@selenic.com>
parents: 1469
diff changeset
2629
24118
76f6ae06ddf5 revlog: add "iscensored()" to revlog public API
Mike Edgar <adgar@google.com>
parents: 24030
diff changeset
2630 def iscensored(self, rev):
76f6ae06ddf5 revlog: add "iscensored()" to revlog public API
Mike Edgar <adgar@google.com>
parents: 24030
diff changeset
2631 """Check if a file revision is censored."""
37443
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
2632 if not self._censorable:
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
2633 return False
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
2634
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
2635 return self.flags(rev) & REVIDX_ISCENSORED
24118
76f6ae06ddf5 revlog: add "iscensored()" to revlog public API
Mike Edgar <adgar@google.com>
parents: 24030
diff changeset
2636
24255
4bfe9f2d9761 revlog: addgroup checks if incoming deltas add censored revs, sets flag bit
Mike Edgar <adgar@google.com>
parents: 24123
diff changeset
2637 def _peek_iscensored(self, baserev, delta, flush):
4bfe9f2d9761 revlog: addgroup checks if incoming deltas add censored revs, sets flag bit
Mike Edgar <adgar@google.com>
parents: 24123
diff changeset
2638 """Quickly check if a delta produces a censored revision."""
37443
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
2639 if not self._censorable:
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
2640 return False
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
2641
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
2642 # Fragile heuristic: unless new file meta keys are added alphabetically
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
2643 # preceding "censored", all censored revisions are prefixed by
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
2644 # "\1\ncensored:". A delta producing such a censored revision must be a
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
2645 # full-replacement delta, so we inspect the first and only patch in the
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
2646 # delta for this prefix.
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
2647 hlen = struct.calcsize(">lll")
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
2648 if len(delta) <= hlen:
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
2649 return False
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
2650
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
2651 oldlen = self.rawsize(baserev)
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
2652 newlen = len(delta) - hlen
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
2653 if delta[:hlen] != mdiff.replacediffheader(oldlen, newlen):
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
2654 return False
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
2655
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
2656 add = "\1\ncensored:"
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
2657 addlen = len(add)
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
2658 return newlen >= addlen and delta[hlen:hlen + addlen] == add
24255
4bfe9f2d9761 revlog: addgroup checks if incoming deltas add censored revs, sets flag bit
Mike Edgar <adgar@google.com>
parents: 24123
diff changeset
2659
20074
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
2660 def getstrippoint(self, minlink):
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
2661 """find the minimum rev that must be stripped to strip the linkrev
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
2662
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
2663 Returns a tuple containing the minimum rev and a set of all revs that
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
2664 have linkrevs that will be broken by this strip.
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
2665 """
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
2666 brokenrevs = set()
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
2667 strippoint = len(self)
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
2668
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
2669 heads = {}
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
2670 futurelargelinkrevs = set()
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
2671 for head in self.headrevs():
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
2672 headlinkrev = self.linkrev(head)
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
2673 heads[head] = headlinkrev
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
2674 if headlinkrev >= minlink:
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
2675 futurelargelinkrevs.add(headlinkrev)
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
2676
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
2677 # This algorithm involves walking down the rev graph, starting at the
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
2678 # heads. Since the revs are topologically sorted according to linkrev,
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
2679 # once all head linkrevs are below the minlink, we know there are
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
2680 # no more revs that could have a linkrev greater than minlink.
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
2681 # So we can stop walking.
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
2682 while futurelargelinkrevs:
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
2683 strippoint -= 1
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
2684 linkrev = heads.pop(strippoint)
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
2685
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
2686 if linkrev < minlink:
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
2687 brokenrevs.add(strippoint)
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
2688 else:
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
2689 futurelargelinkrevs.remove(linkrev)
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
2690
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
2691 for p in self.parentrevs(strippoint):
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
2692 if p != nullrev:
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
2693 plinkrev = self.linkrev(p)
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
2694 heads[p] = plinkrev
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
2695 if plinkrev >= minlink:
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
2696 futurelargelinkrevs.add(plinkrev)
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
2697
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
2698 return strippoint, brokenrevs
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
2699
8073
e8a28556a0a8 strip: make repair.strip transactional to avoid repository corruption
Henrik Stuart <henrik.stuart@edlund.dk>
parents: 8017
diff changeset
2700 def strip(self, minlink, transaction):
5910
b9a830fa10f6 simplify revlog.strip interface and callers; add docstring
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5909
diff changeset
2701 """truncate the revlog on the first revision with a linkrev >= minlink
b9a830fa10f6 simplify revlog.strip interface and callers; add docstring
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5909
diff changeset
2702
b9a830fa10f6 simplify revlog.strip interface and callers; add docstring
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5909
diff changeset
2703 This function is called when we're stripping revision minlink and
b9a830fa10f6 simplify revlog.strip interface and callers; add docstring
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5909
diff changeset
2704 its descendants from the repository.
b9a830fa10f6 simplify revlog.strip interface and callers; add docstring
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5909
diff changeset
2705
b9a830fa10f6 simplify revlog.strip interface and callers; add docstring
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5909
diff changeset
2706 We have to remove all revisions with linkrev >= minlink, because
b9a830fa10f6 simplify revlog.strip interface and callers; add docstring
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5909
diff changeset
2707 the equivalent changelog revisions will be renumbered after the
b9a830fa10f6 simplify revlog.strip interface and callers; add docstring
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5909
diff changeset
2708 strip.
b9a830fa10f6 simplify revlog.strip interface and callers; add docstring
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5909
diff changeset
2709
b9a830fa10f6 simplify revlog.strip interface and callers; add docstring
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5909
diff changeset
2710 So we truncate the revlog on the first of these revisions, and
b9a830fa10f6 simplify revlog.strip interface and callers; add docstring
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5909
diff changeset
2711 trust that the caller has saved the revisions that shouldn't be
15827
1dacf7672556 revlog: clarify strip docstring "readd" -> "re-add"
Steven Brown <StevenGBrown@gmail.com>
parents: 15407
diff changeset
2712 removed and that it'll re-add them after this truncation.
5910
b9a830fa10f6 simplify revlog.strip interface and callers; add docstring
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5909
diff changeset
2713 """
6750
fb42030d79d6 add __len__ and __iter__ methods to repo and revlog
Matt Mackall <mpm@selenic.com>
parents: 6703
diff changeset
2714 if len(self) == 0:
1535
7ae0ce7a3dc4 Add revlog.strip to truncate away revisions.
mason@suse.com
parents: 1533
diff changeset
2715 return
7ae0ce7a3dc4 Add revlog.strip to truncate away revisions.
mason@suse.com
parents: 1533
diff changeset
2716
20074
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
2717 rev, _ = self.getstrippoint(minlink)
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
2718 if rev == len(self):
5909
f45f7390c1c5 strip: calculate list of extra nodes to save and pass it to changegroupsubset
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5659
diff changeset
2719 return
1535
7ae0ce7a3dc4 Add revlog.strip to truncate away revisions.
mason@suse.com
parents: 1533
diff changeset
2720
7ae0ce7a3dc4 Add revlog.strip to truncate away revisions.
mason@suse.com
parents: 1533
diff changeset
2721 # first truncate the files on disk
7ae0ce7a3dc4 Add revlog.strip to truncate away revisions.
mason@suse.com
parents: 1533
diff changeset
2722 end = self.start(rev)
4982
9672e3c42b0c revlog: change _inline from a function to a variable
Matt Mackall <mpm@selenic.com>
parents: 4981
diff changeset
2723 if not self._inline:
8073
e8a28556a0a8 strip: make repair.strip transactional to avoid repository corruption
Henrik Stuart <henrik.stuart@edlund.dk>
parents: 8017
diff changeset
2724 transaction.add(self.datafile, end)
4977
6cb30bc4ca32 revlog: parse revlogv0 indexes into v1 internally
Matt Mackall <mpm@selenic.com>
parents: 4976
diff changeset
2725 end = rev * self._io.size
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
2726 else:
4977
6cb30bc4ca32 revlog: parse revlogv0 indexes into v1 internally
Matt Mackall <mpm@selenic.com>
parents: 4976
diff changeset
2727 end += rev * self._io.size
2072
74d3f5336b66 Implement revlogng.
mason@suse.com
parents: 2002
diff changeset
2728
8073
e8a28556a0a8 strip: make repair.strip transactional to avoid repository corruption
Henrik Stuart <henrik.stuart@edlund.dk>
parents: 8017
diff changeset
2729 transaction.add(self.indexfile, end)
1535
7ae0ce7a3dc4 Add revlog.strip to truncate away revisions.
mason@suse.com
parents: 1533
diff changeset
2730
7ae0ce7a3dc4 Add revlog.strip to truncate away revisions.
mason@suse.com
parents: 1533
diff changeset
2731 # then reset internal state in memory to forget those revisions
4984
b4066fcbd6ba revlog: mark cache private
Matt Mackall <mpm@selenic.com>
parents: 4983
diff changeset
2732 self._cache = None
23306
f7a42f8e82bd revlog: cache chain info after calculating it for a rev (issue4452)
Siddharth Agarwal <sid0@fb.com>
parents: 23288
diff changeset
2733 self._chaininfocache = {}
8650
ef393d6ec030 revlog: refactor chunk cache interface again
Matt Mackall <mpm@selenic.com>
parents: 8643
diff changeset
2734 self._chunkclear()
6750
fb42030d79d6 add __len__ and __iter__ methods to repo and revlog
Matt Mackall <mpm@selenic.com>
parents: 6703
diff changeset
2735 for x in xrange(rev, len(self)):
2072
74d3f5336b66 Implement revlogng.
mason@suse.com
parents: 2002
diff changeset
2736 del self.nodemap[self.node(x)]
1535
7ae0ce7a3dc4 Add revlog.strip to truncate away revisions.
mason@suse.com
parents: 1533
diff changeset
2737
4979
06abdaf78788 revlog: add a magic null revision to our index
Matt Mackall <mpm@selenic.com>
parents: 4978
diff changeset
2738 del self.index[rev:-1]
37494
1ce7a55b09d1 revlog: reset _nodepos after strip
Joerg Sonnenberger <joerg@bec.de>
parents: 37449
diff changeset
2739 self._nodepos = None
1535
7ae0ce7a3dc4 Add revlog.strip to truncate away revisions.
mason@suse.com
parents: 1533
diff changeset
2740
1493
1a216cb4ee64 verify: add check for mismatch of index and data length
Matt Mackall <mpm@selenic.com>
parents: 1469
diff changeset
2741 def checksize(self):
1a216cb4ee64 verify: add check for mismatch of index and data length
Matt Mackall <mpm@selenic.com>
parents: 1469
diff changeset
2742 expected = 0
6750
fb42030d79d6 add __len__ and __iter__ methods to repo and revlog
Matt Mackall <mpm@selenic.com>
parents: 6703
diff changeset
2743 if len(self):
fb42030d79d6 add __len__ and __iter__ methods to repo and revlog
Matt Mackall <mpm@selenic.com>
parents: 6703
diff changeset
2744 expected = max(0, self.end(len(self) - 1))
1667
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
2745
1494
249ca10d37f4 Handle empty logs in repo.checksize
Matt Mackall <mpm@selenic.com>
parents: 1493
diff changeset
2746 try:
35972
82afb1a5ed94 revlog: use context manager for data file lifetime in checksize
Boris Feld <boris.feld@octobus.net>
parents: 35971
diff changeset
2747 with self._datafp() as f:
82afb1a5ed94 revlog: use context manager for data file lifetime in checksize
Boris Feld <boris.feld@octobus.net>
parents: 35971
diff changeset
2748 f.seek(0, 2)
82afb1a5ed94 revlog: use context manager for data file lifetime in checksize
Boris Feld <boris.feld@octobus.net>
parents: 35971
diff changeset
2749 actual = f.tell()
1667
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
2750 dd = actual - expected
25660
328739ea70c3 global: mass rewrite to use modern exception syntax
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25459
diff changeset
2751 except IOError as inst:
1667
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
2752 if inst.errno != errno.ENOENT:
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
2753 raise
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
2754 dd = 0
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
2755
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
2756 try:
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
2757 f = self.opener(self.indexfile)
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
2758 f.seek(0, 2)
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
2759 actual = f.tell()
13400
14f3795a5ed7 explicitly close files
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 13284
diff changeset
2760 f.close()
4977
6cb30bc4ca32 revlog: parse revlogv0 indexes into v1 internally
Matt Mackall <mpm@selenic.com>
parents: 4976
diff changeset
2761 s = self._io.size
9029
0001e49f1c11 compat: use // for integer division
Alejandro Santos <alejolp@alejolp.com>
parents: 8658
diff changeset
2762 i = max(0, actual // s)
1667
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
2763 di = actual - (i * s)
4982
9672e3c42b0c revlog: change _inline from a function to a variable
Matt Mackall <mpm@selenic.com>
parents: 4981
diff changeset
2764 if self._inline:
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
2765 databytes = 0
6750
fb42030d79d6 add __len__ and __iter__ methods to repo and revlog
Matt Mackall <mpm@selenic.com>
parents: 6703
diff changeset
2766 for r in self:
5312
fb070713ff36 revlog: more robust for damaged indexes
Matt Mackall <mpm@selenic.com>
parents: 5007
diff changeset
2767 databytes += max(0, self.length(r))
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
2768 dd = 0
6750
fb42030d79d6 add __len__ and __iter__ methods to repo and revlog
Matt Mackall <mpm@selenic.com>
parents: 6703
diff changeset
2769 di = actual - len(self) * s - databytes
25660
328739ea70c3 global: mass rewrite to use modern exception syntax
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25459
diff changeset
2770 except IOError as inst:
1667
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
2771 if inst.errno != errno.ENOENT:
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
2772 raise
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
2773 di = 0
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
2774
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
2775 return (dd, di)
6891
22cb82433842 revlog: add files method
Adrian Buehlmann <adrian@cadifra.com>
parents: 6872
diff changeset
2776
22cb82433842 revlog: add files method
Adrian Buehlmann <adrian@cadifra.com>
parents: 6872
diff changeset
2777 def files(self):
10282
08a0f04b56bd many, many trivial check-code fixups
Matt Mackall <mpm@selenic.com>
parents: 10264
diff changeset
2778 res = [self.indexfile]
6891
22cb82433842 revlog: add files method
Adrian Buehlmann <adrian@cadifra.com>
parents: 6872
diff changeset
2779 if not self._inline:
22cb82433842 revlog: add files method
Adrian Buehlmann <adrian@cadifra.com>
parents: 6872
diff changeset
2780 res.append(self.datafile)
22cb82433842 revlog: add files method
Adrian Buehlmann <adrian@cadifra.com>
parents: 6872
diff changeset
2781 return res
30778
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2782
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2783 DELTAREUSEALWAYS = 'always'
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2784 DELTAREUSESAMEREVS = 'samerevs'
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2785 DELTAREUSENEVER = 'never'
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2786
35345
6226668a7169 upgrade: add a 'redeltafullall' mode
Boris Feld <boris.feld@octobus.net>
parents: 34898
diff changeset
2787 DELTAREUSEFULLADD = 'fulladd'
6226668a7169 upgrade: add a 'redeltafullall' mode
Boris Feld <boris.feld@octobus.net>
parents: 34898
diff changeset
2788
6226668a7169 upgrade: add a 'redeltafullall' mode
Boris Feld <boris.feld@octobus.net>
parents: 34898
diff changeset
2789 DELTAREUSEALL = {'always', 'samerevs', 'never', 'fulladd'}
30778
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2790
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2791 def clone(self, tr, destrevlog, addrevisioncb=None,
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2792 deltareuse=DELTAREUSESAMEREVS, aggressivemergedeltas=None):
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2793 """Copy this revlog to another, possibly with format changes.
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2794
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2795 The destination revlog will contain the same revisions and nodes.
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2796 However, it may not be bit-for-bit identical due to e.g. delta encoding
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2797 differences.
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2798
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2799 The ``deltareuse`` argument control how deltas from the existing revlog
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2800 are preserved in the destination revlog. The argument can have the
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2801 following values:
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2802
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2803 DELTAREUSEALWAYS
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2804 Deltas will always be reused (if possible), even if the destination
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2805 revlog would not select the same revisions for the delta. This is the
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2806 fastest mode of operation.
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2807 DELTAREUSESAMEREVS
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2808 Deltas will be reused if the destination revlog would pick the same
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2809 revisions for the delta. This mode strikes a balance between speed
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2810 and optimization.
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2811 DELTAREUSENEVER
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2812 Deltas will never be reused. This is the slowest mode of execution.
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2813 This mode can be used to recompute deltas (e.g. if the diff/delta
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2814 algorithm changes).
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2815
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2816 Delta computation can be slow, so the choice of delta reuse policy can
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2817 significantly affect run time.
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2818
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2819 The default policy (``DELTAREUSESAMEREVS``) strikes a balance between
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2820 two extremes. Deltas will be reused if they are appropriate. But if the
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2821 delta could choose a better revision, it will do so. This means if you
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2822 are converting a non-generaldelta revlog to a generaldelta revlog,
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2823 deltas will be recomputed if the delta's parent isn't a parent of the
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2824 revision.
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2825
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2826 In addition to the delta policy, the ``aggressivemergedeltas`` argument
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2827 controls whether to compute deltas against both parents for merges.
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2828 By default, the current default is used.
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2829 """
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2830 if deltareuse not in self.DELTAREUSEALL:
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2831 raise ValueError(_('value for deltareuse invalid: %s') % deltareuse)
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2832
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2833 if len(destrevlog):
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2834 raise ValueError(_('destination revlog is not empty'))
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2835
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2836 if getattr(self, 'filteredrevs', None):
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2837 raise ValueError(_('source revlog has filtered revisions'))
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2838 if getattr(destrevlog, 'filteredrevs', None):
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2839 raise ValueError(_('destination revlog has filtered revisions'))
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2840
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2841 # lazydeltabase controls whether to reuse a cached delta, if possible.
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2842 oldlazydeltabase = destrevlog._lazydeltabase
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2843 oldamd = destrevlog._aggressivemergedeltas
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2844
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2845 try:
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2846 if deltareuse == self.DELTAREUSEALWAYS:
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2847 destrevlog._lazydeltabase = True
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2848 elif deltareuse == self.DELTAREUSESAMEREVS:
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2849 destrevlog._lazydeltabase = False
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2850
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2851 destrevlog._aggressivemergedeltas = aggressivemergedeltas or oldamd
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2852
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2853 populatecachedelta = deltareuse in (self.DELTAREUSEALWAYS,
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2854 self.DELTAREUSESAMEREVS)
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2855
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
2856 deltacomputer = _deltacomputer(destrevlog)
30778
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2857 index = self.index
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2858 for rev in self:
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2859 entry = index[rev]
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2860
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2861 # Some classes override linkrev to take filtered revs into
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2862 # account. Use raw entry from index.
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2863 flags = entry[0] & 0xffff
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2864 linkrev = entry[4]
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2865 p1 = index[entry[5]][7]
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2866 p2 = index[entry[6]][7]
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2867 node = entry[7]
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2868
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2869 # (Possibly) reuse the delta from the revlog if allowed and
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2870 # the revlog chunk is a delta.
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2871 cachedelta = None
31754
5b93c6fdb391 revlog: use raw revisions in clone
Jun Wu <quark@fb.com>
parents: 31753
diff changeset
2872 rawtext = None
30778
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2873 if populatecachedelta:
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2874 dp = self.deltaparent(rev)
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2875 if dp != nullrev:
36722
6bacb2f663cb py3: use bytes() instead of str()
Pulkit Goyal <7895pulkit@gmail.com>
parents: 36721
diff changeset
2876 cachedelta = (dp, bytes(self._chunk(rev)))
30778
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2877
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2878 if not cachedelta:
31754
5b93c6fdb391 revlog: use raw revisions in clone
Jun Wu <quark@fb.com>
parents: 31753
diff changeset
2879 rawtext = self.revision(rev, raw=True)
30778
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2880
35345
6226668a7169 upgrade: add a 'redeltafullall' mode
Boris Feld <boris.feld@octobus.net>
parents: 34898
diff changeset
2881
6226668a7169 upgrade: add a 'redeltafullall' mode
Boris Feld <boris.feld@octobus.net>
parents: 34898
diff changeset
2882 if deltareuse == self.DELTAREUSEFULLADD:
6226668a7169 upgrade: add a 'redeltafullall' mode
Boris Feld <boris.feld@octobus.net>
parents: 34898
diff changeset
2883 destrevlog.addrevision(rawtext, tr, linkrev, p1, p2,
6226668a7169 upgrade: add a 'redeltafullall' mode
Boris Feld <boris.feld@octobus.net>
parents: 34898
diff changeset
2884 cachedelta=cachedelta,
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
2885 node=node, flags=flags,
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
2886 deltacomputer=deltacomputer)
35345
6226668a7169 upgrade: add a 'redeltafullall' mode
Boris Feld <boris.feld@octobus.net>
parents: 34898
diff changeset
2887 else:
6226668a7169 upgrade: add a 'redeltafullall' mode
Boris Feld <boris.feld@octobus.net>
parents: 34898
diff changeset
2888 ifh = destrevlog.opener(destrevlog.indexfile, 'a+',
6226668a7169 upgrade: add a 'redeltafullall' mode
Boris Feld <boris.feld@octobus.net>
parents: 34898
diff changeset
2889 checkambig=False)
6226668a7169 upgrade: add a 'redeltafullall' mode
Boris Feld <boris.feld@octobus.net>
parents: 34898
diff changeset
2890 dfh = None
6226668a7169 upgrade: add a 'redeltafullall' mode
Boris Feld <boris.feld@octobus.net>
parents: 34898
diff changeset
2891 if not destrevlog._inline:
6226668a7169 upgrade: add a 'redeltafullall' mode
Boris Feld <boris.feld@octobus.net>
parents: 34898
diff changeset
2892 dfh = destrevlog.opener(destrevlog.datafile, 'a+')
6226668a7169 upgrade: add a 'redeltafullall' mode
Boris Feld <boris.feld@octobus.net>
parents: 34898
diff changeset
2893 try:
6226668a7169 upgrade: add a 'redeltafullall' mode
Boris Feld <boris.feld@octobus.net>
parents: 34898
diff changeset
2894 destrevlog._addrevision(node, rawtext, tr, linkrev, p1,
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
2895 p2, flags, cachedelta, ifh, dfh,
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
2896 deltacomputer=deltacomputer)
35345
6226668a7169 upgrade: add a 'redeltafullall' mode
Boris Feld <boris.feld@octobus.net>
parents: 34898
diff changeset
2897 finally:
6226668a7169 upgrade: add a 'redeltafullall' mode
Boris Feld <boris.feld@octobus.net>
parents: 34898
diff changeset
2898 if dfh:
6226668a7169 upgrade: add a 'redeltafullall' mode
Boris Feld <boris.feld@octobus.net>
parents: 34898
diff changeset
2899 dfh.close()
6226668a7169 upgrade: add a 'redeltafullall' mode
Boris Feld <boris.feld@octobus.net>
parents: 34898
diff changeset
2900 ifh.close()
30778
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2901
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2902 if addrevisioncb:
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2903 addrevisioncb(self, rev, node)
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2904 finally:
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2905 destrevlog._lazydeltabase = oldlazydeltabase
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
2906 destrevlog._aggressivemergedeltas = oldamd