mercurial/revlog.py
author Matt Mackall <mpm@selenic.com>
Thu, 11 Oct 2007 00:46:54 -0500
changeset 5451 0a43875677b1
parent 5450 c728424d44c6
child 5453 9d77f2b47eb7
permissions -rw-r--r--
revlog: break up compression of large deltas Python's zlib apparently makes an internal copy of strings passed to compress(). To avoid this, compress strings 1M at a time, then join them at the end if the result would be smaller than the original. For initial commits of large but compressible files, this cuts peak memory usage nearly in half.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
     1
"""
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
     2
revlog.py - storage back-end for mercurial
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
     3
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
     4
This provides efficient delta storage with O(1) retrieve and append
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
     5
and O(changes) merge between branches
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
     6
4635
63b9d2deed48 Updated copyright notices and add "and others" to "hg version"
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4497
diff changeset
     7
Copyright 2005-2007 Matt Mackall <mpm@selenic.com>
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
     8
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
     9
This software may be used and distributed according to the terms
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
    10
of the GNU General Public License, incorporated herein by reference.
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
    11
"""
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
    12
1089
142b5d5ec9cc Break apart hg.py
mpm@selenic.com
parents: 1083
diff changeset
    13
from node import *
3891
6b4127c7d52a Simplify i18n imports
Matt Mackall <mpm@selenic.com>
parents: 3877
diff changeset
    14
from i18n import _
3877
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents: 3755
diff changeset
    15
import binascii, changegroup, errno, ancestor, mdiff, os
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents: 3755
diff changeset
    16
import sha, struct, util, zlib
36
da28286bf6b7 Add smart node lookup by substring or by rev number
mpm@selenic.com
parents: 26
diff changeset
    17
5007
3addf4531643 revlog: localize some fastpath functions
Matt Mackall <mpm@selenic.com>
parents: 5006
diff changeset
    18
_pack = struct.pack
3addf4531643 revlog: localize some fastpath functions
Matt Mackall <mpm@selenic.com>
parents: 5006
diff changeset
    19
_unpack = struct.unpack
3addf4531643 revlog: localize some fastpath functions
Matt Mackall <mpm@selenic.com>
parents: 5006
diff changeset
    20
_compress = zlib.compress
3addf4531643 revlog: localize some fastpath functions
Matt Mackall <mpm@selenic.com>
parents: 5006
diff changeset
    21
_decompress = zlib.decompress
3addf4531643 revlog: localize some fastpath functions
Matt Mackall <mpm@selenic.com>
parents: 5006
diff changeset
    22
_sha = sha.new
3addf4531643 revlog: localize some fastpath functions
Matt Mackall <mpm@selenic.com>
parents: 5006
diff changeset
    23
4987
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
    24
# revlog flags
2072
74d3f5336b66 Implement revlogng.
mason@suse.com
parents: 2002
diff changeset
    25
REVLOGV0 = 0
74d3f5336b66 Implement revlogng.
mason@suse.com
parents: 2002
diff changeset
    26
REVLOGNG = 1
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
    27
REVLOGNGINLINEDATA = (1 << 16)
2222
c9e264b115e6 Use revlogng and inlined data files by default
mason@suse.com
parents: 2177
diff changeset
    28
REVLOG_DEFAULT_FLAGS = REVLOGNGINLINEDATA
c9e264b115e6 Use revlogng and inlined data files by default
mason@suse.com
parents: 2177
diff changeset
    29
REVLOG_DEFAULT_FORMAT = REVLOGNG
c9e264b115e6 Use revlogng and inlined data files by default
mason@suse.com
parents: 2177
diff changeset
    30
REVLOG_DEFAULT_VERSION = REVLOG_DEFAULT_FORMAT | REVLOG_DEFAULT_FLAGS
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
    31
4987
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
    32
class RevlogError(Exception):
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
    33
    pass
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
    34
class LookupError(RevlogError):
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
    35
    pass
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
    36
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
    37
def getoffset(q):
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
    38
    return int(q >> 16)
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
    39
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
    40
def gettype(q):
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
    41
    return int(q & 0xFFFF)
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
    42
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
    43
def offset_type(offset, type):
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
    44
    return long(long(offset) << 16 | type)
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
    45
1091
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
    46
def hash(text, p1, p2):
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
    47
    """generate a hash from the given text and its parent hashes
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
    48
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
    49
    This hash combines both the current file contents and its history
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
    50
    in a manner that makes it easy to distinguish nodes with the same
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
    51
    content in the revision graph.
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
    52
    """
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
    53
    l = [p1, p2]
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
    54
    l.sort()
5007
3addf4531643 revlog: localize some fastpath functions
Matt Mackall <mpm@selenic.com>
parents: 5006
diff changeset
    55
    s = _sha(l[0])
1091
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
    56
    s.update(l[1])
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
    57
    s.update(text)
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
    58
    return s.digest()
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
    59
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
    60
def compress(text):
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
    61
    """ generate a possibly-compressed representation of text """
4980
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
    62
    if not text:
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
    63
        return ("", text)
5451
0a43875677b1 revlog: break up compression of large deltas
Matt Mackall <mpm@selenic.com>
parents: 5450
diff changeset
    64
    l = len(text)
0a43875677b1 revlog: break up compression of large deltas
Matt Mackall <mpm@selenic.com>
parents: 5450
diff changeset
    65
    if l < 44:
4980
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
    66
        if text[0] == '\0':
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
    67
            return ("", text)
1533
3d11f81c9145 Reduce string duplication in compression code
mason@suse.com
parents: 1509
diff changeset
    68
        return ('u', text)
5451
0a43875677b1 revlog: break up compression of large deltas
Matt Mackall <mpm@selenic.com>
parents: 5450
diff changeset
    69
    elif l > 1000000:
0a43875677b1 revlog: break up compression of large deltas
Matt Mackall <mpm@selenic.com>
parents: 5450
diff changeset
    70
        # zlib makes an internal copy, thus doubling memory usage for
0a43875677b1 revlog: break up compression of large deltas
Matt Mackall <mpm@selenic.com>
parents: 5450
diff changeset
    71
        # large files, so lets do this in pieces
0a43875677b1 revlog: break up compression of large deltas
Matt Mackall <mpm@selenic.com>
parents: 5450
diff changeset
    72
        z = zlib.compressobj()
0a43875677b1 revlog: break up compression of large deltas
Matt Mackall <mpm@selenic.com>
parents: 5450
diff changeset
    73
        p = []
0a43875677b1 revlog: break up compression of large deltas
Matt Mackall <mpm@selenic.com>
parents: 5450
diff changeset
    74
        pos = 0
0a43875677b1 revlog: break up compression of large deltas
Matt Mackall <mpm@selenic.com>
parents: 5450
diff changeset
    75
        while pos < l:
0a43875677b1 revlog: break up compression of large deltas
Matt Mackall <mpm@selenic.com>
parents: 5450
diff changeset
    76
            pos2 = pos + 2**20
0a43875677b1 revlog: break up compression of large deltas
Matt Mackall <mpm@selenic.com>
parents: 5450
diff changeset
    77
            p.append(z.compress(text[pos:pos2]))
0a43875677b1 revlog: break up compression of large deltas
Matt Mackall <mpm@selenic.com>
parents: 5450
diff changeset
    78
            pos = pos2
0a43875677b1 revlog: break up compression of large deltas
Matt Mackall <mpm@selenic.com>
parents: 5450
diff changeset
    79
        p.append(z.flush())
0a43875677b1 revlog: break up compression of large deltas
Matt Mackall <mpm@selenic.com>
parents: 5450
diff changeset
    80
        if sum(map(len, p)) < l:
0a43875677b1 revlog: break up compression of large deltas
Matt Mackall <mpm@selenic.com>
parents: 5450
diff changeset
    81
            bin = "".join(p)
0a43875677b1 revlog: break up compression of large deltas
Matt Mackall <mpm@selenic.com>
parents: 5450
diff changeset
    82
    else:
0a43875677b1 revlog: break up compression of large deltas
Matt Mackall <mpm@selenic.com>
parents: 5450
diff changeset
    83
        bin = _compress(text)
0a43875677b1 revlog: break up compression of large deltas
Matt Mackall <mpm@selenic.com>
parents: 5450
diff changeset
    84
    if len(bin) > l:
4980
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
    85
        if text[0] == '\0':
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
    86
            return ("", text)
1533
3d11f81c9145 Reduce string duplication in compression code
mason@suse.com
parents: 1509
diff changeset
    87
        return ('u', text)
3d11f81c9145 Reduce string duplication in compression code
mason@suse.com
parents: 1509
diff changeset
    88
    return ("", bin)
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
    89
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
    90
def decompress(bin):
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
    91
    """ decompress the given input """
4980
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
    92
    if not bin:
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
    93
        return bin
112
aea6562add6c Make compression more intelligent:
mpm@selenic.com
parents: 98
diff changeset
    94
    t = bin[0]
4980
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
    95
    if t == '\0':
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
    96
        return bin
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
    97
    if t == 'x':
5007
3addf4531643 revlog: localize some fastpath functions
Matt Mackall <mpm@selenic.com>
parents: 5006
diff changeset
    98
        return _decompress(bin)
4980
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
    99
    if t == 'u':
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
   100
        return bin[1:]
1853
5ac811b720de Fix some problems when working on broken repositories:
Thomas Arendsen Hein <thomas@intevation.de>
parents: 1784
diff changeset
   101
    raise RevlogError(_("unknown compression type %r") % t)
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
   102
1559
59b3639df0a9 Convert all classes to new-style classes by deriving them from object.
Eric Hopper <hopper@omnifarious.org>
parents: 1551
diff changeset
   103
class lazyparser(object):
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   104
    """
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   105
    this class avoids the need to parse the entirety of large indices
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   106
    """
2250
45aef5ddcdbe windows: revlog.lazyparser not always safe to use.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2222
diff changeset
   107
45aef5ddcdbe windows: revlog.lazyparser not always safe to use.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2222
diff changeset
   108
    # lazyparser is not safe to use on windows if win32 extensions not
45aef5ddcdbe windows: revlog.lazyparser not always safe to use.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2222
diff changeset
   109
    # available. it keeps file handle open, which make it not possible
45aef5ddcdbe windows: revlog.lazyparser not always safe to use.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2222
diff changeset
   110
    # to break hardlinks on local cloned repos.
45aef5ddcdbe windows: revlog.lazyparser not always safe to use.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2222
diff changeset
   111
    safe_to_use = os.name != 'nt' or (not util.is_win_9x() and
45aef5ddcdbe windows: revlog.lazyparser not always safe to use.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2222
diff changeset
   112
                                      hasattr(util, 'win32api'))
45aef5ddcdbe windows: revlog.lazyparser not always safe to use.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2222
diff changeset
   113
4976
79c39cc9ff69 revlog: only allow lazy parsing with revlogng files
Matt Mackall <mpm@selenic.com>
parents: 4975
diff changeset
   114
    def __init__(self, dataf, size):
2079
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   115
        self.dataf = dataf
4976
79c39cc9ff69 revlog: only allow lazy parsing with revlogng files
Matt Mackall <mpm@selenic.com>
parents: 4975
diff changeset
   116
        self.s = struct.calcsize(indexformatng)
2079
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   117
        self.datasize = size
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   118
        self.l = size/self.s
76
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
   119
        self.index = [None] * self.l
3578
3b4e00cba57a Define and use nullrev (revision of nullid) instead of -1.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 3508
diff changeset
   120
        self.map = {nullid: nullrev}
2079
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   121
        self.allmap = 0
323
c6f0673ab7e9 lazyparser speed ups
mpm@selenic.com
parents: 306
diff changeset
   122
        self.all = 0
2079
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   123
        self.mapfind_count = 0
76
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
   124
2079
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   125
    def loadmap(self):
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   126
        """
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   127
        during a commit, we need to make sure the rev being added is
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   128
        not a duplicate.  This requires loading the entire index,
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   129
        which is fairly slow.  loadmap can load up just the node map,
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   130
        which takes much less time.
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   131
        """
4980
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
   132
        if self.allmap:
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
   133
            return
2079
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   134
        end = self.datasize
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   135
        self.allmap = 1
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   136
        cur = 0
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   137
        count = 0
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   138
        blocksize = self.s * 256
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   139
        self.dataf.seek(0)
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   140
        while cur < end:
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   141
            data = self.dataf.read(blocksize)
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   142
            off = 0
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   143
            for x in xrange(256):
4976
79c39cc9ff69 revlog: only allow lazy parsing with revlogng files
Matt Mackall <mpm@selenic.com>
parents: 4975
diff changeset
   144
                n = data[off + ngshaoffset:off + ngshaoffset + 20]
2079
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   145
                self.map[n] = count
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   146
                count += 1
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   147
                if count >= self.l:
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   148
                    break
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   149
                off += self.s
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   150
            cur += blocksize
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   151
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   152
    def loadblock(self, blockstart, blocksize, data=None):
4980
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
   153
        if self.all:
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
   154
            return
2079
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   155
        if data is None:
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   156
            self.dataf.seek(blockstart)
3078
baa3873eb387 don't let lazyparser read more data than it can handle
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 2890
diff changeset
   157
            if blockstart + blocksize > self.datasize:
baa3873eb387 don't let lazyparser read more data than it can handle
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 2890
diff changeset
   158
                # the revlog may have grown since we've started running,
baa3873eb387 don't let lazyparser read more data than it can handle
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 2890
diff changeset
   159
                # but we don't have space in self.index for more entries.
baa3873eb387 don't let lazyparser read more data than it can handle
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 2890
diff changeset
   160
                # limit blocksize so that we don't get too much data.
3089
e7fc04dc6349 Avoid negative block sizes in lazyparser.
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 3078
diff changeset
   161
                blocksize = max(self.datasize - blockstart, 0)
2079
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   162
            data = self.dataf.read(blocksize)
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   163
        lend = len(data) / self.s
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   164
        i = blockstart / self.s
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   165
        off = 0
4061
40030c1b6bc6 lazyindex: handle __delitem__ in loadblock
Brendan Cully <brendan@kublai.com>
parents: 3930
diff changeset
   166
        # lazyindex supports __delitem__
40030c1b6bc6 lazyindex: handle __delitem__ in loadblock
Brendan Cully <brendan@kublai.com>
parents: 3930
diff changeset
   167
        if lend > len(self.index) - i:
40030c1b6bc6 lazyindex: handle __delitem__ in loadblock
Brendan Cully <brendan@kublai.com>
parents: 3930
diff changeset
   168
            lend = len(self.index) - i
2079
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   169
        for x in xrange(lend):
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   170
            if self.index[i + x] == None:
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   171
                b = data[off : off + self.s]
2080
1cbb14c048cb Reduce index memory usage by storing the bare string instead of tuples
mason@suse.com
parents: 2079
diff changeset
   172
                self.index[i + x] = b
4976
79c39cc9ff69 revlog: only allow lazy parsing with revlogng files
Matt Mackall <mpm@selenic.com>
parents: 4975
diff changeset
   173
                n = b[ngshaoffset:ngshaoffset + 20]
2080
1cbb14c048cb Reduce index memory usage by storing the bare string instead of tuples
mason@suse.com
parents: 2079
diff changeset
   174
                self.map[n] = i + x
2079
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   175
            off += self.s
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   176
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   177
    def findnode(self, node):
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   178
        """search backwards through the index file for a specific node"""
4980
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
   179
        if self.allmap:
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
   180
            return None
2079
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   181
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   182
        # hg log will cause many many searches for the manifest
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   183
        # nodes.  After we get called a few times, just load the whole
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   184
        # thing.
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   185
        if self.mapfind_count > 8:
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   186
            self.loadmap()
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   187
            if node in self.map:
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   188
                return node
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   189
            return None
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   190
        self.mapfind_count += 1
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   191
        last = self.l - 1
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   192
        while self.index[last] != None:
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   193
            if last == 0:
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   194
                self.all = 1
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   195
                self.allmap = 1
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   196
                return None
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   197
            last -= 1
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   198
        end = (last + 1) * self.s
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   199
        blocksize = self.s * 256
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   200
        while end >= 0:
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   201
            start = max(end - blocksize, 0)
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   202
            self.dataf.seek(start)
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   203
            data = self.dataf.read(end - start)
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   204
            findend = end - start
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   205
            while True:
4994
d36310dd51d7 lazyparser.findnode: fix typo and s/rfind/find/
Matt Mackall <mpm@selenic.com>
parents: 4993
diff changeset
   206
                # we're searching backwards, so we have to make sure
2079
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   207
                # we don't find a changeset where this node is a parent
4994
d36310dd51d7 lazyparser.findnode: fix typo and s/rfind/find/
Matt Mackall <mpm@selenic.com>
parents: 4993
diff changeset
   208
                off = data.find(node, 0, findend)
2079
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   209
                findend = off
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   210
                if off >= 0:
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   211
                    i = off / self.s
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   212
                    off = i * self.s
4976
79c39cc9ff69 revlog: only allow lazy parsing with revlogng files
Matt Mackall <mpm@selenic.com>
parents: 4975
diff changeset
   213
                    n = data[off + ngshaoffset:off + ngshaoffset + 20]
2079
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   214
                    if n == node:
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   215
                        self.map[n] = i + start / self.s
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   216
                        return node
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   217
                else:
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   218
                    break
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   219
            end -= blocksize
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   220
        return None
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   221
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   222
    def loadindex(self, i=None, end=None):
4980
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
   223
        if self.all:
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
   224
            return
2079
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   225
        all = False
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   226
        if i == None:
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   227
            blockstart = 0
4992
0a676643687b lazyparser: up the blocksize from 512 bytes to 64k
Matt Mackall <mpm@selenic.com>
parents: 4991
diff changeset
   228
            blocksize = (65536 / self.s) * self.s
2079
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   229
            end = self.datasize
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   230
            all = True
323
c6f0673ab7e9 lazyparser speed ups
mpm@selenic.com
parents: 306
diff changeset
   231
        else:
2079
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   232
            if end:
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   233
                blockstart = i * self.s
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   234
                end = end * self.s
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   235
                blocksize = end - blockstart
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   236
            else:
4992
0a676643687b lazyparser: up the blocksize from 512 bytes to 64k
Matt Mackall <mpm@selenic.com>
parents: 4991
diff changeset
   237
                blockstart = (i & ~1023) * self.s
0a676643687b lazyparser: up the blocksize from 512 bytes to 64k
Matt Mackall <mpm@selenic.com>
parents: 4991
diff changeset
   238
                blocksize = self.s * 1024
2079
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   239
                end = blockstart + blocksize
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   240
        while blockstart < end:
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   241
            self.loadblock(blockstart, blocksize)
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   242
            blockstart += blocksize
4980
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
   243
        if all:
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
   244
            self.all = True
515
03f27b1381f9 Whitespace cleanups
mpm@selenic.com
parents: 484
diff changeset
   245
1559
59b3639df0a9 Convert all classes to new-style classes by deriving them from object.
Eric Hopper <hopper@omnifarious.org>
parents: 1551
diff changeset
   246
class lazyindex(object):
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   247
    """a lazy version of the index array"""
76
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
   248
    def __init__(self, parser):
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
   249
        self.p = parser
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
   250
    def __len__(self):
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
   251
        return len(self.p.index)
115
39b438eeb25a Make lazyindex load slightly faster
mpm@selenic.com
parents: 112
diff changeset
   252
    def load(self, pos):
1403
bc3e66edb04c lazyindex fix, make load handle negative indexes properly.
Eric Hopper <hopper@omnifarious.org>
parents: 1402
diff changeset
   253
        if pos < 0:
bc3e66edb04c lazyindex fix, make load handle negative indexes properly.
Eric Hopper <hopper@omnifarious.org>
parents: 1402
diff changeset
   254
            pos += len(self.p.index)
2079
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   255
        self.p.loadindex(pos)
115
39b438eeb25a Make lazyindex load slightly faster
mpm@selenic.com
parents: 112
diff changeset
   256
        return self.p.index[pos]
76
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
   257
    def __getitem__(self, pos):
5007
3addf4531643 revlog: localize some fastpath functions
Matt Mackall <mpm@selenic.com>
parents: 5006
diff changeset
   258
        return _unpack(indexformatng, self.p.index[pos] or self.load(pos))
2072
74d3f5336b66 Implement revlogng.
mason@suse.com
parents: 2002
diff changeset
   259
    def __setitem__(self, pos, item):
5007
3addf4531643 revlog: localize some fastpath functions
Matt Mackall <mpm@selenic.com>
parents: 5006
diff changeset
   260
        self.p.index[pos] = _pack(indexformatng, *item)
1535
7ae0ce7a3dc4 Add revlog.strip to truncate away revisions.
mason@suse.com
parents: 1533
diff changeset
   261
    def __delitem__(self, pos):
7ae0ce7a3dc4 Add revlog.strip to truncate away revisions.
mason@suse.com
parents: 1533
diff changeset
   262
        del self.p.index[pos]
4979
06abdaf78788 revlog: add a magic null revision to our index
Matt Mackall <mpm@selenic.com>
parents: 4978
diff changeset
   263
    def insert(self, pos, e):
5007
3addf4531643 revlog: localize some fastpath functions
Matt Mackall <mpm@selenic.com>
parents: 5006
diff changeset
   264
        self.p.index.insert(pos, _pack(indexformatng, *e))
76
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
   265
    def append(self, e):
5007
3addf4531643 revlog: localize some fastpath functions
Matt Mackall <mpm@selenic.com>
parents: 5006
diff changeset
   266
        self.p.index.append(_pack(indexformatng, *e))
515
03f27b1381f9 Whitespace cleanups
mpm@selenic.com
parents: 484
diff changeset
   267
1559
59b3639df0a9 Convert all classes to new-style classes by deriving them from object.
Eric Hopper <hopper@omnifarious.org>
parents: 1551
diff changeset
   268
class lazymap(object):
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   269
    """a lazy version of the node map"""
76
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
   270
    def __init__(self, parser):
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
   271
        self.p = parser
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
   272
    def load(self, key):
2079
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   273
        n = self.p.findnode(key)
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   274
        if n == None:
1214
34706a835d4a Smarter handling of revlog key errors
mpm@selenic.com
parents: 1201
diff changeset
   275
            raise KeyError(key)
76
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
   276
    def __contains__(self, key):
2079
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   277
        if key in self.p.map:
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   278
            return True
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   279
        self.p.loadmap()
323
c6f0673ab7e9 lazyparser speed ups
mpm@selenic.com
parents: 306
diff changeset
   280
        return key in self.p.map
97
7a2abee6b0c2 Add iterator to the lazymap code
mpm@selenic.com
parents: 94
diff changeset
   281
    def __iter__(self):
469
e205194ca7ef Various node id lookup tweaks
mpm@selenic.com
parents: 451
diff changeset
   282
        yield nullid
97
7a2abee6b0c2 Add iterator to the lazymap code
mpm@selenic.com
parents: 94
diff changeset
   283
        for i in xrange(self.p.l):
2080
1cbb14c048cb Reduce index memory usage by storing the bare string instead of tuples
mason@suse.com
parents: 2079
diff changeset
   284
            ret = self.p.index[i]
1cbb14c048cb Reduce index memory usage by storing the bare string instead of tuples
mason@suse.com
parents: 2079
diff changeset
   285
            if not ret:
2079
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   286
                self.p.loadindex(i)
2080
1cbb14c048cb Reduce index memory usage by storing the bare string instead of tuples
mason@suse.com
parents: 2079
diff changeset
   287
                ret = self.p.index[i]
1cbb14c048cb Reduce index memory usage by storing the bare string instead of tuples
mason@suse.com
parents: 2079
diff changeset
   288
            if isinstance(ret, str):
5007
3addf4531643 revlog: localize some fastpath functions
Matt Mackall <mpm@selenic.com>
parents: 5006
diff changeset
   289
                ret = _unpack(indexformatng, ret)
4978
93d48a8fa496 revlog: change accesses to index entry elements to use positive offsets
Matt Mackall <mpm@selenic.com>
parents: 4977
diff changeset
   290
            yield ret[7]
76
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
   291
    def __getitem__(self, key):
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
   292
        try:
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
   293
            return self.p.map[key]
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
   294
        except KeyError:
86
1b945e8ba67b Friendlier exceptions for unknown node errors
mpm@selenic.com
parents: 84
diff changeset
   295
            try:
1b945e8ba67b Friendlier exceptions for unknown node errors
mpm@selenic.com
parents: 84
diff changeset
   296
                self.load(key)
1b945e8ba67b Friendlier exceptions for unknown node errors
mpm@selenic.com
parents: 84
diff changeset
   297
                return self.p.map[key]
1b945e8ba67b Friendlier exceptions for unknown node errors
mpm@selenic.com
parents: 84
diff changeset
   298
            except KeyError:
1b945e8ba67b Friendlier exceptions for unknown node errors
mpm@selenic.com
parents: 84
diff changeset
   299
                raise KeyError("node " + hex(key))
76
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
   300
    def __setitem__(self, key, val):
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
   301
        self.p.map[key] = val
1535
7ae0ce7a3dc4 Add revlog.strip to truncate away revisions.
mason@suse.com
parents: 1533
diff changeset
   302
    def __delitem__(self, key):
7ae0ce7a3dc4 Add revlog.strip to truncate away revisions.
mason@suse.com
parents: 1533
diff changeset
   303
        del self.p.map[key]
76
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
   304
4987
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
   305
indexformatv0 = ">4l20s20s20s"
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
   306
v0shaoffset = 56
4918
e017d3a82e1d revlog: raise offset/type helpers to global scope
Matt Mackall <mpm@selenic.com>
parents: 4746
diff changeset
   307
4972
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   308
class revlogoldio(object):
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   309
    def __init__(self):
4977
6cb30bc4ca32 revlog: parse revlogv0 indexes into v1 internally
Matt Mackall <mpm@selenic.com>
parents: 4976
diff changeset
   310
        self.size = struct.calcsize(indexformatv0)
4972
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   311
4985
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
   312
    def parseindex(self, fp, inline):
4977
6cb30bc4ca32 revlog: parse revlogv0 indexes into v1 internally
Matt Mackall <mpm@selenic.com>
parents: 4976
diff changeset
   313
        s = self.size
4972
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   314
        index = []
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   315
        nodemap =  {nullid: nullrev}
4973
a386a6e4fe46 revlog: simplify the v0 parser
Matt Mackall <mpm@selenic.com>
parents: 4972
diff changeset
   316
        n = off = 0
a386a6e4fe46 revlog: simplify the v0 parser
Matt Mackall <mpm@selenic.com>
parents: 4972
diff changeset
   317
        data = fp.read()
a386a6e4fe46 revlog: simplify the v0 parser
Matt Mackall <mpm@selenic.com>
parents: 4972
diff changeset
   318
        l = len(data)
a386a6e4fe46 revlog: simplify the v0 parser
Matt Mackall <mpm@selenic.com>
parents: 4972
diff changeset
   319
        while off + s <= l:
a386a6e4fe46 revlog: simplify the v0 parser
Matt Mackall <mpm@selenic.com>
parents: 4972
diff changeset
   320
            cur = data[off:off + s]
a386a6e4fe46 revlog: simplify the v0 parser
Matt Mackall <mpm@selenic.com>
parents: 4972
diff changeset
   321
            off += s
5007
3addf4531643 revlog: localize some fastpath functions
Matt Mackall <mpm@selenic.com>
parents: 5006
diff changeset
   322
            e = _unpack(indexformatv0, cur)
4977
6cb30bc4ca32 revlog: parse revlogv0 indexes into v1 internally
Matt Mackall <mpm@selenic.com>
parents: 4976
diff changeset
   323
            # transform to revlogv1 format
6cb30bc4ca32 revlog: parse revlogv0 indexes into v1 internally
Matt Mackall <mpm@selenic.com>
parents: 4976
diff changeset
   324
            e2 = (offset_type(e[0], 0), e[1], -1, e[2], e[3],
6cb30bc4ca32 revlog: parse revlogv0 indexes into v1 internally
Matt Mackall <mpm@selenic.com>
parents: 4976
diff changeset
   325
                  nodemap[e[4]], nodemap[e[5]], e[6])
6cb30bc4ca32 revlog: parse revlogv0 indexes into v1 internally
Matt Mackall <mpm@selenic.com>
parents: 4976
diff changeset
   326
            index.append(e2)
6cb30bc4ca32 revlog: parse revlogv0 indexes into v1 internally
Matt Mackall <mpm@selenic.com>
parents: 4976
diff changeset
   327
            nodemap[e[6]] = n
4973
a386a6e4fe46 revlog: simplify the v0 parser
Matt Mackall <mpm@selenic.com>
parents: 4972
diff changeset
   328
            n += 1
4972
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   329
4983
4dbcfc6e359e revlog: pull chunkcache back into revlog
Matt Mackall <mpm@selenic.com>
parents: 4982
diff changeset
   330
        return index, nodemap, None
4972
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   331
5338
f87685355c9c revlog: fix revlogio.packentry corner case
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5325
diff changeset
   332
    def packentry(self, entry, node, version, rev):
4986
58cc017ec7e0 revlog: abstract out index entry packing
Matt Mackall <mpm@selenic.com>
parents: 4985
diff changeset
   333
        e2 = (getoffset(entry[0]), entry[1], entry[3], entry[4],
58cc017ec7e0 revlog: abstract out index entry packing
Matt Mackall <mpm@selenic.com>
parents: 4985
diff changeset
   334
              node(entry[5]), node(entry[6]), entry[7])
5007
3addf4531643 revlog: localize some fastpath functions
Matt Mackall <mpm@selenic.com>
parents: 5006
diff changeset
   335
        return _pack(indexformatv0, *e2)
4986
58cc017ec7e0 revlog: abstract out index entry packing
Matt Mackall <mpm@selenic.com>
parents: 4985
diff changeset
   336
4987
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
   337
# index ng:
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
   338
# 6 bytes offset
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
   339
# 2 bytes flags
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
   340
# 4 bytes compressed length
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
   341
# 4 bytes uncompressed length
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
   342
# 4 bytes: base rev
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
   343
# 4 bytes link rev
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
   344
# 4 bytes parent 1 rev
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
   345
# 4 bytes parent 2 rev
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
   346
# 32 bytes: nodeid
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
   347
indexformatng = ">Qiiiiii20s12x"
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
   348
ngshaoffset = 32
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
   349
versionformat = ">I"
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
   350
4972
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   351
class revlogio(object):
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   352
    def __init__(self):
4977
6cb30bc4ca32 revlog: parse revlogv0 indexes into v1 internally
Matt Mackall <mpm@selenic.com>
parents: 4976
diff changeset
   353
        self.size = struct.calcsize(indexformatng)
4972
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   354
4985
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
   355
    def parseindex(self, fp, inline):
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
   356
        try:
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
   357
            size = util.fstat(fp).st_size
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
   358
        except AttributeError:
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
   359
            size = 0
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
   360
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
   361
        if lazyparser.safe_to_use and not inline and size > 1000000:
4972
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   362
            # big index, let's parse it on demand
4985
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
   363
            parser = lazyparser(fp, size)
4972
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   364
            index = lazyindex(parser)
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   365
            nodemap = lazymap(parser)
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   366
            e = list(index[0])
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   367
            type = gettype(e[0])
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   368
            e[0] = offset_type(0, type)
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   369
            index[0] = e
4983
4dbcfc6e359e revlog: pull chunkcache back into revlog
Matt Mackall <mpm@selenic.com>
parents: 4982
diff changeset
   370
            return index, nodemap, None
4972
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   371
4977
6cb30bc4ca32 revlog: parse revlogv0 indexes into v1 internally
Matt Mackall <mpm@selenic.com>
parents: 4976
diff changeset
   372
        s = self.size
4983
4dbcfc6e359e revlog: pull chunkcache back into revlog
Matt Mackall <mpm@selenic.com>
parents: 4982
diff changeset
   373
        cache = None
4972
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   374
        index = []
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   375
        nodemap =  {nullid: nullrev}
4975
8b7e480a7603 revlog: simplify the v1 immediate parser
Matt Mackall <mpm@selenic.com>
parents: 4974
diff changeset
   376
        n = off = 0
8b7e480a7603 revlog: simplify the v1 immediate parser
Matt Mackall <mpm@selenic.com>
parents: 4974
diff changeset
   377
        # if we're not using lazymap, always read the whole index
8b7e480a7603 revlog: simplify the v1 immediate parser
Matt Mackall <mpm@selenic.com>
parents: 4974
diff changeset
   378
        data = fp.read()
4990
4491125c0f21 revlogio: speed up parsing
Matt Mackall <mpm@selenic.com>
parents: 4989
diff changeset
   379
        l = len(data) - s
4491125c0f21 revlogio: speed up parsing
Matt Mackall <mpm@selenic.com>
parents: 4989
diff changeset
   380
        append = index.append
4975
8b7e480a7603 revlog: simplify the v1 immediate parser
Matt Mackall <mpm@selenic.com>
parents: 4974
diff changeset
   381
        if inline:
4983
4dbcfc6e359e revlog: pull chunkcache back into revlog
Matt Mackall <mpm@selenic.com>
parents: 4982
diff changeset
   382
            cache = (0, data)
4990
4491125c0f21 revlogio: speed up parsing
Matt Mackall <mpm@selenic.com>
parents: 4989
diff changeset
   383
            while off <= l:
5007
3addf4531643 revlog: localize some fastpath functions
Matt Mackall <mpm@selenic.com>
parents: 5006
diff changeset
   384
                e = _unpack(indexformatng, data[off:off + s])
4990
4491125c0f21 revlogio: speed up parsing
Matt Mackall <mpm@selenic.com>
parents: 4989
diff changeset
   385
                nodemap[e[7]] = n
4491125c0f21 revlogio: speed up parsing
Matt Mackall <mpm@selenic.com>
parents: 4989
diff changeset
   386
                append(e)
4491125c0f21 revlogio: speed up parsing
Matt Mackall <mpm@selenic.com>
parents: 4989
diff changeset
   387
                n += 1
4975
8b7e480a7603 revlog: simplify the v1 immediate parser
Matt Mackall <mpm@selenic.com>
parents: 4974
diff changeset
   388
                if e[1] < 0:
4972
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   389
                    break
4990
4491125c0f21 revlogio: speed up parsing
Matt Mackall <mpm@selenic.com>
parents: 4989
diff changeset
   390
                off += e[1] + s
4491125c0f21 revlogio: speed up parsing
Matt Mackall <mpm@selenic.com>
parents: 4989
diff changeset
   391
        else:
4491125c0f21 revlogio: speed up parsing
Matt Mackall <mpm@selenic.com>
parents: 4989
diff changeset
   392
            while off <= l:
5007
3addf4531643 revlog: localize some fastpath functions
Matt Mackall <mpm@selenic.com>
parents: 5006
diff changeset
   393
                e = _unpack(indexformatng, data[off:off + s])
4990
4491125c0f21 revlogio: speed up parsing
Matt Mackall <mpm@selenic.com>
parents: 4989
diff changeset
   394
                nodemap[e[7]] = n
4491125c0f21 revlogio: speed up parsing
Matt Mackall <mpm@selenic.com>
parents: 4989
diff changeset
   395
                append(e)
4491125c0f21 revlogio: speed up parsing
Matt Mackall <mpm@selenic.com>
parents: 4989
diff changeset
   396
                n += 1
4491125c0f21 revlogio: speed up parsing
Matt Mackall <mpm@selenic.com>
parents: 4989
diff changeset
   397
                off += s
4972
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   398
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   399
        e = list(index[0])
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   400
        type = gettype(e[0])
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   401
        e[0] = offset_type(0, type)
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   402
        index[0] = e
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   403
4983
4dbcfc6e359e revlog: pull chunkcache back into revlog
Matt Mackall <mpm@selenic.com>
parents: 4982
diff changeset
   404
        return index, nodemap, cache
4972
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   405
5338
f87685355c9c revlog: fix revlogio.packentry corner case
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5325
diff changeset
   406
    def packentry(self, entry, node, version, rev):
5007
3addf4531643 revlog: localize some fastpath functions
Matt Mackall <mpm@selenic.com>
parents: 5006
diff changeset
   407
        p = _pack(indexformatng, *entry)
5338
f87685355c9c revlog: fix revlogio.packentry corner case
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5325
diff changeset
   408
        if rev == 0:
5007
3addf4531643 revlog: localize some fastpath functions
Matt Mackall <mpm@selenic.com>
parents: 5006
diff changeset
   409
            p = _pack(versionformat, version) + p[4:]
4986
58cc017ec7e0 revlog: abstract out index entry packing
Matt Mackall <mpm@selenic.com>
parents: 4985
diff changeset
   410
        return p
58cc017ec7e0 revlog: abstract out index entry packing
Matt Mackall <mpm@selenic.com>
parents: 4985
diff changeset
   411
1559
59b3639df0a9 Convert all classes to new-style classes by deriving them from object.
Eric Hopper <hopper@omnifarious.org>
parents: 1551
diff changeset
   412
class revlog(object):
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   413
    """
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   414
    the underlying revision storage object
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   415
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   416
    A revlog consists of two parts, an index and the revision data.
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   417
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   418
    The index is a file with a fixed record size containing
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   419
    information on each revision, includings its nodeid (hash), the
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   420
    nodeids of its parents, the position and offset of its data within
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   421
    the data file, and the revision it's based on. Finally, each entry
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   422
    contains a linkrev entry that can serve as a pointer to external
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   423
    data.
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   424
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   425
    The revision data itself is a linear collection of data chunks.
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   426
    Each chunk represents a revision and is usually represented as a
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   427
    delta against the previous chunk. To bound lookup time, runs of
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   428
    deltas are limited to about 2 times the length of the original
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   429
    version data. This makes retrieval of a version proportional to
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   430
    its size, or O(1) relative to the number of revisions.
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   431
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   432
    Both pieces of the revlog are written to in an append-only
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   433
    fashion, which means we never need to rewrite a file to insert or
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   434
    remove data, and can use some simple techniques to avoid the need
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   435
    for locking while reading.
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   436
    """
4258
b11a2fb59cf5 revlog: simplify revlog version handling
Matt Mackall <mpm@selenic.com>
parents: 4257
diff changeset
   437
    def __init__(self, opener, indexfile):
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   438
        """
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   439
        create a revlog object
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   440
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   441
        opener is a function that abstracts the file opening operation
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   442
        and can be used to implement COW semantics or the like.
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   443
        """
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
   444
        self.indexfile = indexfile
4257
1b5c38e9d7aa revlog: don't pass datafile as an argument
Matt Mackall <mpm@selenic.com>
parents: 4224
diff changeset
   445
        self.datafile = indexfile[:-2] + ".d"
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
   446
        self.opener = opener
4984
b4066fcbd6ba revlog: mark cache private
Matt Mackall <mpm@selenic.com>
parents: 4983
diff changeset
   447
        self._cache = None
4983
4dbcfc6e359e revlog: pull chunkcache back into revlog
Matt Mackall <mpm@selenic.com>
parents: 4982
diff changeset
   448
        self._chunkcache = None
4985
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
   449
        self.nodemap = {nullid: nullrev}
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
   450
        self.index = []
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
   451
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
   452
        v = REVLOG_DEFAULT_VERSION
4258
b11a2fb59cf5 revlog: simplify revlog version handling
Matt Mackall <mpm@selenic.com>
parents: 4257
diff changeset
   453
        if hasattr(opener, "defversion"):
4985
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
   454
            v = opener.defversion
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
   455
            if v & REVLOGNG:
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
   456
                v |= REVLOGNGINLINEDATA
116
e484cd5ec282 Only use lazy indexing for big indices and avoid the overhead of the
mpm@selenic.com
parents: 115
diff changeset
   457
4985
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
   458
        i = ""
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
   459
        try:
1784
2e0a288ca93e revalidate revlog data after locking the repo (issue132)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1749
diff changeset
   460
            f = self.opener(self.indexfile)
2079
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   461
            i = f.read(4)
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   462
            f.seek(0)
4918
e017d3a82e1d revlog: raise offset/type helpers to global scope
Matt Mackall <mpm@selenic.com>
parents: 4746
diff changeset
   463
            if len(i) > 0:
e017d3a82e1d revlog: raise offset/type helpers to global scope
Matt Mackall <mpm@selenic.com>
parents: 4746
diff changeset
   464
                v = struct.unpack(versionformat, i)[0]
1322
b3d44e9b3092 Make revlog constructor more discerning in its treatment of errors.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1232
diff changeset
   465
        except IOError, inst:
b3d44e9b3092 Make revlog constructor more discerning in its treatment of errors.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1232
diff changeset
   466
            if inst.errno != errno.ENOENT:
b3d44e9b3092 Make revlog constructor more discerning in its treatment of errors.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1232
diff changeset
   467
                raise
4985
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
   468
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
   469
        self.version = v
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
   470
        self._inline = v & REVLOGNGINLINEDATA
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
   471
        flags = v & ~0xFFFF
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
   472
        fmt = v & 0xFFFF
4985
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
   473
        if fmt == REVLOGV0 and flags:
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
   474
            raise RevlogError(_("index %s unknown flags %#04x for format v0")
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
   475
                              % (self.indexfile, flags >> 16))
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
   476
        elif fmt == REVLOGNG and flags & ~REVLOGNGINLINEDATA:
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
   477
            raise RevlogError(_("index %s unknown flags %#04x for revlogng")
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
   478
                              % (self.indexfile, flags >> 16))
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
   479
        elif fmt > REVLOGNG:
3743
3a099154b110 Make revlog error slightly less scary
Matt Mackall <mpm@selenic.com>
parents: 3683
diff changeset
   480
            raise RevlogError(_("index %s unknown format %d")
3680
69cf255a55a1 Indentation cleanups for 2956948b81f3.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 3679
diff changeset
   481
                              % (self.indexfile, fmt))
4985
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
   482
4972
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   483
        self._io = revlogio()
4971
3e6dae278c99 revlog: regroup parsing code
Matt Mackall <mpm@selenic.com>
parents: 4920
diff changeset
   484
        if self.version == REVLOGV0:
4972
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   485
            self._io = revlogoldio()
2072
74d3f5336b66 Implement revlogng.
mason@suse.com
parents: 2002
diff changeset
   486
        if i:
4985
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
   487
            d = self._io.parseindex(f, self._inline)
4983
4dbcfc6e359e revlog: pull chunkcache back into revlog
Matt Mackall <mpm@selenic.com>
parents: 4982
diff changeset
   488
            self.index, self.nodemap, self._chunkcache = d
4985
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
   489
4979
06abdaf78788 revlog: add a magic null revision to our index
Matt Mackall <mpm@selenic.com>
parents: 4978
diff changeset
   490
        # add the magic null revision at -1
06abdaf78788 revlog: add a magic null revision to our index
Matt Mackall <mpm@selenic.com>
parents: 4978
diff changeset
   491
        self.index.append((0, 0, 0, -1, -1, -1, -1, nullid))
116
e484cd5ec282 Only use lazy indexing for big indices and avoid the overhead of the
mpm@selenic.com
parents: 115
diff changeset
   492
4920
ee983d0dbea8 revlog: privatize some methods
Matt Mackall <mpm@selenic.com>
parents: 4919
diff changeset
   493
    def _loadindex(self, start, end):
2079
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   494
        """load a block of indexes all at once from the lazy parser"""
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   495
        if isinstance(self.index, lazyindex):
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   496
            self.index.p.loadindex(start, end)
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   497
4920
ee983d0dbea8 revlog: privatize some methods
Matt Mackall <mpm@selenic.com>
parents: 4919
diff changeset
   498
    def _loadindexmap(self):
2072
74d3f5336b66 Implement revlogng.
mason@suse.com
parents: 2002
diff changeset
   499
        """loads both the map and the index from the lazy parser"""
74d3f5336b66 Implement revlogng.
mason@suse.com
parents: 2002
diff changeset
   500
        if isinstance(self.index, lazyindex):
74d3f5336b66 Implement revlogng.
mason@suse.com
parents: 2002
diff changeset
   501
            p = self.index.p
2079
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   502
            p.loadindex()
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   503
            self.nodemap = p.map
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   504
4920
ee983d0dbea8 revlog: privatize some methods
Matt Mackall <mpm@selenic.com>
parents: 4919
diff changeset
   505
    def _loadmap(self):
2079
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   506
        """loads the map from the lazy parser"""
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   507
        if isinstance(self.nodemap, lazymap):
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   508
            self.nodemap.p.loadmap()
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   509
            self.nodemap = self.nodemap.p.map
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
   510
4980
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
   511
    def tip(self):
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
   512
        return self.node(len(self.index) - 2)
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
   513
    def count(self):
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
   514
        return len(self.index) - 1
4920
ee983d0dbea8 revlog: privatize some methods
Matt Mackall <mpm@selenic.com>
parents: 4919
diff changeset
   515
1201
59bfbdbc38f6 revlog: raise informative exception if file is missing.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1099
diff changeset
   516
    def rev(self, node):
59bfbdbc38f6 revlog: raise informative exception if file is missing.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1099
diff changeset
   517
        try:
59bfbdbc38f6 revlog: raise informative exception if file is missing.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1099
diff changeset
   518
            return self.nodemap[node]
59bfbdbc38f6 revlog: raise informative exception if file is missing.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1099
diff changeset
   519
        except KeyError:
3930
01d98d68d697 Add revlog.LookupError exception, and use it instead of RevlogError.
Brendan Cully <brendan@kublai.com>
parents: 3928
diff changeset
   520
            raise LookupError(_('%s: no node %s') % (self.indexfile, hex(node)))
4980
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
   521
    def node(self, rev):
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
   522
        return self.index[rev][7]
2642
6414ee2eb688 correct the handling of linkrev with nullid
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 2641
diff changeset
   523
    def linkrev(self, node):
4979
06abdaf78788 revlog: add a magic null revision to our index
Matt Mackall <mpm@selenic.com>
parents: 4978
diff changeset
   524
        return self.index[self.rev(node)][4]
2
ecf3fd948051 Handle nullid better for ancestor
mpm@selenic.com
parents: 0
diff changeset
   525
    def parents(self, node):
4979
06abdaf78788 revlog: add a magic null revision to our index
Matt Mackall <mpm@selenic.com>
parents: 4978
diff changeset
   526
        d = self.index[self.rev(node)][5:7]
3508
0aef94f45ebf revlog.py: always return tuples from parents and parentrevs
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 3490
diff changeset
   527
        return (self.node(d[0]), self.node(d[1]))
2489
568e58eed096 Add revlog.parentrevs function.
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 2354
diff changeset
   528
    def parentrevs(self, rev):
4979
06abdaf78788 revlog: add a magic null revision to our index
Matt Mackall <mpm@selenic.com>
parents: 4978
diff changeset
   529
        return self.index[rev][5:7]
2072
74d3f5336b66 Implement revlogng.
mason@suse.com
parents: 2002
diff changeset
   530
    def start(self, rev):
5006
c2febf5420e9 revlog: minor chunk speed-up
Matt Mackall <mpm@selenic.com>
parents: 5005
diff changeset
   531
        return int(self.index[rev][0] >> 16)
4980
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
   532
    def end(self, rev):
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
   533
        return self.start(rev) + self.length(rev)
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
   534
    def length(self, rev):
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
   535
        return self.index[rev][1]
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
   536
    def base(self, rev):
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
   537
        return self.index[rev][3]
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
   538
2078
441ea218414e Fill in the uncompressed size during revlog.addgroup
mason@suse.com
parents: 2077
diff changeset
   539
    def size(self, rev):
441ea218414e Fill in the uncompressed size during revlog.addgroup
mason@suse.com
parents: 2077
diff changeset
   540
        """return the length of the uncompressed text for a given revision"""
4977
6cb30bc4ca32 revlog: parse revlogv0 indexes into v1 internally
Matt Mackall <mpm@selenic.com>
parents: 4976
diff changeset
   541
        l = self.index[rev][2]
2078
441ea218414e Fill in the uncompressed size during revlog.addgroup
mason@suse.com
parents: 2077
diff changeset
   542
        if l >= 0:
441ea218414e Fill in the uncompressed size during revlog.addgroup
mason@suse.com
parents: 2077
diff changeset
   543
            return l
441ea218414e Fill in the uncompressed size during revlog.addgroup
mason@suse.com
parents: 2077
diff changeset
   544
441ea218414e Fill in the uncompressed size during revlog.addgroup
mason@suse.com
parents: 2077
diff changeset
   545
        t = self.revision(self.node(rev))
441ea218414e Fill in the uncompressed size during revlog.addgroup
mason@suse.com
parents: 2077
diff changeset
   546
        return len(t)
441ea218414e Fill in the uncompressed size during revlog.addgroup
mason@suse.com
parents: 2077
diff changeset
   547
441ea218414e Fill in the uncompressed size during revlog.addgroup
mason@suse.com
parents: 2077
diff changeset
   548
        # alternate implementation, The advantage to this code is it
441ea218414e Fill in the uncompressed size during revlog.addgroup
mason@suse.com
parents: 2077
diff changeset
   549
        # will be faster for a single revision.  But, the results are not
441ea218414e Fill in the uncompressed size during revlog.addgroup
mason@suse.com
parents: 2077
diff changeset
   550
        # cached, so finding the size of every revision will be slower.
441ea218414e Fill in the uncompressed size during revlog.addgroup
mason@suse.com
parents: 2077
diff changeset
   551
        """
441ea218414e Fill in the uncompressed size during revlog.addgroup
mason@suse.com
parents: 2077
diff changeset
   552
        if self.cache and self.cache[1] == rev:
441ea218414e Fill in the uncompressed size during revlog.addgroup
mason@suse.com
parents: 2077
diff changeset
   553
            return len(self.cache[2])
441ea218414e Fill in the uncompressed size during revlog.addgroup
mason@suse.com
parents: 2077
diff changeset
   554
441ea218414e Fill in the uncompressed size during revlog.addgroup
mason@suse.com
parents: 2077
diff changeset
   555
        base = self.base(rev)
441ea218414e Fill in the uncompressed size during revlog.addgroup
mason@suse.com
parents: 2077
diff changeset
   556
        if self.cache and self.cache[1] >= base and self.cache[1] < rev:
441ea218414e Fill in the uncompressed size during revlog.addgroup
mason@suse.com
parents: 2077
diff changeset
   557
            base = self.cache[1]
441ea218414e Fill in the uncompressed size during revlog.addgroup
mason@suse.com
parents: 2077
diff changeset
   558
            text = self.cache[2]
441ea218414e Fill in the uncompressed size during revlog.addgroup
mason@suse.com
parents: 2077
diff changeset
   559
        else:
441ea218414e Fill in the uncompressed size during revlog.addgroup
mason@suse.com
parents: 2077
diff changeset
   560
            text = self.revision(self.node(base))
441ea218414e Fill in the uncompressed size during revlog.addgroup
mason@suse.com
parents: 2077
diff changeset
   561
441ea218414e Fill in the uncompressed size during revlog.addgroup
mason@suse.com
parents: 2077
diff changeset
   562
        l = len(text)
441ea218414e Fill in the uncompressed size during revlog.addgroup
mason@suse.com
parents: 2077
diff changeset
   563
        for x in xrange(base + 1, rev + 1):
441ea218414e Fill in the uncompressed size during revlog.addgroup
mason@suse.com
parents: 2077
diff changeset
   564
            l = mdiff.patchedsize(l, self.chunk(x))
441ea218414e Fill in the uncompressed size during revlog.addgroup
mason@suse.com
parents: 2077
diff changeset
   565
        return l
441ea218414e Fill in the uncompressed size during revlog.addgroup
mason@suse.com
parents: 2077
diff changeset
   566
        """
441ea218414e Fill in the uncompressed size during revlog.addgroup
mason@suse.com
parents: 2077
diff changeset
   567
3630
508036290b00 revlog: reachable actually takes a node
Matt Mackall <mpm@selenic.com>
parents: 3585
diff changeset
   568
    def reachable(self, node, stop=None):
3683
b0fd43effdb3 add docstring to reachable
Matt Mackall <mpm@selenic.com>
parents: 3680
diff changeset
   569
        """return a hash of all nodes ancestral to a given node, including
b0fd43effdb3 add docstring to reachable
Matt Mackall <mpm@selenic.com>
parents: 3680
diff changeset
   570
         the node itself, stopping when stop is matched"""
1074
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
   571
        reachable = {}
3630
508036290b00 revlog: reachable actually takes a node
Matt Mackall <mpm@selenic.com>
parents: 3585
diff changeset
   572
        visit = [node]
508036290b00 revlog: reachable actually takes a node
Matt Mackall <mpm@selenic.com>
parents: 3585
diff changeset
   573
        reachable[node] = 1
1074
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
   574
        if stop:
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
   575
            stopn = self.rev(stop)
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
   576
        else:
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
   577
            stopn = 0
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
   578
        while visit:
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
   579
            n = visit.pop(0)
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
   580
            if n == stop:
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
   581
                continue
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
   582
            if n == nullid:
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
   583
                continue
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
   584
            for p in self.parents(n):
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
   585
                if self.rev(p) < stopn:
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
   586
                    continue
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
   587
                if p not in reachable:
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
   588
                    reachable[p] = 1
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
   589
                    visit.append(p)
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
   590
        return reachable
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
   591
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   592
    def nodesbetween(self, roots=None, heads=None):
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   593
        """Return a tuple containing three elements. Elements 1 and 2 contain
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   594
        a final list bases and heads after all the unreachable ones have been
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   595
        pruned.  Element 0 contains a topologically sorted list of all
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   596
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   597
        nodes that satisfy these constraints:
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   598
        1. All nodes must be descended from a node in roots (the nodes on
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   599
           roots are considered descended from themselves).
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   600
        2. All nodes must also be ancestors of a node in heads (the nodes in
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   601
           heads are considered to be their own ancestors).
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   602
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   603
        If roots is unspecified, nullid is assumed as the only root.
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   604
        If heads is unspecified, it is taken to be the output of the
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   605
        heads method (i.e. a list of all nodes in the repository that
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   606
        have no children)."""
1463
26e73acc0cdf Fix to handle case of empty list for roots or heads in nodesbetween.
Eric Hopper <hopper@omnifarious.org>
parents: 1459
diff changeset
   607
        nonodes = ([], [], [])
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   608
        if roots is not None:
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   609
            roots = list(roots)
1463
26e73acc0cdf Fix to handle case of empty list for roots or heads in nodesbetween.
Eric Hopper <hopper@omnifarious.org>
parents: 1459
diff changeset
   610
            if not roots:
26e73acc0cdf Fix to handle case of empty list for roots or heads in nodesbetween.
Eric Hopper <hopper@omnifarious.org>
parents: 1459
diff changeset
   611
                return nonodes
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   612
            lowestrev = min([self.rev(n) for n in roots])
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   613
        else:
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   614
            roots = [nullid] # Everybody's a descendent of nullid
3578
3b4e00cba57a Define and use nullrev (revision of nullid) instead of -1.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 3508
diff changeset
   615
            lowestrev = nullrev
3b4e00cba57a Define and use nullrev (revision of nullid) instead of -1.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 3508
diff changeset
   616
        if (lowestrev == nullrev) and (heads is None):
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   617
            # We want _all_ the nodes!
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   618
            return ([self.node(r) for r in xrange(0, self.count())],
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   619
                    [nullid], list(self.heads()))
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   620
        if heads is None:
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   621
            # All nodes are ancestors, so the latest ancestor is the last
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   622
            # node.
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   623
            highestrev = self.count() - 1
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   624
            # Set ancestors to None to signal that every node is an ancestor.
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   625
            ancestors = None
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   626
            # Set heads to an empty dictionary for later discovery of heads
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   627
            heads = {}
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   628
        else:
1463
26e73acc0cdf Fix to handle case of empty list for roots or heads in nodesbetween.
Eric Hopper <hopper@omnifarious.org>
parents: 1459
diff changeset
   629
            heads = list(heads)
26e73acc0cdf Fix to handle case of empty list for roots or heads in nodesbetween.
Eric Hopper <hopper@omnifarious.org>
parents: 1459
diff changeset
   630
            if not heads:
26e73acc0cdf Fix to handle case of empty list for roots or heads in nodesbetween.
Eric Hopper <hopper@omnifarious.org>
parents: 1459
diff changeset
   631
                return nonodes
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   632
            ancestors = {}
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   633
            # Turn heads into a dictionary so we can remove 'fake' heads.
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   634
            # Also, later we will be using it to filter out the heads we can't
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   635
            # find from roots.
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   636
            heads = dict.fromkeys(heads, 0)
3360
ef8307585b41 nodesbetween: fix a bug with duplicate heads
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 3335
diff changeset
   637
            # Start at the top and keep marking parents until we're done.
ef8307585b41 nodesbetween: fix a bug with duplicate heads
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 3335
diff changeset
   638
            nodestotag = heads.keys()
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   639
            # Remember where the top was so we can use it as a limit later.
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   640
            highestrev = max([self.rev(n) for n in nodestotag])
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   641
            while nodestotag:
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   642
                # grab a node to tag
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   643
                n = nodestotag.pop()
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   644
                # Never tag nullid
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   645
                if n == nullid:
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   646
                    continue
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   647
                # A node's revision number represents its place in a
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   648
                # topologically sorted list of nodes.
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   649
                r = self.rev(n)
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   650
                if r >= lowestrev:
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   651
                    if n not in ancestors:
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   652
                        # If we are possibly a descendent of one of the roots
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   653
                        # and we haven't already been marked as an ancestor
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   654
                        ancestors[n] = 1 # Mark as ancestor
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   655
                        # Add non-nullid parents to list of nodes to tag.
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   656
                        nodestotag.extend([p for p in self.parents(n) if
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   657
                                           p != nullid])
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   658
                    elif n in heads: # We've seen it before, is it a fake head?
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   659
                        # So it is, real heads should not be the ancestors of
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   660
                        # any other heads.
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   661
                        heads.pop(n)
1459
106fdec8e1fb Fix small bug in nodesbetween if heads is [nullid].
Eric Hopper <hopper@omnifarious.org>
parents: 1458
diff changeset
   662
            if not ancestors:
1463
26e73acc0cdf Fix to handle case of empty list for roots or heads in nodesbetween.
Eric Hopper <hopper@omnifarious.org>
parents: 1459
diff changeset
   663
                return nonodes
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   664
            # Now that we have our set of ancestors, we want to remove any
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   665
            # roots that are not ancestors.
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   666
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   667
            # If one of the roots was nullid, everything is included anyway.
3578
3b4e00cba57a Define and use nullrev (revision of nullid) instead of -1.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 3508
diff changeset
   668
            if lowestrev > nullrev:
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   669
                # But, since we weren't, let's recompute the lowest rev to not
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   670
                # include roots that aren't ancestors.
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   671
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   672
                # Filter out roots that aren't ancestors of heads
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   673
                roots = [n for n in roots if n in ancestors]
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   674
                # Recompute the lowest revision
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   675
                if roots:
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   676
                    lowestrev = min([self.rev(n) for n in roots])
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   677
                else:
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   678
                    # No more roots?  Return empty list
1463
26e73acc0cdf Fix to handle case of empty list for roots or heads in nodesbetween.
Eric Hopper <hopper@omnifarious.org>
parents: 1459
diff changeset
   679
                    return nonodes
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   680
            else:
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   681
                # We are descending from nullid, and don't need to care about
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   682
                # any other roots.
3578
3b4e00cba57a Define and use nullrev (revision of nullid) instead of -1.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 3508
diff changeset
   683
                lowestrev = nullrev
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   684
                roots = [nullid]
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   685
        # Transform our roots list into a 'set' (i.e. a dictionary where the
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   686
        # values don't matter.
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   687
        descendents = dict.fromkeys(roots, 1)
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   688
        # Also, keep the original roots so we can filter out roots that aren't
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   689
        # 'real' roots (i.e. are descended from other roots).
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   690
        roots = descendents.copy()
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   691
        # Our topologically sorted list of output nodes.
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   692
        orderedout = []
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   693
        # Don't start at nullid since we don't want nullid in our output list,
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   694
        # and if nullid shows up in descedents, empty parents will look like
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   695
        # they're descendents.
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   696
        for r in xrange(max(lowestrev, 0), highestrev + 1):
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   697
            n = self.node(r)
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   698
            isdescendent = False
3578
3b4e00cba57a Define and use nullrev (revision of nullid) instead of -1.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 3508
diff changeset
   699
            if lowestrev == nullrev:  # Everybody is a descendent of nullid
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   700
                isdescendent = True
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   701
            elif n in descendents:
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   702
                # n is already a descendent
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   703
                isdescendent = True
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   704
                # This check only needs to be done here because all the roots
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   705
                # will start being marked is descendents before the loop.
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   706
                if n in roots:
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   707
                    # If n was a root, check if it's a 'real' root.
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   708
                    p = tuple(self.parents(n))
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   709
                    # If any of its parents are descendents, it's not a root.
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   710
                    if (p[0] in descendents) or (p[1] in descendents):
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   711
                        roots.pop(n)
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   712
            else:
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   713
                p = tuple(self.parents(n))
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   714
                # A node is a descendent if either of its parents are
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   715
                # descendents.  (We seeded the dependents list with the roots
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   716
                # up there, remember?)
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   717
                if (p[0] in descendents) or (p[1] in descendents):
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   718
                    descendents[n] = 1
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   719
                    isdescendent = True
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   720
            if isdescendent and ((ancestors is None) or (n in ancestors)):
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   721
                # Only include nodes that are both descendents and ancestors.
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   722
                orderedout.append(n)
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   723
                if (ancestors is not None) and (n in heads):
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   724
                    # We're trying to figure out which heads are reachable
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   725
                    # from roots.
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   726
                    # Mark this head as having been reached
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   727
                    heads[n] = 1
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   728
                elif ancestors is None:
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   729
                    # Otherwise, we're trying to discover the heads.
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   730
                    # Assume this is a head because if it isn't, the next step
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   731
                    # will eventually remove it.
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   732
                    heads[n] = 1
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   733
                    # But, obviously its parents aren't.
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   734
                    for p in self.parents(n):
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   735
                        heads.pop(p, None)
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   736
        heads = [n for n in heads.iterkeys() if heads[n] != 0]
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   737
        roots = roots.keys()
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   738
        assert orderedout
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   739
        assert roots
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   740
        assert heads
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   741
        return (orderedout, roots, heads)
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   742
3923
27230c29bfec fix calculation of new heads added during push with -r
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 3755
diff changeset
   743
    def heads(self, start=None, stop=None):
1550
ccb9b62de892 add a -r/--rev option to heads to show only heads descendant from rev
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1535
diff changeset
   744
        """return the list of all nodes that have no children
1551
e793cbc8be00 Fixes to "hg heads -r FOO":
Thomas Arendsen Hein <thomas@intevation.de>
parents: 1550
diff changeset
   745
e793cbc8be00 Fixes to "hg heads -r FOO":
Thomas Arendsen Hein <thomas@intevation.de>
parents: 1550
diff changeset
   746
        if start is specified, only heads that are descendants of
e793cbc8be00 Fixes to "hg heads -r FOO":
Thomas Arendsen Hein <thomas@intevation.de>
parents: 1550
diff changeset
   747
        start will be returned
3923
27230c29bfec fix calculation of new heads added during push with -r
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 3755
diff changeset
   748
        if stop is specified, it will consider all the revs from stop
27230c29bfec fix calculation of new heads added during push with -r
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 3755
diff changeset
   749
        as if they had no children
1551
e793cbc8be00 Fixes to "hg heads -r FOO":
Thomas Arendsen Hein <thomas@intevation.de>
parents: 1550
diff changeset
   750
        """
4991
9c8c42bcf17a revlog: implement a fast path for heads
Matt Mackall <mpm@selenic.com>
parents: 4990
diff changeset
   751
        if start is None and stop is None:
9c8c42bcf17a revlog: implement a fast path for heads
Matt Mackall <mpm@selenic.com>
parents: 4990
diff changeset
   752
            count = self.count()
9c8c42bcf17a revlog: implement a fast path for heads
Matt Mackall <mpm@selenic.com>
parents: 4990
diff changeset
   753
            if not count:
9c8c42bcf17a revlog: implement a fast path for heads
Matt Mackall <mpm@selenic.com>
parents: 4990
diff changeset
   754
                return [nullid]
9c8c42bcf17a revlog: implement a fast path for heads
Matt Mackall <mpm@selenic.com>
parents: 4990
diff changeset
   755
            ishead = [1] * (count + 1)
9c8c42bcf17a revlog: implement a fast path for heads
Matt Mackall <mpm@selenic.com>
parents: 4990
diff changeset
   756
            index = self.index
9c8c42bcf17a revlog: implement a fast path for heads
Matt Mackall <mpm@selenic.com>
parents: 4990
diff changeset
   757
            for r in xrange(count):
9c8c42bcf17a revlog: implement a fast path for heads
Matt Mackall <mpm@selenic.com>
parents: 4990
diff changeset
   758
                e = index[r]
9c8c42bcf17a revlog: implement a fast path for heads
Matt Mackall <mpm@selenic.com>
parents: 4990
diff changeset
   759
                ishead[e[5]] = ishead[e[6]] = 0
9c8c42bcf17a revlog: implement a fast path for heads
Matt Mackall <mpm@selenic.com>
parents: 4990
diff changeset
   760
            return [self.node(r) for r in xrange(count) if ishead[r]]
9c8c42bcf17a revlog: implement a fast path for heads
Matt Mackall <mpm@selenic.com>
parents: 4990
diff changeset
   761
1551
e793cbc8be00 Fixes to "hg heads -r FOO":
Thomas Arendsen Hein <thomas@intevation.de>
parents: 1550
diff changeset
   762
        if start is None:
e793cbc8be00 Fixes to "hg heads -r FOO":
Thomas Arendsen Hein <thomas@intevation.de>
parents: 1550
diff changeset
   763
            start = nullid
3923
27230c29bfec fix calculation of new heads added during push with -r
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 3755
diff changeset
   764
        if stop is None:
27230c29bfec fix calculation of new heads added during push with -r
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 3755
diff changeset
   765
            stop = []
27230c29bfec fix calculation of new heads added during push with -r
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 3755
diff changeset
   766
        stoprevs = dict.fromkeys([self.rev(n) for n in stop])
1550
ccb9b62de892 add a -r/--rev option to heads to show only heads descendant from rev
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1535
diff changeset
   767
        startrev = self.rev(start)
2490
6ff82ec1f4b8 Change revlog.heads to walk the revision graph using revision numbers
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 2489
diff changeset
   768
        reachable = {startrev: 1}
6ff82ec1f4b8 Change revlog.heads to walk the revision graph using revision numbers
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 2489
diff changeset
   769
        heads = {startrev: 1}
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   770
2490
6ff82ec1f4b8 Change revlog.heads to walk the revision graph using revision numbers
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 2489
diff changeset
   771
        parentrevs = self.parentrevs
1550
ccb9b62de892 add a -r/--rev option to heads to show only heads descendant from rev
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1535
diff changeset
   772
        for r in xrange(startrev + 1, self.count()):
2490
6ff82ec1f4b8 Change revlog.heads to walk the revision graph using revision numbers
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 2489
diff changeset
   773
            for p in parentrevs(r):
6ff82ec1f4b8 Change revlog.heads to walk the revision graph using revision numbers
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 2489
diff changeset
   774
                if p in reachable:
3923
27230c29bfec fix calculation of new heads added during push with -r
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 3755
diff changeset
   775
                    if r not in stoprevs:
27230c29bfec fix calculation of new heads added during push with -r
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 3755
diff changeset
   776
                        reachable[r] = 1
2490
6ff82ec1f4b8 Change revlog.heads to walk the revision graph using revision numbers
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 2489
diff changeset
   777
                    heads[r] = 1
3923
27230c29bfec fix calculation of new heads added during push with -r
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 3755
diff changeset
   778
                if p in heads and p not in stoprevs:
2490
6ff82ec1f4b8 Change revlog.heads to walk the revision graph using revision numbers
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 2489
diff changeset
   779
                    del heads[p]
3923
27230c29bfec fix calculation of new heads added during push with -r
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 3755
diff changeset
   780
2490
6ff82ec1f4b8 Change revlog.heads to walk the revision graph using revision numbers
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 2489
diff changeset
   781
        return [self.node(r) for r in heads]
370
c90385d82aec revlog: add a children function
mpm@selenic.com
parents: 330
diff changeset
   782
c90385d82aec revlog: add a children function
mpm@selenic.com
parents: 330
diff changeset
   783
    def children(self, node):
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   784
        """find the children of a given node"""
370
c90385d82aec revlog: add a children function
mpm@selenic.com
parents: 330
diff changeset
   785
        c = []
c90385d82aec revlog: add a children function
mpm@selenic.com
parents: 330
diff changeset
   786
        p = self.rev(node)
c90385d82aec revlog: add a children function
mpm@selenic.com
parents: 330
diff changeset
   787
        for r in range(p + 1, self.count()):
4746
62c56d8f368b Fix revlog.children so the real children of the null revision can be calculated.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4635
diff changeset
   788
            prevs = [pr for pr in self.parentrevs(r) if pr != nullrev]
62c56d8f368b Fix revlog.children so the real children of the null revision can be calculated.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4635
diff changeset
   789
            if prevs:
62c56d8f368b Fix revlog.children so the real children of the null revision can be calculated.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4635
diff changeset
   790
                for pr in prevs:
62c56d8f368b Fix revlog.children so the real children of the null revision can be calculated.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4635
diff changeset
   791
                    if pr == p:
62c56d8f368b Fix revlog.children so the real children of the null revision can be calculated.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4635
diff changeset
   792
                        c.append(self.node(r))
62c56d8f368b Fix revlog.children so the real children of the null revision can be calculated.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4635
diff changeset
   793
            elif p == nullrev:
62c56d8f368b Fix revlog.children so the real children of the null revision can be calculated.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4635
diff changeset
   794
                c.append(self.node(r))
370
c90385d82aec revlog: add a children function
mpm@selenic.com
parents: 330
diff changeset
   795
        return c
515
03f27b1381f9 Whitespace cleanups
mpm@selenic.com
parents: 484
diff changeset
   796
3453
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
   797
    def _match(self, id):
3210
7240f9e47144 correctly find the type of 'id' in revlog.lookup
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 3157
diff changeset
   798
        if isinstance(id, (long, int)):
3156
d01e4cb2f5f2 cleanups in revlog.lookup
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 3139
diff changeset
   799
            # rev
2641
156fb1feab62 lookup should allow -1 to represent nullid (if passed an int as arg)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 2600
diff changeset
   800
            return self.node(id)
3438
b17f9d3eda74 revlog.lookup tweaks
Matt Mackall <mpm@selenic.com>
parents: 3390
diff changeset
   801
        if len(id) == 20:
b17f9d3eda74 revlog.lookup tweaks
Matt Mackall <mpm@selenic.com>
parents: 3390
diff changeset
   802
            # possibly a binary node
b17f9d3eda74 revlog.lookup tweaks
Matt Mackall <mpm@selenic.com>
parents: 3390
diff changeset
   803
            # odds of a binary node being all hex in ASCII are 1 in 10**25
b17f9d3eda74 revlog.lookup tweaks
Matt Mackall <mpm@selenic.com>
parents: 3390
diff changeset
   804
            try:
b17f9d3eda74 revlog.lookup tweaks
Matt Mackall <mpm@selenic.com>
parents: 3390
diff changeset
   805
                node = id
b17f9d3eda74 revlog.lookup tweaks
Matt Mackall <mpm@selenic.com>
parents: 3390
diff changeset
   806
                r = self.rev(node) # quick search the index
b17f9d3eda74 revlog.lookup tweaks
Matt Mackall <mpm@selenic.com>
parents: 3390
diff changeset
   807
                return node
3930
01d98d68d697 Add revlog.LookupError exception, and use it instead of RevlogError.
Brendan Cully <brendan@kublai.com>
parents: 3928
diff changeset
   808
            except LookupError:
3438
b17f9d3eda74 revlog.lookup tweaks
Matt Mackall <mpm@selenic.com>
parents: 3390
diff changeset
   809
                pass # may be partial hex id
36
da28286bf6b7 Add smart node lookup by substring or by rev number
mpm@selenic.com
parents: 26
diff changeset
   810
        try:
3156
d01e4cb2f5f2 cleanups in revlog.lookup
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 3139
diff changeset
   811
            # str(rev)
36
da28286bf6b7 Add smart node lookup by substring or by rev number
mpm@selenic.com
parents: 26
diff changeset
   812
            rev = int(id)
4980
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
   813
            if str(rev) != id:
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
   814
                raise ValueError
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
   815
            if rev < 0:
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
   816
                rev = self.count() + rev
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
   817
            if rev < 0 or rev >= self.count():
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
   818
                raise ValueError
36
da28286bf6b7 Add smart node lookup by substring or by rev number
mpm@selenic.com
parents: 26
diff changeset
   819
            return self.node(rev)
469
e205194ca7ef Various node id lookup tweaks
mpm@selenic.com
parents: 451
diff changeset
   820
        except (ValueError, OverflowError):
3156
d01e4cb2f5f2 cleanups in revlog.lookup
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 3139
diff changeset
   821
            pass
3453
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
   822
        if len(id) == 40:
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
   823
            try:
3438
b17f9d3eda74 revlog.lookup tweaks
Matt Mackall <mpm@selenic.com>
parents: 3390
diff changeset
   824
                # a full hex nodeid?
b17f9d3eda74 revlog.lookup tweaks
Matt Mackall <mpm@selenic.com>
parents: 3390
diff changeset
   825
                node = bin(id)
b17f9d3eda74 revlog.lookup tweaks
Matt Mackall <mpm@selenic.com>
parents: 3390
diff changeset
   826
                r = self.rev(node)
3157
4fe41a9e4591 optimize revlog.lookup when passed hex(node)[:...]
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 3156
diff changeset
   827
                return node
3453
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
   828
            except TypeError:
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
   829
                pass
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
   830
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
   831
    def _partialmatch(self, id):
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
   832
        if len(id) < 40:
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
   833
            try:
3438
b17f9d3eda74 revlog.lookup tweaks
Matt Mackall <mpm@selenic.com>
parents: 3390
diff changeset
   834
                # hex(node)[:...]
b17f9d3eda74 revlog.lookup tweaks
Matt Mackall <mpm@selenic.com>
parents: 3390
diff changeset
   835
                bin_id = bin(id[:len(id) & ~1]) # grab an even number of digits
b17f9d3eda74 revlog.lookup tweaks
Matt Mackall <mpm@selenic.com>
parents: 3390
diff changeset
   836
                node = None
b17f9d3eda74 revlog.lookup tweaks
Matt Mackall <mpm@selenic.com>
parents: 3390
diff changeset
   837
                for n in self.nodemap:
b17f9d3eda74 revlog.lookup tweaks
Matt Mackall <mpm@selenic.com>
parents: 3390
diff changeset
   838
                    if n.startswith(bin_id) and hex(n).startswith(id):
b17f9d3eda74 revlog.lookup tweaks
Matt Mackall <mpm@selenic.com>
parents: 3390
diff changeset
   839
                        if node is not None:
3930
01d98d68d697 Add revlog.LookupError exception, and use it instead of RevlogError.
Brendan Cully <brendan@kublai.com>
parents: 3928
diff changeset
   840
                            raise LookupError(_("Ambiguous identifier"))
3438
b17f9d3eda74 revlog.lookup tweaks
Matt Mackall <mpm@selenic.com>
parents: 3390
diff changeset
   841
                        node = n
b17f9d3eda74 revlog.lookup tweaks
Matt Mackall <mpm@selenic.com>
parents: 3390
diff changeset
   842
                if node is not None:
b17f9d3eda74 revlog.lookup tweaks
Matt Mackall <mpm@selenic.com>
parents: 3390
diff changeset
   843
                    return node
3453
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
   844
            except TypeError:
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
   845
                pass
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
   846
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
   847
    def lookup(self, id):
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
   848
        """locate a node based on:
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
   849
            - revision number or str(revision number)
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
   850
            - nodeid or subset of hex nodeid
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
   851
        """
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
   852
        n = self._match(id)
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
   853
        if n is not None:
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
   854
            return n
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
   855
        n = self._partialmatch(id)
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
   856
        if n:
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
   857
            return n
515
03f27b1381f9 Whitespace cleanups
mpm@selenic.com
parents: 484
diff changeset
   858
3930
01d98d68d697 Add revlog.LookupError exception, and use it instead of RevlogError.
Brendan Cully <brendan@kublai.com>
parents: 3928
diff changeset
   859
        raise LookupError(_("No match found"))
36
da28286bf6b7 Add smart node lookup by substring or by rev number
mpm@selenic.com
parents: 26
diff changeset
   860
2890
5df3e5cf16bc Move cmp bits from filelog to revlog
Matt Mackall <mpm@selenic.com>
parents: 2859
diff changeset
   861
    def cmp(self, node, text):
5df3e5cf16bc Move cmp bits from filelog to revlog
Matt Mackall <mpm@selenic.com>
parents: 2859
diff changeset
   862
        """compare text with a given file revision"""
5df3e5cf16bc Move cmp bits from filelog to revlog
Matt Mackall <mpm@selenic.com>
parents: 2859
diff changeset
   863
        p1, p2 = self.parents(node)
5df3e5cf16bc Move cmp bits from filelog to revlog
Matt Mackall <mpm@selenic.com>
parents: 2859
diff changeset
   864
        return hash(text, p1, p2) != node
5df3e5cf16bc Move cmp bits from filelog to revlog
Matt Mackall <mpm@selenic.com>
parents: 2859
diff changeset
   865
4988
14486eea8e7a revlog: speed up chunkcache
Matt Mackall <mpm@selenic.com>
parents: 4987
diff changeset
   866
    def chunk(self, rev, df=None):
2072
74d3f5336b66 Implement revlogng.
mason@suse.com
parents: 2002
diff changeset
   867
        def loadcache(df):
74d3f5336b66 Implement revlogng.
mason@suse.com
parents: 2002
diff changeset
   868
            if not df:
4988
14486eea8e7a revlog: speed up chunkcache
Matt Mackall <mpm@selenic.com>
parents: 4987
diff changeset
   869
                if self._inline:
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
   870
                    df = self.opener(self.indexfile)
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
   871
                else:
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
   872
                    df = self.opener(self.datafile)
1598
14d1f1868bf6 cleanup of revlog.group when repository is local
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1559
diff changeset
   873
            df.seek(start)
4983
4dbcfc6e359e revlog: pull chunkcache back into revlog
Matt Mackall <mpm@selenic.com>
parents: 4982
diff changeset
   874
            self._chunkcache = (start, df.read(cache_length))
1598
14d1f1868bf6 cleanup of revlog.group when repository is local
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1559
diff changeset
   875
5006
c2febf5420e9 revlog: minor chunk speed-up
Matt Mackall <mpm@selenic.com>
parents: 5005
diff changeset
   876
        start, length = self.start(rev), self.length(rev)
c2febf5420e9 revlog: minor chunk speed-up
Matt Mackall <mpm@selenic.com>
parents: 5005
diff changeset
   877
        if self._inline:
c2febf5420e9 revlog: minor chunk speed-up
Matt Mackall <mpm@selenic.com>
parents: 5005
diff changeset
   878
            start += (rev + 1) * self._io.size
c2febf5420e9 revlog: minor chunk speed-up
Matt Mackall <mpm@selenic.com>
parents: 5005
diff changeset
   879
        end = start + length
1598
14d1f1868bf6 cleanup of revlog.group when repository is local
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1559
diff changeset
   880
5006
c2febf5420e9 revlog: minor chunk speed-up
Matt Mackall <mpm@selenic.com>
parents: 5005
diff changeset
   881
        offset = 0
c2febf5420e9 revlog: minor chunk speed-up
Matt Mackall <mpm@selenic.com>
parents: 5005
diff changeset
   882
        if not self._chunkcache:
c2febf5420e9 revlog: minor chunk speed-up
Matt Mackall <mpm@selenic.com>
parents: 5005
diff changeset
   883
            cache_length = max(65536, length)
c2febf5420e9 revlog: minor chunk speed-up
Matt Mackall <mpm@selenic.com>
parents: 5005
diff changeset
   884
            loadcache(df)
1598
14d1f1868bf6 cleanup of revlog.group when repository is local
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1559
diff changeset
   885
        else:
5006
c2febf5420e9 revlog: minor chunk speed-up
Matt Mackall <mpm@selenic.com>
parents: 5005
diff changeset
   886
            cache_start = self._chunkcache[0]
c2febf5420e9 revlog: minor chunk speed-up
Matt Mackall <mpm@selenic.com>
parents: 5005
diff changeset
   887
            cache_length = len(self._chunkcache[1])
c2febf5420e9 revlog: minor chunk speed-up
Matt Mackall <mpm@selenic.com>
parents: 5005
diff changeset
   888
            cache_end = cache_start + cache_length
c2febf5420e9 revlog: minor chunk speed-up
Matt Mackall <mpm@selenic.com>
parents: 5005
diff changeset
   889
            if start >= cache_start and end <= cache_end:
c2febf5420e9 revlog: minor chunk speed-up
Matt Mackall <mpm@selenic.com>
parents: 5005
diff changeset
   890
                # it is cached
c2febf5420e9 revlog: minor chunk speed-up
Matt Mackall <mpm@selenic.com>
parents: 5005
diff changeset
   891
                offset = start - cache_start
c2febf5420e9 revlog: minor chunk speed-up
Matt Mackall <mpm@selenic.com>
parents: 5005
diff changeset
   892
            else:
c2febf5420e9 revlog: minor chunk speed-up
Matt Mackall <mpm@selenic.com>
parents: 5005
diff changeset
   893
                cache_length = max(65536, length)
c2febf5420e9 revlog: minor chunk speed-up
Matt Mackall <mpm@selenic.com>
parents: 5005
diff changeset
   894
                loadcache(df)
1598
14d1f1868bf6 cleanup of revlog.group when repository is local
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1559
diff changeset
   895
4988
14486eea8e7a revlog: speed up chunkcache
Matt Mackall <mpm@selenic.com>
parents: 4987
diff changeset
   896
        # avoid copying large chunks
14486eea8e7a revlog: speed up chunkcache
Matt Mackall <mpm@selenic.com>
parents: 4987
diff changeset
   897
        c = self._chunkcache[1]
5006
c2febf5420e9 revlog: minor chunk speed-up
Matt Mackall <mpm@selenic.com>
parents: 5005
diff changeset
   898
        if cache_length != length:
4988
14486eea8e7a revlog: speed up chunkcache
Matt Mackall <mpm@selenic.com>
parents: 4987
diff changeset
   899
            c = c[offset:offset + length]
14486eea8e7a revlog: speed up chunkcache
Matt Mackall <mpm@selenic.com>
parents: 4987
diff changeset
   900
14486eea8e7a revlog: speed up chunkcache
Matt Mackall <mpm@selenic.com>
parents: 4987
diff changeset
   901
        return decompress(c)
1598
14d1f1868bf6 cleanup of revlog.group when repository is local
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1559
diff changeset
   902
119
c7a66f9752a4 Add code to retrieve or construct a revlog delta
mpm@selenic.com
parents: 117
diff changeset
   903
    def delta(self, node):
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   904
        """return or calculate a delta between a node and its predecessor"""
119
c7a66f9752a4 Add code to retrieve or construct a revlog delta
mpm@selenic.com
parents: 117
diff changeset
   905
        r = self.rev(node)
1941
7518823709a2 revlog.py: factorization and fixes for rev < 0 (nullid)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1853
diff changeset
   906
        return self.revdiff(r - 1, r)
7518823709a2 revlog.py: factorization and fixes for rev < 0 (nullid)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1853
diff changeset
   907
7518823709a2 revlog.py: factorization and fixes for rev < 0 (nullid)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1853
diff changeset
   908
    def revdiff(self, rev1, rev2):
7518823709a2 revlog.py: factorization and fixes for rev < 0 (nullid)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1853
diff changeset
   909
        """return or calculate a delta between two revisions"""
5005
72082bfced9a revlog: minor revdiff reorganization
Matt Mackall <mpm@selenic.com>
parents: 5004
diff changeset
   910
        if rev1 + 1 == rev2 and self.base(rev1) == self.base(rev2):
1941
7518823709a2 revlog.py: factorization and fixes for rev < 0 (nullid)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1853
diff changeset
   911
            return self.chunk(rev2)
5005
72082bfced9a revlog: minor revdiff reorganization
Matt Mackall <mpm@selenic.com>
parents: 5004
diff changeset
   912
72082bfced9a revlog: minor revdiff reorganization
Matt Mackall <mpm@selenic.com>
parents: 5004
diff changeset
   913
        return mdiff.textdiff(self.revision(self.node(rev1)),
72082bfced9a revlog: minor revdiff reorganization
Matt Mackall <mpm@selenic.com>
parents: 5004
diff changeset
   914
                              self.revision(self.node(rev2)))
119
c7a66f9752a4 Add code to retrieve or construct a revlog delta
mpm@selenic.com
parents: 117
diff changeset
   915
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
   916
    def revision(self, node):
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   917
        """return an uncompressed revision of a given"""
4980
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
   918
        if node == nullid:
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
   919
            return ""
4984
b4066fcbd6ba revlog: mark cache private
Matt Mackall <mpm@selenic.com>
parents: 4983
diff changeset
   920
        if self._cache and self._cache[0] == node:
5450
c728424d44c6 revlog: fix caching of buffer objects
Matt Mackall <mpm@selenic.com>
parents: 5448
diff changeset
   921
            return str(self._cache[2])
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
   922
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   923
        # look up what we need to read
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
   924
        text = None
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
   925
        rev = self.rev(node)
1598
14d1f1868bf6 cleanup of revlog.group when repository is local
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1559
diff changeset
   926
        base = self.base(rev)
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
   927
5004
825516d16b25 revlog: move flag checking out of the offset fastpath
Matt Mackall <mpm@selenic.com>
parents: 4996
diff changeset
   928
        # check rev flags
825516d16b25 revlog: move flag checking out of the offset fastpath
Matt Mackall <mpm@selenic.com>
parents: 4996
diff changeset
   929
        if self.index[rev][0] & 0xFFFF:
5312
fb070713ff36 revlog: more robust for damaged indexes
Matt Mackall <mpm@selenic.com>
parents: 5007
diff changeset
   930
            raise RevlogError(_('incompatible revision flag %x') %
fb070713ff36 revlog: more robust for damaged indexes
Matt Mackall <mpm@selenic.com>
parents: 5007
diff changeset
   931
                              (self.index[rev][0] & 0xFFFF))
5004
825516d16b25 revlog: move flag checking out of the offset fastpath
Matt Mackall <mpm@selenic.com>
parents: 4996
diff changeset
   932
4982
9672e3c42b0c revlog: change _inline from a function to a variable
Matt Mackall <mpm@selenic.com>
parents: 4981
diff changeset
   933
        if self._inline:
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
   934
            # we probably have the whole chunk cached
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
   935
            df = None
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
   936
        else:
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
   937
            df = self.opener(self.datafile)
2072
74d3f5336b66 Implement revlogng.
mason@suse.com
parents: 2002
diff changeset
   938
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   939
        # do we have useful data cached?
4984
b4066fcbd6ba revlog: mark cache private
Matt Mackall <mpm@selenic.com>
parents: 4983
diff changeset
   940
        if self._cache and self._cache[1] >= base and self._cache[1] < rev:
b4066fcbd6ba revlog: mark cache private
Matt Mackall <mpm@selenic.com>
parents: 4983
diff changeset
   941
            base = self._cache[1]
5450
c728424d44c6 revlog: fix caching of buffer objects
Matt Mackall <mpm@selenic.com>
parents: 5448
diff changeset
   942
            text = str(self._cache[2])
4920
ee983d0dbea8 revlog: privatize some methods
Matt Mackall <mpm@selenic.com>
parents: 4919
diff changeset
   943
            self._loadindex(base, rev + 1)
1598
14d1f1868bf6 cleanup of revlog.group when repository is local
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1559
diff changeset
   944
        else:
4920
ee983d0dbea8 revlog: privatize some methods
Matt Mackall <mpm@selenic.com>
parents: 4919
diff changeset
   945
            self._loadindex(base, rev + 1)
2072
74d3f5336b66 Implement revlogng.
mason@suse.com
parents: 2002
diff changeset
   946
            text = self.chunk(base, df=df)
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
   947
4989
1aaed3d69772 revlog: eliminate diff and patches functions
Matt Mackall <mpm@selenic.com>
parents: 4988
diff changeset
   948
        bins = [self.chunk(r, df) for r in xrange(base + 1, rev + 1)]
1aaed3d69772 revlog: eliminate diff and patches functions
Matt Mackall <mpm@selenic.com>
parents: 4988
diff changeset
   949
        text = mdiff.patches(text, bins)
1598
14d1f1868bf6 cleanup of revlog.group when repository is local
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1559
diff changeset
   950
        p1, p2 = self.parents(node)
26
9cf83bf9ad38 Simplify integrity checking
mpm@selenic.com
parents: 14
diff changeset
   951
        if node != hash(text, p1, p2):
1402
9d2c2e6b32b5 i18n part2: use '_' for all strings who are part of the user interface
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1400
diff changeset
   952
            raise RevlogError(_("integrity check failed on %s:%d")
3680
69cf255a55a1 Indentation cleanups for 2956948b81f3.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 3679
diff changeset
   953
                              % (self.datafile, rev))
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
   954
4984
b4066fcbd6ba revlog: mark cache private
Matt Mackall <mpm@selenic.com>
parents: 4983
diff changeset
   955
        self._cache = (node, rev, text)
515
03f27b1381f9 Whitespace cleanups
mpm@selenic.com
parents: 484
diff changeset
   956
        return text
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
   957
2075
343aeefb553b Make the appendfile class inline-data index friendly
mason@suse.com
parents: 2073
diff changeset
   958
    def checkinlinesize(self, tr, fp=None):
4982
9672e3c42b0c revlog: change _inline from a function to a variable
Matt Mackall <mpm@selenic.com>
parents: 4981
diff changeset
   959
        if not self._inline:
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
   960
            return
2075
343aeefb553b Make the appendfile class inline-data index friendly
mason@suse.com
parents: 2073
diff changeset
   961
        if not fp:
343aeefb553b Make the appendfile class inline-data index friendly
mason@suse.com
parents: 2073
diff changeset
   962
            fp = self.opener(self.indexfile, 'r')
2082
856f0ba200bc Additional appendfile fixes for interleaved data/index files
mason@suse.com
parents: 2081
diff changeset
   963
            fp.seek(0, 2)
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
   964
        size = fp.tell()
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
   965
        if size < 131072:
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
   966
            return
2084
Chris Mason <mason@suse.com>
parents: 2082
diff changeset
   967
        trinfo = tr.find(self.indexfile)
Chris Mason <mason@suse.com>
parents: 2082
diff changeset
   968
        if trinfo == None:
3680
69cf255a55a1 Indentation cleanups for 2956948b81f3.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 3679
diff changeset
   969
            raise RevlogError(_("%s not found in the transaction")
69cf255a55a1 Indentation cleanups for 2956948b81f3.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 3679
diff changeset
   970
                              % self.indexfile)
2084
Chris Mason <mason@suse.com>
parents: 2082
diff changeset
   971
Chris Mason <mason@suse.com>
parents: 2082
diff changeset
   972
        trindex = trinfo[2]
Chris Mason <mason@suse.com>
parents: 2082
diff changeset
   973
        dataoff = self.start(trindex)
Chris Mason <mason@suse.com>
parents: 2082
diff changeset
   974
Chris Mason <mason@suse.com>
parents: 2082
diff changeset
   975
        tr.add(self.datafile, dataoff)
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
   976
        df = self.opener(self.datafile, 'w')
4977
6cb30bc4ca32 revlog: parse revlogv0 indexes into v1 internally
Matt Mackall <mpm@selenic.com>
parents: 4976
diff changeset
   977
        calc = self._io.size
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
   978
        for r in xrange(self.count()):
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
   979
            start = self.start(r) + (r + 1) * calc
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
   980
            length = self.length(r)
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
   981
            fp.seek(start)
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
   982
            d = fp.read(length)
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
   983
            df.write(d)
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
   984
        fp.close()
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
   985
        df.close()
2076
d007df6daf8e Create an atomic opener that does not automatically rename on close
mason@suse.com
parents: 2075
diff changeset
   986
        fp = self.opener(self.indexfile, 'w', atomictemp=True)
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
   987
        self.version &= ~(REVLOGNGINLINEDATA)
4982
9672e3c42b0c revlog: change _inline from a function to a variable
Matt Mackall <mpm@selenic.com>
parents: 4981
diff changeset
   988
        self._inline = False
4986
58cc017ec7e0 revlog: abstract out index entry packing
Matt Mackall <mpm@selenic.com>
parents: 4985
diff changeset
   989
        for i in xrange(self.count()):
5338
f87685355c9c revlog: fix revlogio.packentry corner case
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5325
diff changeset
   990
            e = self._io.packentry(self.index[i], self.node, self.version, i)
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
   991
            fp.write(e)
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
   992
2076
d007df6daf8e Create an atomic opener that does not automatically rename on close
mason@suse.com
parents: 2075
diff changeset
   993
        # if we don't call rename, the temp file will never replace the
d007df6daf8e Create an atomic opener that does not automatically rename on close
mason@suse.com
parents: 2075
diff changeset
   994
        # real index
d007df6daf8e Create an atomic opener that does not automatically rename on close
mason@suse.com
parents: 2075
diff changeset
   995
        fp.rename()
2084
Chris Mason <mason@suse.com>
parents: 2082
diff changeset
   996
Chris Mason <mason@suse.com>
parents: 2082
diff changeset
   997
        tr.replace(self.indexfile, trindex * calc)
4983
4dbcfc6e359e revlog: pull chunkcache back into revlog
Matt Mackall <mpm@selenic.com>
parents: 4982
diff changeset
   998
        self._chunkcache = None
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
   999
4981
e7131935fbb3 revlog: simplify addrevision
Matt Mackall <mpm@selenic.com>
parents: 4980
diff changeset
  1000
    def addrevision(self, text, transaction, link, p1, p2, d=None):
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
  1001
        """add a revision to the log
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
  1002
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
  1003
        text - the revision data to add
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
  1004
        transaction - the transaction object used for rollback
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
  1005
        link - the linkrev data to add
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
  1006
        p1, p2 - the parent nodeids of the revision
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
  1007
        d - an optional precomputed delta
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
  1008
        """
4981
e7131935fbb3 revlog: simplify addrevision
Matt Mackall <mpm@selenic.com>
parents: 4980
diff changeset
  1009
        dfh = None
4982
9672e3c42b0c revlog: change _inline from a function to a variable
Matt Mackall <mpm@selenic.com>
parents: 4981
diff changeset
  1010
        if not self._inline:
3390
a74addddd092 make revlog.addgroup pass its file handles to addrevision
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 3360
diff changeset
  1011
            dfh = self.opener(self.datafile, "a")
a74addddd092 make revlog.addgroup pass its file handles to addrevision
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 3360
diff changeset
  1012
        ifh = self.opener(self.indexfile, "a+")
a74addddd092 make revlog.addgroup pass its file handles to addrevision
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 3360
diff changeset
  1013
        return self._addrevision(text, transaction, link, p1, p2, d, ifh, dfh)
a74addddd092 make revlog.addgroup pass its file handles to addrevision
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 3360
diff changeset
  1014
a74addddd092 make revlog.addgroup pass its file handles to addrevision
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 3360
diff changeset
  1015
    def _addrevision(self, text, transaction, link, p1, p2, d, ifh, dfh):
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
  1016
        node = hash(text, p1, p2)
301
5add718d92db revlog: allow duplicates
mpm@selenic.com
parents: 243
diff changeset
  1017
        if node in self.nodemap:
5add718d92db revlog: allow duplicates
mpm@selenic.com
parents: 243
diff changeset
  1018
            return node
5add718d92db revlog: allow duplicates
mpm@selenic.com
parents: 243
diff changeset
  1019
4981
e7131935fbb3 revlog: simplify addrevision
Matt Mackall <mpm@selenic.com>
parents: 4980
diff changeset
  1020
        curr = self.count()
e7131935fbb3 revlog: simplify addrevision
Matt Mackall <mpm@selenic.com>
parents: 4980
diff changeset
  1021
        prev = curr - 1
e7131935fbb3 revlog: simplify addrevision
Matt Mackall <mpm@selenic.com>
parents: 4980
diff changeset
  1022
        base = self.base(prev)
e7131935fbb3 revlog: simplify addrevision
Matt Mackall <mpm@selenic.com>
parents: 4980
diff changeset
  1023
        offset = self.end(prev)
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
  1024
4981
e7131935fbb3 revlog: simplify addrevision
Matt Mackall <mpm@selenic.com>
parents: 4980
diff changeset
  1025
        if curr:
644
6ebe118280bd Performance enhancements for manifest.add()
mason@suse.com
parents: 547
diff changeset
  1026
            if not d:
4981
e7131935fbb3 revlog: simplify addrevision
Matt Mackall <mpm@selenic.com>
parents: 4980
diff changeset
  1027
                ptext = self.revision(self.node(prev))
4989
1aaed3d69772 revlog: eliminate diff and patches functions
Matt Mackall <mpm@selenic.com>
parents: 4988
diff changeset
  1028
                d = mdiff.textdiff(ptext, text)
98
3dde7c87e36d Add paranoia to diff code
mpm@selenic.com
parents: 97
diff changeset
  1029
            data = compress(d)
1533
3d11f81c9145 Reduce string duplication in compression code
mason@suse.com
parents: 1509
diff changeset
  1030
            l = len(data[1]) + len(data[0])
4981
e7131935fbb3 revlog: simplify addrevision
Matt Mackall <mpm@selenic.com>
parents: 4980
diff changeset
  1031
            dist = l + offset - self.start(base)
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
  1032
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
  1033
        # full versions are inserted when the needed deltas
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
  1034
        # become comparable to the uncompressed text
4981
e7131935fbb3 revlog: simplify addrevision
Matt Mackall <mpm@selenic.com>
parents: 4980
diff changeset
  1035
        if not curr or dist > len(text) * 2:
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
  1036
            data = compress(text)
1533
3d11f81c9145 Reduce string duplication in compression code
mason@suse.com
parents: 1509
diff changeset
  1037
            l = len(data[1]) + len(data[0])
4981
e7131935fbb3 revlog: simplify addrevision
Matt Mackall <mpm@selenic.com>
parents: 4980
diff changeset
  1038
            base = curr
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
  1039
4977
6cb30bc4ca32 revlog: parse revlogv0 indexes into v1 internally
Matt Mackall <mpm@selenic.com>
parents: 4976
diff changeset
  1040
        e = (offset_type(offset, 0), l, len(text),
6cb30bc4ca32 revlog: parse revlogv0 indexes into v1 internally
Matt Mackall <mpm@selenic.com>
parents: 4976
diff changeset
  1041
             base, link, self.rev(p1), self.rev(p2), node)
4979
06abdaf78788 revlog: add a magic null revision to our index
Matt Mackall <mpm@selenic.com>
parents: 4978
diff changeset
  1042
        self.index.insert(-1, e)
4981
e7131935fbb3 revlog: simplify addrevision
Matt Mackall <mpm@selenic.com>
parents: 4980
diff changeset
  1043
        self.nodemap[node] = curr
4977
6cb30bc4ca32 revlog: parse revlogv0 indexes into v1 internally
Matt Mackall <mpm@selenic.com>
parents: 4976
diff changeset
  1044
5338
f87685355c9c revlog: fix revlogio.packentry corner case
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5325
diff changeset
  1045
        entry = self._io.packentry(e, self.node, self.version, curr)
4982
9672e3c42b0c revlog: change _inline from a function to a variable
Matt Mackall <mpm@selenic.com>
parents: 4981
diff changeset
  1046
        if not self._inline:
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
  1047
            transaction.add(self.datafile, offset)
4981
e7131935fbb3 revlog: simplify addrevision
Matt Mackall <mpm@selenic.com>
parents: 4980
diff changeset
  1048
            transaction.add(self.indexfile, curr * len(entry))
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
  1049
            if data[0]:
3390
a74addddd092 make revlog.addgroup pass its file handles to addrevision
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 3360
diff changeset
  1050
                dfh.write(data[0])
a74addddd092 make revlog.addgroup pass its file handles to addrevision
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 3360
diff changeset
  1051
            dfh.write(data[1])
a74addddd092 make revlog.addgroup pass its file handles to addrevision
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 3360
diff changeset
  1052
            dfh.flush()
4981
e7131935fbb3 revlog: simplify addrevision
Matt Mackall <mpm@selenic.com>
parents: 4980
diff changeset
  1053
            ifh.write(entry)
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
  1054
        else:
4996
a0d37976cd5b revlog: avoid some unnecessary seek/tell syscalls
Matt Mackall <mpm@selenic.com>
parents: 4994
diff changeset
  1055
            offset += curr * self._io.size
5324
8409a2e3a78d revlog: fix inlined revision transaction extra data (issue 749)
Patrick Mezard <pmezard@gmail.com>
parents: 5007
diff changeset
  1056
            transaction.add(self.indexfile, offset, curr)
4981
e7131935fbb3 revlog: simplify addrevision
Matt Mackall <mpm@selenic.com>
parents: 4980
diff changeset
  1057
            ifh.write(entry)
3390
a74addddd092 make revlog.addgroup pass its file handles to addrevision
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 3360
diff changeset
  1058
            ifh.write(data[0])
a74addddd092 make revlog.addgroup pass its file handles to addrevision
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 3360
diff changeset
  1059
            ifh.write(data[1])
a74addddd092 make revlog.addgroup pass its file handles to addrevision
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 3360
diff changeset
  1060
            self.checkinlinesize(transaction, ifh)
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
  1061
4984
b4066fcbd6ba revlog: mark cache private
Matt Mackall <mpm@selenic.com>
parents: 4983
diff changeset
  1062
        self._cache = (node, curr, text)
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
  1063
        return node
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
  1064
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
  1065
    def ancestor(self, a, b):
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
  1066
        """calculate the least common ancestor of nodes a and b"""
2081
416d8b2a75b8 Speedup revlog.ancestors for the linear case
Chris Mason <mason@suse.com>
parents: 2080
diff changeset
  1067
3139
1fd1cdcc4200 Switch revlog.ancestor to use revisions rather than nodeids
Matt Mackall <mpm@selenic.com>
parents: 3135
diff changeset
  1068
        def parents(rev):
3578
3b4e00cba57a Define and use nullrev (revision of nullid) instead of -1.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 3508
diff changeset
  1069
            return [p for p in self.parentrevs(rev) if p != nullrev]
515
03f27b1381f9 Whitespace cleanups
mpm@selenic.com
parents: 484
diff changeset
  1070
3139
1fd1cdcc4200 Switch revlog.ancestor to use revisions rather than nodeids
Matt Mackall <mpm@selenic.com>
parents: 3135
diff changeset
  1071
        c = ancestor.ancestor(self.rev(a), self.rev(b), parents)
1fd1cdcc4200 Switch revlog.ancestor to use revisions rather than nodeids
Matt Mackall <mpm@selenic.com>
parents: 3135
diff changeset
  1072
        if c is None:
1fd1cdcc4200 Switch revlog.ancestor to use revisions rather than nodeids
Matt Mackall <mpm@selenic.com>
parents: 3135
diff changeset
  1073
            return nullid
1fd1cdcc4200 Switch revlog.ancestor to use revisions rather than nodeids
Matt Mackall <mpm@selenic.com>
parents: 3135
diff changeset
  1074
1fd1cdcc4200 Switch revlog.ancestor to use revisions rather than nodeids
Matt Mackall <mpm@selenic.com>
parents: 3135
diff changeset
  1075
        return self.node(c)
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
  1076
1598
14d1f1868bf6 cleanup of revlog.group when repository is local
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1559
diff changeset
  1077
    def group(self, nodelist, lookup, infocollect=None):
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
  1078
        """calculate a delta group
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1079
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
  1080
        Given a list of changeset revs, return a set of deltas and
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
  1081
        metadata corresponding to nodes. the first delta is
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
  1082
        parent(nodes[0]) -> nodes[0] the receiver is guaranteed to
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
  1083
        have this parent as it has all history before these
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
  1084
        changesets. parent is parent[0]
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
  1085
        """
1458
1033892bbb87 This changes the revlog.group and re-implements the localrepo.changeroup
Eric Hopper <hopper@omnifarious.org>
parents: 1457
diff changeset
  1086
        revs = [self.rev(n) for n in nodelist]
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1087
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1088
        # if we don't have any revisions touched by these changesets, bail
192
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
  1089
        if not revs:
1981
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 1941
diff changeset
  1090
            yield changegroup.closechunk()
192
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
  1091
            return
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1092
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1093
        # add the parent of the first rev
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1094
        p = self.parents(self.node(revs[0]))[0]
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1095
        revs.insert(0, self.rev(p))
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1096
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1097
        # build deltas
71
47c9a869adee Add mdiff.patches to speed up applying thousands of patches to the manifest
mpm@selenic.com
parents: 67
diff changeset
  1098
        for d in xrange(0, len(revs) - 1):
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1099
            a, b = revs[d], revs[d + 1]
1598
14d1f1868bf6 cleanup of revlog.group when repository is local
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1559
diff changeset
  1100
            nb = self.node(b)
192
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
  1101
1458
1033892bbb87 This changes the revlog.group and re-implements the localrepo.changeroup
Eric Hopper <hopper@omnifarious.org>
parents: 1457
diff changeset
  1102
            if infocollect is not None:
1598
14d1f1868bf6 cleanup of revlog.group when repository is local
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1559
diff changeset
  1103
                infocollect(nb)
1458
1033892bbb87 This changes the revlog.group and re-implements the localrepo.changeroup
Eric Hopper <hopper@omnifarious.org>
parents: 1457
diff changeset
  1104
1598
14d1f1868bf6 cleanup of revlog.group when repository is local
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1559
diff changeset
  1105
            p = self.parents(nb)
14d1f1868bf6 cleanup of revlog.group when repository is local
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1559
diff changeset
  1106
            meta = nb + p[0] + p[1] + lookup(nb)
5367
7530334bf301 revlog: generate trivial deltas against null revision
Matt Mackall <mpm@selenic.com>
parents: 5338
diff changeset
  1107
            if a == -1:
7530334bf301 revlog: generate trivial deltas against null revision
Matt Mackall <mpm@selenic.com>
parents: 5338
diff changeset
  1108
                d = self.revision(nb)
7530334bf301 revlog: generate trivial deltas against null revision
Matt Mackall <mpm@selenic.com>
parents: 5338
diff changeset
  1109
                meta += mdiff.trivialdiffheader(len(d))
7530334bf301 revlog: generate trivial deltas against null revision
Matt Mackall <mpm@selenic.com>
parents: 5338
diff changeset
  1110
            else:
7530334bf301 revlog: generate trivial deltas against null revision
Matt Mackall <mpm@selenic.com>
parents: 5338
diff changeset
  1111
                d = self.revdiff(a, b)
5368
61462e7d62ed changegroup: avoid large copies
Matt Mackall <mpm@selenic.com>
parents: 5367
diff changeset
  1112
            yield changegroup.chunkheader(len(meta) + len(d))
61462e7d62ed changegroup: avoid large copies
Matt Mackall <mpm@selenic.com>
parents: 5367
diff changeset
  1113
            yield meta
5448
e038738714fd revlog: avoid large yields in group()
Matt Mackall <mpm@selenic.com>
parents: 5445
diff changeset
  1114
            if len(d) > 2**20:
e038738714fd revlog: avoid large yields in group()
Matt Mackall <mpm@selenic.com>
parents: 5445
diff changeset
  1115
                pos = 0
e038738714fd revlog: avoid large yields in group()
Matt Mackall <mpm@selenic.com>
parents: 5445
diff changeset
  1116
                while pos < len(d):
e038738714fd revlog: avoid large yields in group()
Matt Mackall <mpm@selenic.com>
parents: 5445
diff changeset
  1117
                    pos2 = pos + 2 ** 18
e038738714fd revlog: avoid large yields in group()
Matt Mackall <mpm@selenic.com>
parents: 5445
diff changeset
  1118
                    yield d[pos:pos2]
e038738714fd revlog: avoid large yields in group()
Matt Mackall <mpm@selenic.com>
parents: 5445
diff changeset
  1119
                    pos = pos2
e038738714fd revlog: avoid large yields in group()
Matt Mackall <mpm@selenic.com>
parents: 5445
diff changeset
  1120
            else:
e038738714fd revlog: avoid large yields in group()
Matt Mackall <mpm@selenic.com>
parents: 5445
diff changeset
  1121
                yield d
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1122
1981
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 1941
diff changeset
  1123
        yield changegroup.closechunk()
192
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
  1124
1062
6d5a62a549fa pep-0008 cleanup
benoit.boissinot@ens-lyon.fr
parents: 902
diff changeset
  1125
    def addgroup(self, revs, linkmapper, transaction, unique=0):
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
  1126
        """
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
  1127
        add a delta group
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1128
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
  1129
        given a set of deltas, add them to the revision log. the
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
  1130
        first delta is against its parent, which should be in our
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
  1131
        log, the rest are against the previous delta.
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
  1132
        """
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
  1133
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
  1134
        #track the base of the current delta log
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1135
        r = self.count()
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1136
        t = r - 1
2002
4aab906517c6 Calling revlog.addgroup with an empty changegroup now raises RevlogError.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 1981
diff changeset
  1137
        node = None
515
03f27b1381f9 Whitespace cleanups
mpm@selenic.com
parents: 484
diff changeset
  1138
3578
3b4e00cba57a Define and use nullrev (revision of nullid) instead of -1.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 3508
diff changeset
  1139
        base = prev = nullrev
2078
441ea218414e Fill in the uncompressed size during revlog.addgroup
mason@suse.com
parents: 2077
diff changeset
  1140
        start = end = textlen = 0
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1141
        if r:
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1142
            end = self.end(t)
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1143
2072
74d3f5336b66 Implement revlogng.
mason@suse.com
parents: 2002
diff changeset
  1144
        ifh = self.opener(self.indexfile, "a+")
4996
a0d37976cd5b revlog: avoid some unnecessary seek/tell syscalls
Matt Mackall <mpm@selenic.com>
parents: 4994
diff changeset
  1145
        isize = r * self._io.size
4982
9672e3c42b0c revlog: change _inline from a function to a variable
Matt Mackall <mpm@selenic.com>
parents: 4981
diff changeset
  1146
        if self._inline:
4996
a0d37976cd5b revlog: avoid some unnecessary seek/tell syscalls
Matt Mackall <mpm@selenic.com>
parents: 4994
diff changeset
  1147
            transaction.add(self.indexfile, end + isize, r)
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
  1148
            dfh = None
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
  1149
        else:
4996
a0d37976cd5b revlog: avoid some unnecessary seek/tell syscalls
Matt Mackall <mpm@selenic.com>
parents: 4994
diff changeset
  1150
            transaction.add(self.indexfile, isize, r)
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
  1151
            transaction.add(self.datafile, end)
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
  1152
            dfh = self.opener(self.datafile, "a")
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1153
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1154
        # loop through our set of deltas
192
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
  1155
        chain = None
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
  1156
        for chunk in revs:
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
  1157
            node, p1, p2, cs = struct.unpack("20s20s20s20s", chunk[:80])
94
7daef883134f Refactor merge code
mpm@selenic.com
parents: 86
diff changeset
  1158
            link = linkmapper(cs)
77
bed15e766511 Fix bug in lazymap code
mpm@selenic.com
parents: 76
diff changeset
  1159
            if node in self.nodemap:
224
ccbcc4d76f81 fix bad assumption about uniqueness of file versions
mpm@selenic.com
parents: 221
diff changeset
  1160
                # this can happen if two branches make the same change
1218
cde6818e082a Add preliminary support for the bundle and unbundle commands
mpm@selenic.com
parents: 1214
diff changeset
  1161
                # if unique:
1402
9d2c2e6b32b5 i18n part2: use '_' for all strings who are part of the user interface
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1400
diff changeset
  1162
                #    raise RevlogError(_("already have %s") % hex(node[:4]))
653
94cdd02792b5 Fix corruption resulting from skipping parts of a revision group
Matt Mackall <mpm@selenic.com>
parents: 651
diff changeset
  1163
                chain = node
224
ccbcc4d76f81 fix bad assumption about uniqueness of file versions
mpm@selenic.com
parents: 221
diff changeset
  1164
                continue
5445
64cf1c853674 revlog: reduce memory usage in addgroup
Matt Mackall <mpm@selenic.com>
parents: 5368
diff changeset
  1165
            delta = buffer(chunk, 80)
64cf1c853674 revlog: reduce memory usage in addgroup
Matt Mackall <mpm@selenic.com>
parents: 5368
diff changeset
  1166
            del chunk
192
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
  1167
1509
46a07392cf28 Add safety check for addgroup
Matt Mackall <mpm@selenic.com>
parents: 1494
diff changeset
  1168
            for p in (p1, p2):
46a07392cf28 Add safety check for addgroup
Matt Mackall <mpm@selenic.com>
parents: 1494
diff changeset
  1169
                if not p in self.nodemap:
3930
01d98d68d697 Add revlog.LookupError exception, and use it instead of RevlogError.
Brendan Cully <brendan@kublai.com>
parents: 3928
diff changeset
  1170
                    raise LookupError(_("unknown parent %s") % short(p))
1509
46a07392cf28 Add safety check for addgroup
Matt Mackall <mpm@selenic.com>
parents: 1494
diff changeset
  1171
192
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
  1172
            if not chain:
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
  1173
                # retrieve the parent revision of the delta chain
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
  1174
                chain = p1
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
  1175
                if not chain in self.nodemap:
3930
01d98d68d697 Add revlog.LookupError exception, and use it instead of RevlogError.
Brendan Cully <brendan@kublai.com>
parents: 3928
diff changeset
  1176
                    raise LookupError(_("unknown base %s") % short(chain[:4]))
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1177
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1178
            # full versions are inserted when the needed deltas become
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1179
            # comparable to the uncompressed text or when the previous
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1180
            # version is not the one we have a delta against. We use
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1181
            # the size of the previous full rev as a proxy for the
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1182
            # current size.
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1183
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1184
            if chain == prev:
5445
64cf1c853674 revlog: reduce memory usage in addgroup
Matt Mackall <mpm@selenic.com>
parents: 5368
diff changeset
  1185
                cdelta = compress(delta)
64cf1c853674 revlog: reduce memory usage in addgroup
Matt Mackall <mpm@selenic.com>
parents: 5368
diff changeset
  1186
                cdeltalen = len(cdelta[0]) + len(cdelta[1])
2078
441ea218414e Fill in the uncompressed size during revlog.addgroup
mason@suse.com
parents: 2077
diff changeset
  1187
                textlen = mdiff.patchedsize(textlen, delta)
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1188
5445
64cf1c853674 revlog: reduce memory usage in addgroup
Matt Mackall <mpm@selenic.com>
parents: 5368
diff changeset
  1189
            if chain != prev or (end - start + cdeltalen) > textlen * 2:
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1190
                # flush our writes here so we can read it in revision
2072
74d3f5336b66 Implement revlogng.
mason@suse.com
parents: 2002
diff changeset
  1191
                if dfh:
74d3f5336b66 Implement revlogng.
mason@suse.com
parents: 2002
diff changeset
  1192
                    dfh.flush()
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1193
                ifh.flush()
65
d40cc5aacc31 Fix up a bunch of bugs in the new merge code
mpm@selenic.com
parents: 64
diff changeset
  1194
                text = self.revision(chain)
5445
64cf1c853674 revlog: reduce memory usage in addgroup
Matt Mackall <mpm@selenic.com>
parents: 5368
diff changeset
  1195
                if len(text) == 0:
64cf1c853674 revlog: reduce memory usage in addgroup
Matt Mackall <mpm@selenic.com>
parents: 5368
diff changeset
  1196
                    # skip over trivial delta header
64cf1c853674 revlog: reduce memory usage in addgroup
Matt Mackall <mpm@selenic.com>
parents: 5368
diff changeset
  1197
                    text = buffer(delta, 12)
64cf1c853674 revlog: reduce memory usage in addgroup
Matt Mackall <mpm@selenic.com>
parents: 5368
diff changeset
  1198
                else:
64cf1c853674 revlog: reduce memory usage in addgroup
Matt Mackall <mpm@selenic.com>
parents: 5368
diff changeset
  1199
                    text = mdiff.patches(text, [delta])
64cf1c853674 revlog: reduce memory usage in addgroup
Matt Mackall <mpm@selenic.com>
parents: 5368
diff changeset
  1200
                del delta
3390
a74addddd092 make revlog.addgroup pass its file handles to addrevision
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 3360
diff changeset
  1201
                chk = self._addrevision(text, transaction, link, p1, p2, None,
a74addddd092 make revlog.addgroup pass its file handles to addrevision
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 3360
diff changeset
  1202
                                        ifh, dfh)
4982
9672e3c42b0c revlog: change _inline from a function to a variable
Matt Mackall <mpm@selenic.com>
parents: 4981
diff changeset
  1203
                if not dfh and not self._inline:
3390
a74addddd092 make revlog.addgroup pass its file handles to addrevision
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 3360
diff changeset
  1204
                    # addrevision switched from inline to conventional
a74addddd092 make revlog.addgroup pass its file handles to addrevision
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 3360
diff changeset
  1205
                    # reopen the index
a74addddd092 make revlog.addgroup pass its file handles to addrevision
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 3360
diff changeset
  1206
                    dfh = self.opener(self.datafile, "a")
a74addddd092 make revlog.addgroup pass its file handles to addrevision
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 3360
diff changeset
  1207
                    ifh = self.opener(self.indexfile, "a")
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1208
                if chk != node:
1402
9d2c2e6b32b5 i18n part2: use '_' for all strings who are part of the user interface
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1400
diff changeset
  1209
                    raise RevlogError(_("consistency error adding group"))
2078
441ea218414e Fill in the uncompressed size during revlog.addgroup
mason@suse.com
parents: 2077
diff changeset
  1210
                textlen = len(text)
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1211
            else:
5445
64cf1c853674 revlog: reduce memory usage in addgroup
Matt Mackall <mpm@selenic.com>
parents: 5368
diff changeset
  1212
                e = (offset_type(end, 0), cdeltalen, textlen, base,
4977
6cb30bc4ca32 revlog: parse revlogv0 indexes into v1 internally
Matt Mackall <mpm@selenic.com>
parents: 4976
diff changeset
  1213
                     link, self.rev(p1), self.rev(p2), node)
4979
06abdaf78788 revlog: add a magic null revision to our index
Matt Mackall <mpm@selenic.com>
parents: 4978
diff changeset
  1214
                self.index.insert(-1, e)
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1215
                self.nodemap[node] = r
5338
f87685355c9c revlog: fix revlogio.packentry corner case
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5325
diff changeset
  1216
                entry = self._io.packentry(e, self.node, self.version, r)
4982
9672e3c42b0c revlog: change _inline from a function to a variable
Matt Mackall <mpm@selenic.com>
parents: 4981
diff changeset
  1217
                if self._inline:
4986
58cc017ec7e0 revlog: abstract out index entry packing
Matt Mackall <mpm@selenic.com>
parents: 4985
diff changeset
  1218
                    ifh.write(entry)
5445
64cf1c853674 revlog: reduce memory usage in addgroup
Matt Mackall <mpm@selenic.com>
parents: 5368
diff changeset
  1219
                    ifh.write(cdelta[0])
64cf1c853674 revlog: reduce memory usage in addgroup
Matt Mackall <mpm@selenic.com>
parents: 5368
diff changeset
  1220
                    ifh.write(cdelta[1])
2075
343aeefb553b Make the appendfile class inline-data index friendly
mason@suse.com
parents: 2073
diff changeset
  1221
                    self.checkinlinesize(transaction, ifh)
4982
9672e3c42b0c revlog: change _inline from a function to a variable
Matt Mackall <mpm@selenic.com>
parents: 4981
diff changeset
  1222
                    if not self._inline:
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
  1223
                        dfh = self.opener(self.datafile, "a")
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
  1224
                        ifh = self.opener(self.indexfile, "a")
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
  1225
                else:
5445
64cf1c853674 revlog: reduce memory usage in addgroup
Matt Mackall <mpm@selenic.com>
parents: 5368
diff changeset
  1226
                    dfh.write(cdelta[0])
64cf1c853674 revlog: reduce memory usage in addgroup
Matt Mackall <mpm@selenic.com>
parents: 5368
diff changeset
  1227
                    dfh.write(cdelta[1])
4977
6cb30bc4ca32 revlog: parse revlogv0 indexes into v1 internally
Matt Mackall <mpm@selenic.com>
parents: 4976
diff changeset
  1228
                    ifh.write(entry)
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1229
65
d40cc5aacc31 Fix up a bunch of bugs in the new merge code
mpm@selenic.com
parents: 64
diff changeset
  1230
            t, r, chain, prev = r, r + 1, node, node
1749
d457fec76ab0 fix warnings from pychecker (unused variables and shadowing)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1711
diff changeset
  1231
            base = self.base(t)
d457fec76ab0 fix warnings from pychecker (unused variables and shadowing)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1711
diff changeset
  1232
            start = self.start(base)
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1233
            end = self.end(t)
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1234
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1235
        return node
1493
1a216cb4ee64 verify: add check for mismatch of index and data length
Matt Mackall <mpm@selenic.com>
parents: 1469
diff changeset
  1236
1535
7ae0ce7a3dc4 Add revlog.strip to truncate away revisions.
mason@suse.com
parents: 1533
diff changeset
  1237
    def strip(self, rev, minlink):
7ae0ce7a3dc4 Add revlog.strip to truncate away revisions.
mason@suse.com
parents: 1533
diff changeset
  1238
        if self.count() == 0 or rev >= self.count():
7ae0ce7a3dc4 Add revlog.strip to truncate away revisions.
mason@suse.com
parents: 1533
diff changeset
  1239
            return
7ae0ce7a3dc4 Add revlog.strip to truncate away revisions.
mason@suse.com
parents: 1533
diff changeset
  1240
2072
74d3f5336b66 Implement revlogng.
mason@suse.com
parents: 2002
diff changeset
  1241
        if isinstance(self.index, lazyindex):
4920
ee983d0dbea8 revlog: privatize some methods
Matt Mackall <mpm@selenic.com>
parents: 4919
diff changeset
  1242
            self._loadindexmap()
2072
74d3f5336b66 Implement revlogng.
mason@suse.com
parents: 2002
diff changeset
  1243
1535
7ae0ce7a3dc4 Add revlog.strip to truncate away revisions.
mason@suse.com
parents: 1533
diff changeset
  1244
        # When stripping away a revision, we need to make sure it
7ae0ce7a3dc4 Add revlog.strip to truncate away revisions.
mason@suse.com
parents: 1533
diff changeset
  1245
        # does not actually belong to an older changeset.
7ae0ce7a3dc4 Add revlog.strip to truncate away revisions.
mason@suse.com
parents: 1533
diff changeset
  1246
        # The minlink parameter defines the oldest revision
7ae0ce7a3dc4 Add revlog.strip to truncate away revisions.
mason@suse.com
parents: 1533
diff changeset
  1247
        # we're allowed to strip away.
4978
93d48a8fa496 revlog: change accesses to index entry elements to use positive offsets
Matt Mackall <mpm@selenic.com>
parents: 4977
diff changeset
  1248
        while minlink > self.index[rev][4]:
1535
7ae0ce7a3dc4 Add revlog.strip to truncate away revisions.
mason@suse.com
parents: 1533
diff changeset
  1249
            rev += 1
7ae0ce7a3dc4 Add revlog.strip to truncate away revisions.
mason@suse.com
parents: 1533
diff changeset
  1250
            if rev >= self.count():
7ae0ce7a3dc4 Add revlog.strip to truncate away revisions.
mason@suse.com
parents: 1533
diff changeset
  1251
                return
7ae0ce7a3dc4 Add revlog.strip to truncate away revisions.
mason@suse.com
parents: 1533
diff changeset
  1252
7ae0ce7a3dc4 Add revlog.strip to truncate away revisions.
mason@suse.com
parents: 1533
diff changeset
  1253
        # first truncate the files on disk
7ae0ce7a3dc4 Add revlog.strip to truncate away revisions.
mason@suse.com
parents: 1533
diff changeset
  1254
        end = self.start(rev)
4982
9672e3c42b0c revlog: change _inline from a function to a variable
Matt Mackall <mpm@selenic.com>
parents: 4981
diff changeset
  1255
        if not self._inline:
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
  1256
            df = self.opener(self.datafile, "a")
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
  1257
            df.truncate(end)
4977
6cb30bc4ca32 revlog: parse revlogv0 indexes into v1 internally
Matt Mackall <mpm@selenic.com>
parents: 4976
diff changeset
  1258
            end = rev * self._io.size
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
  1259
        else:
4977
6cb30bc4ca32 revlog: parse revlogv0 indexes into v1 internally
Matt Mackall <mpm@selenic.com>
parents: 4976
diff changeset
  1260
            end += rev * self._io.size
2072
74d3f5336b66 Implement revlogng.
mason@suse.com
parents: 2002
diff changeset
  1261
74d3f5336b66 Implement revlogng.
mason@suse.com
parents: 2002
diff changeset
  1262
        indexf = self.opener(self.indexfile, "a")
74d3f5336b66 Implement revlogng.
mason@suse.com
parents: 2002
diff changeset
  1263
        indexf.truncate(end)
1535
7ae0ce7a3dc4 Add revlog.strip to truncate away revisions.
mason@suse.com
parents: 1533
diff changeset
  1264
7ae0ce7a3dc4 Add revlog.strip to truncate away revisions.
mason@suse.com
parents: 1533
diff changeset
  1265
        # then reset internal state in memory to forget those revisions
4984
b4066fcbd6ba revlog: mark cache private
Matt Mackall <mpm@selenic.com>
parents: 4983
diff changeset
  1266
        self._cache = None
4983
4dbcfc6e359e revlog: pull chunkcache back into revlog
Matt Mackall <mpm@selenic.com>
parents: 4982
diff changeset
  1267
        self._chunkcache = None
2072
74d3f5336b66 Implement revlogng.
mason@suse.com
parents: 2002
diff changeset
  1268
        for x in xrange(rev, self.count()):
74d3f5336b66 Implement revlogng.
mason@suse.com
parents: 2002
diff changeset
  1269
            del self.nodemap[self.node(x)]
1535
7ae0ce7a3dc4 Add revlog.strip to truncate away revisions.
mason@suse.com
parents: 1533
diff changeset
  1270
4979
06abdaf78788 revlog: add a magic null revision to our index
Matt Mackall <mpm@selenic.com>
parents: 4978
diff changeset
  1271
        del self.index[rev:-1]
1535
7ae0ce7a3dc4 Add revlog.strip to truncate away revisions.
mason@suse.com
parents: 1533
diff changeset
  1272
1493
1a216cb4ee64 verify: add check for mismatch of index and data length
Matt Mackall <mpm@selenic.com>
parents: 1469
diff changeset
  1273
    def checksize(self):
1a216cb4ee64 verify: add check for mismatch of index and data length
Matt Mackall <mpm@selenic.com>
parents: 1469
diff changeset
  1274
        expected = 0
1a216cb4ee64 verify: add check for mismatch of index and data length
Matt Mackall <mpm@selenic.com>
parents: 1469
diff changeset
  1275
        if self.count():
5312
fb070713ff36 revlog: more robust for damaged indexes
Matt Mackall <mpm@selenic.com>
parents: 5007
diff changeset
  1276
            expected = max(0, self.end(self.count() - 1))
1667
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
  1277
1494
249ca10d37f4 Handle empty logs in repo.checksize
Matt Mackall <mpm@selenic.com>
parents: 1493
diff changeset
  1278
        try:
249ca10d37f4 Handle empty logs in repo.checksize
Matt Mackall <mpm@selenic.com>
parents: 1493
diff changeset
  1279
            f = self.opener(self.datafile)
249ca10d37f4 Handle empty logs in repo.checksize
Matt Mackall <mpm@selenic.com>
parents: 1493
diff changeset
  1280
            f.seek(0, 2)
249ca10d37f4 Handle empty logs in repo.checksize
Matt Mackall <mpm@selenic.com>
parents: 1493
diff changeset
  1281
            actual = f.tell()
1667
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
  1282
            dd = actual - expected
1494
249ca10d37f4 Handle empty logs in repo.checksize
Matt Mackall <mpm@selenic.com>
parents: 1493
diff changeset
  1283
        except IOError, inst:
1667
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
  1284
            if inst.errno != errno.ENOENT:
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
  1285
                raise
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
  1286
            dd = 0
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
  1287
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
  1288
        try:
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
  1289
            f = self.opener(self.indexfile)
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
  1290
            f.seek(0, 2)
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
  1291
            actual = f.tell()
4977
6cb30bc4ca32 revlog: parse revlogv0 indexes into v1 internally
Matt Mackall <mpm@selenic.com>
parents: 4976
diff changeset
  1292
            s = self._io.size
5312
fb070713ff36 revlog: more robust for damaged indexes
Matt Mackall <mpm@selenic.com>
parents: 5007
diff changeset
  1293
            i = max(0, actual / s)
1667
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
  1294
            di = actual - (i * s)
4982
9672e3c42b0c revlog: change _inline from a function to a variable
Matt Mackall <mpm@selenic.com>
parents: 4981
diff changeset
  1295
            if self._inline:
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
  1296
                databytes = 0
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
  1297
                for r in xrange(self.count()):
5312
fb070713ff36 revlog: more robust for damaged indexes
Matt Mackall <mpm@selenic.com>
parents: 5007
diff changeset
  1298
                    databytes += max(0, self.length(r))
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
  1299
                dd = 0
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
  1300
                di = actual - self.count() * s - databytes
1667
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
  1301
        except IOError, inst:
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
  1302
            if inst.errno != errno.ENOENT:
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
  1303
                raise
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
  1304
            di = 0
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
  1305
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
  1306
        return (dd, di)