mercurial/revlog.py
author Gregory Szorc <gregory.szorc@gmail.com>
Thu, 09 Aug 2018 16:02:14 -0700
changeset 39231 b41d023a412a
parent 39192 dbc5ead9f40f
child 39232 0a5b20c107a6
permissions -rw-r--r--
repository: establish API for emitting revision deltas With our revision delta and revision delta request interfaces defined, it is now time to define a method on storage interfaces for using them. So far, the only storage interface that is well-defined and used is file storage. So that is the only interface we need to add a method on. We define an ``emitrevisiondeltas()`` method that takes an iterable of ``irevisiondeltarequest``s and turns them into ``irevisiondelta`` instances. changegroup._handlerevisiondeltarequest() and the looping logic from changegroup.deltagroup() has effectively been moved to revlog.emitrevisiondeltas(). Our filelog wrapper class proxies its emitrevisiondeltas() to the internal revlog instance. The simple store test extension used to verify sanity of storage abstractions has also implemented emitrevisiondeltas() for file storage and the test harness when run with this extension doesn't seem to exhibit any regressions. Rather than create a shared type to represent revision deltas, each storage backend has its own type and the class name identifies where the revision delta was derived from. Differential Revision: https://phab.mercurial-scm.org/D4226
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
8226
8b2cd04a6e97 put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents: 8225
diff changeset
     1
# revlog.py - storage back-end for mercurial
8b2cd04a6e97 put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents: 8225
diff changeset
     2
#
8b2cd04a6e97 put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents: 8225
diff changeset
     3
# Copyright 2005-2007 Matt Mackall <mpm@selenic.com>
8b2cd04a6e97 put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents: 8225
diff changeset
     4
#
8b2cd04a6e97 put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents: 8225
diff changeset
     5
# This software may be used and distributed according to the terms of the
10263
25e572394f5c Update license to GPLv2+
Matt Mackall <mpm@selenic.com>
parents: 10047
diff changeset
     6
# GNU General Public License version 2 or any later version.
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
     7
8227
0a9542703300 turn some comments back into module docstrings
Martin Geisler <mg@lazybytes.net>
parents: 8226
diff changeset
     8
"""Storage back-end for Mercurial.
0a9542703300 turn some comments back into module docstrings
Martin Geisler <mg@lazybytes.net>
parents: 8226
diff changeset
     9
0a9542703300 turn some comments back into module docstrings
Martin Geisler <mg@lazybytes.net>
parents: 8226
diff changeset
    10
This provides efficient delta storage with O(1) retrieve and append
0a9542703300 turn some comments back into module docstrings
Martin Geisler <mg@lazybytes.net>
parents: 8226
diff changeset
    11
and O(changes) merge between branches.
0a9542703300 turn some comments back into module docstrings
Martin Geisler <mg@lazybytes.net>
parents: 8226
diff changeset
    12
"""
0a9542703300 turn some comments back into module docstrings
Martin Geisler <mg@lazybytes.net>
parents: 8226
diff changeset
    13
27361
29f50344fa83 revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27251
diff changeset
    14
from __future__ import absolute_import
29f50344fa83 revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27251
diff changeset
    15
25113
0ca8410ea345 util: drop alias for collections.deque
Martin von Zweigbergk <martinvonz@google.com>
parents: 24454
diff changeset
    16
import collections
35973
4d66993bdcff revlog: add a _datareadfp context manager for data access needs
Boris Feld <boris.feld@octobus.net>
parents: 35972
diff changeset
    17
import contextlib
27361
29f50344fa83 revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27251
diff changeset
    18
import errno
29339
a9e010cd66e1 revlog: use hashlib.sha1 directly instead of through util
Augie Fackler <raf@durin42.com>
parents: 27650
diff changeset
    19
import hashlib
34880
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
    20
import heapq
27430
e240e914d226 revlog: seek to end of file before writing (issue4943)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26705
diff changeset
    21
import os
37442
0596d27457c6 revlog: move parsemeta() and packmeta() from filelog (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37084
diff changeset
    22
import re
27361
29f50344fa83 revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27251
diff changeset
    23
import struct
29f50344fa83 revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27251
diff changeset
    24
import zlib
29f50344fa83 revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27251
diff changeset
    25
29f50344fa83 revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27251
diff changeset
    26
# import stuff from node for others to import from revlog
29f50344fa83 revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27251
diff changeset
    27
from .node import (
29f50344fa83 revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27251
diff changeset
    28
    bin,
29f50344fa83 revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27251
diff changeset
    29
    hex,
39191
5517d62c1bcc revlog: fix pure version of _partialmatch() to include nullid
Martin von Zweigbergk <martinvonz@google.com>
parents: 39157
diff changeset
    30
    nullhex,
27361
29f50344fa83 revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27251
diff changeset
    31
    nullid,
29f50344fa83 revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27251
diff changeset
    32
    nullrev,
37449
a0d71618074f revlog: detect pseudo file nodeids to raise WdirUnsupported exception
Yuya Nishihara <yuya@tcha.org>
parents: 37443
diff changeset
    33
    wdirfilenodeids,
32684
af854b1b36f8 revlog: add support for partial matching of wdir node id
Yuya Nishihara <yuya@tcha.org>
parents: 32659
diff changeset
    34
    wdirhex,
32659
7b17f9de6d3e revlog: map rev(wdirid) to WdirUnsupported exception
Yuya Nishihara <yuya@tcha.org>
parents: 32443
diff changeset
    35
    wdirid,
32403
a28c76e1cea9 revlog: raise WdirUnsupported when wdirrev is passed
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32393
diff changeset
    36
    wdirrev,
27361
29f50344fa83 revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27251
diff changeset
    37
)
29f50344fa83 revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27251
diff changeset
    38
from .i18n import _
35638
edc9330acac1 revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents: 35637
diff changeset
    39
from .thirdparty import (
edc9330acac1 revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents: 35637
diff changeset
    40
    attr,
edc9330acac1 revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents: 35637
diff changeset
    41
)
27361
29f50344fa83 revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27251
diff changeset
    42
from . import (
29f50344fa83 revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27251
diff changeset
    43
    ancestor,
29f50344fa83 revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27251
diff changeset
    44
    error,
29f50344fa83 revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27251
diff changeset
    45
    mdiff,
32372
df448de7cf3b parsers: switch to policy importer
Yuya Nishihara <yuya@tcha.org>
parents: 32315
diff changeset
    46
    policy,
31574
a8e55d6f1d67 revlog: use pycompat.maplist to eagerly evaluate map on Python 3
Augie Fackler <augie@google.com>
parents: 31504
diff changeset
    47
    pycompat,
39231
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
    48
    repository,
27361
29f50344fa83 revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27251
diff changeset
    49
    templatefilters,
29f50344fa83 revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27251
diff changeset
    50
    util,
29f50344fa83 revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27251
diff changeset
    51
)
37084
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module
Yuya Nishihara <yuya@tcha.org>
parents: 36760
diff changeset
    52
from .utils import (
39231
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
    53
    interfaceutil,
37084
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module
Yuya Nishihara <yuya@tcha.org>
parents: 36760
diff changeset
    54
    stringutil,
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module
Yuya Nishihara <yuya@tcha.org>
parents: 36760
diff changeset
    55
)
36
da28286bf6b7 Add smart node lookup by substring or by rev number
mpm@selenic.com
parents: 26
diff changeset
    56
32372
df448de7cf3b parsers: switch to policy importer
Yuya Nishihara <yuya@tcha.org>
parents: 32315
diff changeset
    57
parsers = policy.importmod(r'parsers')
df448de7cf3b parsers: switch to policy importer
Yuya Nishihara <yuya@tcha.org>
parents: 32315
diff changeset
    58
30817
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
    59
# Aliased for performance.
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
    60
_zlibdecompress = zlib.decompress
5007
3addf4531643 revlog: localize some fastpath functions
Matt Mackall <mpm@selenic.com>
parents: 5006
diff changeset
    61
11746
46ac30b17978 revlog: add shallow header flag
Vishakh H <vsh426@gmail.com>
parents: 11745
diff changeset
    62
# revlog header flags
2072
74d3f5336b66 Implement revlogng.
mason@suse.com
parents: 2002
diff changeset
    63
REVLOGV0 = 0
32315
67026d65a4fc revlog: rename constants (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32307
diff changeset
    64
REVLOGV1 = 1
32697
19b9fc40cc51 revlog: skeleton support for version 2 revlogs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32684
diff changeset
    65
# Dummy value until file format is finalized.
19b9fc40cc51 revlog: skeleton support for version 2 revlogs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32684
diff changeset
    66
# Reminder: change the bounds check in revlog.__init__ when this is changed.
19b9fc40cc51 revlog: skeleton support for version 2 revlogs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32684
diff changeset
    67
REVLOGV2 = 0xDEAD
32315
67026d65a4fc revlog: rename constants (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32307
diff changeset
    68
FLAG_INLINE_DATA = (1 << 16)
67026d65a4fc revlog: rename constants (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32307
diff changeset
    69
FLAG_GENERALDELTA = (1 << 17)
67026d65a4fc revlog: rename constants (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32307
diff changeset
    70
REVLOG_DEFAULT_FLAGS = FLAG_INLINE_DATA
67026d65a4fc revlog: rename constants (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32307
diff changeset
    71
REVLOG_DEFAULT_FORMAT = REVLOGV1
2222
c9e264b115e6 Use revlogng and inlined data files by default
mason@suse.com
parents: 2177
diff changeset
    72
REVLOG_DEFAULT_VERSION = REVLOG_DEFAULT_FORMAT | REVLOG_DEFAULT_FLAGS
32315
67026d65a4fc revlog: rename constants (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32307
diff changeset
    73
REVLOGV1_FLAGS = FLAG_INLINE_DATA | FLAG_GENERALDELTA
32697
19b9fc40cc51 revlog: skeleton support for version 2 revlogs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32684
diff changeset
    74
REVLOGV2_FLAGS = REVLOGV1_FLAGS
11746
46ac30b17978 revlog: add shallow header flag
Vishakh H <vsh426@gmail.com>
parents: 11745
diff changeset
    75
46ac30b17978 revlog: add shallow header flag
Vishakh H <vsh426@gmail.com>
parents: 11745
diff changeset
    76
# revlog index flags
23855
4f23081c901e revlog: define censored flag for revlogng index
Mike Edgar <adgar@google.com>
parents: 23338
diff changeset
    77
REVIDX_ISCENSORED = (1 << 15) # revision has censor metadata, must be verified
30829
08b34c3a6f74 revlog: give EXTSTORED flag value to narrowhg
Martin von Zweigbergk <martinvonz@google.com>
parents: 30818
diff changeset
    78
REVIDX_ELLIPSIS = (1 << 14) # revision hash does not match data (narrowhg)
08b34c3a6f74 revlog: give EXTSTORED flag value to narrowhg
Martin von Zweigbergk <martinvonz@google.com>
parents: 30818
diff changeset
    79
REVIDX_EXTSTORED = (1 << 13) # revision data is stored externally
23855
4f23081c901e revlog: define censored flag for revlogng index
Mike Edgar <adgar@google.com>
parents: 23338
diff changeset
    80
REVIDX_DEFAULT_FLAGS = 0
30745
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
    81
# stable order in which flags need to be processed and their processors applied
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
    82
REVIDX_FLAGS_ORDER = [
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
    83
    REVIDX_ISCENSORED,
30829
08b34c3a6f74 revlog: give EXTSTORED flag value to narrowhg
Martin von Zweigbergk <martinvonz@google.com>
parents: 30818
diff changeset
    84
    REVIDX_ELLIPSIS,
30746
9cb0bb0f29f0 revlog: REVIDX_EXTSTORED flag
Remi Chaintron <remi@fb.com>
parents: 30745
diff changeset
    85
    REVIDX_EXTSTORED,
30745
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
    86
]
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
    87
REVIDX_KNOWN_FLAGS = util.bitsfrom(REVIDX_FLAGS_ORDER)
36743
d031609b3cb7 changegroup: do not delta lfs revisions
Jun Wu <quark@fb.com>
parents: 35738
diff changeset
    88
# bitmark for flags that could cause rawdata content change
d031609b3cb7 changegroup: do not delta lfs revisions
Jun Wu <quark@fb.com>
parents: 35738
diff changeset
    89
REVIDX_RAWTEXT_CHANGING_FLAGS = REVIDX_ISCENSORED | REVIDX_EXTSTORED
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
    90
10916
9c84395a338e add documentation for revlog._prereadsize
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 10914
diff changeset
    91
# max size of revlog with inline data
9c84395a338e add documentation for revlog._prereadsize
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 10914
diff changeset
    92
_maxinline = 131072
13253
61c9bc3da402 revlog: remove lazy index
Matt Mackall <mpm@selenic.com>
parents: 13239
diff changeset
    93
_chunksize = 1048576
10913
f2ecc5733c89 revlog: factor out _maxinline global.
Greg Ward <greg-hg@gerg.ca>
parents: 10404
diff changeset
    94
7633
08cabecfa8a8 errors: move revlog errors
Matt Mackall <mpm@selenic.com>
parents: 7365
diff changeset
    95
RevlogError = error.RevlogError
08cabecfa8a8 errors: move revlog errors
Matt Mackall <mpm@selenic.com>
parents: 7365
diff changeset
    96
LookupError = error.LookupError
38841
df0873ab5c14 revlog: use specialized exception for ambiguous prefix lookup
Martin von Zweigbergk <martinvonz@google.com>
parents: 38783
diff changeset
    97
AmbiguousPrefixLookupError = error.AmbiguousPrefixLookupError
22934
8a096d4d0862 revlog: support importing censored file revision tombstones
Mike Edgar <adgar@google.com>
parents: 22785
diff changeset
    98
CensoredNodeError = error.CensoredNodeError
30745
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
    99
ProgrammingError = error.ProgrammingError
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
   100
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
   101
# Store flag processors (cf. 'addflagprocessor()' to register)
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
   102
_flagprocessors = {
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
   103
    REVIDX_ISCENSORED: None,
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
   104
}
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
   105
37442
0596d27457c6 revlog: move parsemeta() and packmeta() from filelog (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37084
diff changeset
   106
_mdre = re.compile('\1\n')
0596d27457c6 revlog: move parsemeta() and packmeta() from filelog (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37084
diff changeset
   107
def parsemeta(text):
0596d27457c6 revlog: move parsemeta() and packmeta() from filelog (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37084
diff changeset
   108
    """return (metadatadict, metadatasize)"""
0596d27457c6 revlog: move parsemeta() and packmeta() from filelog (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37084
diff changeset
   109
    # text can be buffer, so we can't use .startswith or .index
0596d27457c6 revlog: move parsemeta() and packmeta() from filelog (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37084
diff changeset
   110
    if text[:2] != '\1\n':
0596d27457c6 revlog: move parsemeta() and packmeta() from filelog (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37084
diff changeset
   111
        return None, None
0596d27457c6 revlog: move parsemeta() and packmeta() from filelog (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37084
diff changeset
   112
    s = _mdre.search(text, 2).start()
0596d27457c6 revlog: move parsemeta() and packmeta() from filelog (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37084
diff changeset
   113
    mtext = text[2:s]
0596d27457c6 revlog: move parsemeta() and packmeta() from filelog (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37084
diff changeset
   114
    meta = {}
0596d27457c6 revlog: move parsemeta() and packmeta() from filelog (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37084
diff changeset
   115
    for l in mtext.splitlines():
0596d27457c6 revlog: move parsemeta() and packmeta() from filelog (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37084
diff changeset
   116
        k, v = l.split(": ", 1)
0596d27457c6 revlog: move parsemeta() and packmeta() from filelog (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37084
diff changeset
   117
        meta[k] = v
0596d27457c6 revlog: move parsemeta() and packmeta() from filelog (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37084
diff changeset
   118
    return meta, (s + 2)
0596d27457c6 revlog: move parsemeta() and packmeta() from filelog (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37084
diff changeset
   119
0596d27457c6 revlog: move parsemeta() and packmeta() from filelog (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37084
diff changeset
   120
def packmeta(meta, text):
0596d27457c6 revlog: move parsemeta() and packmeta() from filelog (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37084
diff changeset
   121
    keys = sorted(meta)
0596d27457c6 revlog: move parsemeta() and packmeta() from filelog (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37084
diff changeset
   122
    metatext = "".join("%s: %s\n" % (k, meta[k]) for k in keys)
0596d27457c6 revlog: move parsemeta() and packmeta() from filelog (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37084
diff changeset
   123
    return "\1\n%s\1\n%s" % (metatext, text)
0596d27457c6 revlog: move parsemeta() and packmeta() from filelog (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37084
diff changeset
   124
37443
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
   125
def _censoredtext(text):
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
   126
    m, offs = parsemeta(text)
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
   127
    return m and "censored" in m
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
   128
30745
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
   129
def addflagprocessor(flag, processor):
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
   130
    """Register a flag processor on a revision data flag.
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
   131
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
   132
    Invariant:
36743
d031609b3cb7 changegroup: do not delta lfs revisions
Jun Wu <quark@fb.com>
parents: 35738
diff changeset
   133
    - Flags need to be defined in REVIDX_KNOWN_FLAGS and REVIDX_FLAGS_ORDER,
d031609b3cb7 changegroup: do not delta lfs revisions
Jun Wu <quark@fb.com>
parents: 35738
diff changeset
   134
      and REVIDX_RAWTEXT_CHANGING_FLAGS if they can alter rawtext.
30745
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
   135
    - Only one flag processor can be registered on a specific flag.
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
   136
    - flagprocessors must be 3-tuples of functions (read, write, raw) with the
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
   137
      following signatures:
31749
17d0dab7b2b6 revlog: clarify flagprocessor documentation
Jun Wu <quark@fb.com>
parents: 31722
diff changeset
   138
          - (read)  f(self, rawtext) -> text, bool
17d0dab7b2b6 revlog: clarify flagprocessor documentation
Jun Wu <quark@fb.com>
parents: 31722
diff changeset
   139
          - (write) f(self, text) -> rawtext, bool
17d0dab7b2b6 revlog: clarify flagprocessor documentation
Jun Wu <quark@fb.com>
parents: 31722
diff changeset
   140
          - (raw)   f(self, rawtext) -> bool
17d0dab7b2b6 revlog: clarify flagprocessor documentation
Jun Wu <quark@fb.com>
parents: 31722
diff changeset
   141
      "text" is presented to the user. "rawtext" is stored in revlog data, not
17d0dab7b2b6 revlog: clarify flagprocessor documentation
Jun Wu <quark@fb.com>
parents: 31722
diff changeset
   142
      directly visible to the user.
30745
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
   143
      The boolean returned by these transforms is used to determine whether
31749
17d0dab7b2b6 revlog: clarify flagprocessor documentation
Jun Wu <quark@fb.com>
parents: 31722
diff changeset
   144
      the returned text can be used for hash integrity checking. For example,
17d0dab7b2b6 revlog: clarify flagprocessor documentation
Jun Wu <quark@fb.com>
parents: 31722
diff changeset
   145
      if "write" returns False, then "text" is used to generate hash. If
17d0dab7b2b6 revlog: clarify flagprocessor documentation
Jun Wu <quark@fb.com>
parents: 31722
diff changeset
   146
      "write" returns True, that basically means "rawtext" returned by "write"
17d0dab7b2b6 revlog: clarify flagprocessor documentation
Jun Wu <quark@fb.com>
parents: 31722
diff changeset
   147
      should be used to generate hash. Usually, "write" and "read" return
17d0dab7b2b6 revlog: clarify flagprocessor documentation
Jun Wu <quark@fb.com>
parents: 31722
diff changeset
   148
      different booleans. And "raw" returns a same boolean as "write".
30745
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
   149
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
   150
      Note: The 'raw' transform is used for changegroup generation and in some
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
   151
      debug commands. In this case the transform only indicates whether the
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
   152
      contents can be used for hash integrity checks.
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
   153
    """
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
   154
    if not flag & REVIDX_KNOWN_FLAGS:
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
   155
        msg = _("cannot register processor on unknown flag '%#x'.") % (flag)
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
   156
        raise ProgrammingError(msg)
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
   157
    if flag not in REVIDX_FLAGS_ORDER:
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
   158
        msg = _("flag '%#x' undefined in REVIDX_FLAGS_ORDER.") % (flag)
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
   159
        raise ProgrammingError(msg)
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
   160
    if flag in _flagprocessors:
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
   161
        msg = _("cannot register multiple processors on flag '%#x'.") % (flag)
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
   162
        raise error.Abort(msg)
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
   163
    _flagprocessors[flag] = processor
6703
bacfee67c1a9 LookupError should have same __str__ as RevlogError
Alexander Solovyov <piranha@piranha.org.ua>
parents: 6700
diff changeset
   164
4987
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
   165
def getoffset(q):
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
   166
    return int(q >> 16)
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
   167
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
   168
def gettype(q):
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
   169
    return int(q & 0xFFFF)
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
   170
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
   171
def offset_type(offset, type):
30543
03fae9048fa1 revlog: ensure that flags do not overflow 2 bytes
Cotizo Sima <cotizo@fb.com>
parents: 30391
diff changeset
   172
    if (type & ~REVIDX_KNOWN_FLAGS) != 0:
03fae9048fa1 revlog: ensure that flags do not overflow 2 bytes
Cotizo Sima <cotizo@fb.com>
parents: 30391
diff changeset
   173
        raise ValueError('unknown revlog index flags')
31504
73aa13bc8dac revlog: use int instead of long
Augie Fackler <augie@google.com>
parents: 31369
diff changeset
   174
    return int(int(offset) << 16 | type)
4987
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
   175
29339
a9e010cd66e1 revlog: use hashlib.sha1 directly instead of through util
Augie Fackler <raf@durin42.com>
parents: 27650
diff changeset
   176
_nullhash = hashlib.sha1(nullid)
7883
c63c30ae9e39 revlog: faster hash computation when one of the parent node is null
Nicolas Dumazet <nicdumz.commits@gmail.com>
parents: 7874
diff changeset
   177
1091
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
   178
def hash(text, p1, p2):
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
   179
    """generate a hash from the given text and its parent hashes
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
   180
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
   181
    This hash combines both the current file contents and its history
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
   182
    in a manner that makes it easy to distinguish nodes with the same
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
   183
    content in the revision graph.
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
   184
    """
7883
c63c30ae9e39 revlog: faster hash computation when one of the parent node is null
Nicolas Dumazet <nicdumz.commits@gmail.com>
parents: 7874
diff changeset
   185
    # As of now, if one of the parent node is null, p2 is null
c63c30ae9e39 revlog: faster hash computation when one of the parent node is null
Nicolas Dumazet <nicdumz.commits@gmail.com>
parents: 7874
diff changeset
   186
    if p2 == nullid:
c63c30ae9e39 revlog: faster hash computation when one of the parent node is null
Nicolas Dumazet <nicdumz.commits@gmail.com>
parents: 7874
diff changeset
   187
        # deep copy of a hash is faster than creating one
22784
0f4e655136ef revlog: mark nullhash as module-private
Augie Fackler <raf@durin42.com>
parents: 22389
diff changeset
   188
        s = _nullhash.copy()
7883
c63c30ae9e39 revlog: faster hash computation when one of the parent node is null
Nicolas Dumazet <nicdumz.commits@gmail.com>
parents: 7874
diff changeset
   189
        s.update(p1)
c63c30ae9e39 revlog: faster hash computation when one of the parent node is null
Nicolas Dumazet <nicdumz.commits@gmail.com>
parents: 7874
diff changeset
   190
    else:
c63c30ae9e39 revlog: faster hash computation when one of the parent node is null
Nicolas Dumazet <nicdumz.commits@gmail.com>
parents: 7874
diff changeset
   191
        # none of the parent nodes are nullid
33391
943b8c37f49d revlog: micro-optimize the computation of hashes
Alex Gaynor <agaynor@mozilla.com>
parents: 33260
diff changeset
   192
        if p1 < p2:
943b8c37f49d revlog: micro-optimize the computation of hashes
Alex Gaynor <agaynor@mozilla.com>
parents: 33260
diff changeset
   193
            a = p1
943b8c37f49d revlog: micro-optimize the computation of hashes
Alex Gaynor <agaynor@mozilla.com>
parents: 33260
diff changeset
   194
            b = p2
943b8c37f49d revlog: micro-optimize the computation of hashes
Alex Gaynor <agaynor@mozilla.com>
parents: 33260
diff changeset
   195
        else:
943b8c37f49d revlog: micro-optimize the computation of hashes
Alex Gaynor <agaynor@mozilla.com>
parents: 33260
diff changeset
   196
            a = p2
943b8c37f49d revlog: micro-optimize the computation of hashes
Alex Gaynor <agaynor@mozilla.com>
parents: 33260
diff changeset
   197
            b = p1
943b8c37f49d revlog: micro-optimize the computation of hashes
Alex Gaynor <agaynor@mozilla.com>
parents: 33260
diff changeset
   198
        s = hashlib.sha1(a)
943b8c37f49d revlog: micro-optimize the computation of hashes
Alex Gaynor <agaynor@mozilla.com>
parents: 33260
diff changeset
   199
        s.update(b)
1091
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
   200
    s.update(text)
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
   201
    return s.digest()
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
   202
38637
e33f784f2a44 revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents: 38636
diff changeset
   203
class _testrevlog(object):
e33f784f2a44 revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents: 38636
diff changeset
   204
    """minimalist fake revlog to use in doctests"""
e33f784f2a44 revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents: 38636
diff changeset
   205
e33f784f2a44 revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents: 38636
diff changeset
   206
    def __init__(self, data, density=0.5, mingap=0):
e33f784f2a44 revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents: 38636
diff changeset
   207
        """data is an list of revision payload boundaries"""
e33f784f2a44 revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents: 38636
diff changeset
   208
        self._data = data
e33f784f2a44 revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents: 38636
diff changeset
   209
        self._srdensitythreshold = density
e33f784f2a44 revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents: 38636
diff changeset
   210
        self._srmingapsize = mingap
e33f784f2a44 revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents: 38636
diff changeset
   211
e33f784f2a44 revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents: 38636
diff changeset
   212
    def start(self, rev):
e33f784f2a44 revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents: 38636
diff changeset
   213
        if rev == 0:
e33f784f2a44 revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents: 38636
diff changeset
   214
            return 0
e33f784f2a44 revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents: 38636
diff changeset
   215
        return self._data[rev - 1]
e33f784f2a44 revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents: 38636
diff changeset
   216
e33f784f2a44 revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents: 38636
diff changeset
   217
    def end(self, rev):
e33f784f2a44 revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents: 38636
diff changeset
   218
        return self._data[rev]
e33f784f2a44 revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents: 38636
diff changeset
   219
e33f784f2a44 revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents: 38636
diff changeset
   220
    def length(self, rev):
e33f784f2a44 revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents: 38636
diff changeset
   221
        return self.end(rev) - self.start(rev)
e33f784f2a44 revlog: introduce a tiny mock of a revlog class
Boris Feld <boris.feld@octobus.net>
parents: 38636
diff changeset
   222
38718
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents: 38717
diff changeset
   223
    def __len__(self):
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents: 38717
diff changeset
   224
        return len(self._data)
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents: 38717
diff changeset
   225
34898
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents: 34881
diff changeset
   226
def _trimchunk(revlog, revs, startidx, endidx=None):
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents: 34881
diff changeset
   227
    """returns revs[startidx:endidx] without empty trailing revs
38638
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
   228
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
   229
    Doctest Setup
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
   230
    >>> revlog = _testrevlog([
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
   231
    ...  5,  #0
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
   232
    ...  10, #1
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
   233
    ...  12, #2
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
   234
    ...  12, #3 (empty)
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
   235
    ...  17, #4
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
   236
    ...  21, #5
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
   237
    ...  21, #6 (empty)
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
   238
    ... ])
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
   239
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
   240
    Contiguous cases:
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
   241
    >>> _trimchunk(revlog, [0, 1, 2, 3, 4, 5, 6], 0)
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
   242
    [0, 1, 2, 3, 4, 5]
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
   243
    >>> _trimchunk(revlog, [0, 1, 2, 3, 4, 5, 6], 0, 5)
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
   244
    [0, 1, 2, 3, 4]
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
   245
    >>> _trimchunk(revlog, [0, 1, 2, 3, 4, 5, 6], 0, 4)
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
   246
    [0, 1, 2]
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
   247
    >>> _trimchunk(revlog, [0, 1, 2, 3, 4, 5, 6], 2, 4)
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
   248
    [2]
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
   249
    >>> _trimchunk(revlog, [0, 1, 2, 3, 4, 5, 6], 3)
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
   250
    [3, 4, 5]
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
   251
    >>> _trimchunk(revlog, [0, 1, 2, 3, 4, 5, 6], 3, 5)
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
   252
    [3, 4]
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
   253
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
   254
    Discontiguous cases:
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
   255
    >>> _trimchunk(revlog, [1, 3, 5, 6], 0)
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
   256
    [1, 3, 5]
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
   257
    >>> _trimchunk(revlog, [1, 3, 5, 6], 0, 2)
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
   258
    [1]
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
   259
    >>> _trimchunk(revlog, [1, 3, 5, 6], 1, 3)
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
   260
    [3, 5]
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
   261
    >>> _trimchunk(revlog, [1, 3, 5, 6], 1)
740f7d447222 revlog: add a doctest to _trimchunk
Boris Feld <boris.feld@octobus.net>
parents: 38637
diff changeset
   262
    [3, 5]
34898
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents: 34881
diff changeset
   263
    """
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents: 34881
diff changeset
   264
    length = revlog.length
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents: 34881
diff changeset
   265
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents: 34881
diff changeset
   266
    if endidx is None:
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents: 34881
diff changeset
   267
        endidx = len(revs)
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents: 34881
diff changeset
   268
39139
3730b779ed5b sparse-revlog: fix delta validity computation
Boris Feld <boris.feld@octobus.net>
parents: 38763
diff changeset
   269
    # If we have a non-emtpy delta candidate, there are nothing to trim
3730b779ed5b sparse-revlog: fix delta validity computation
Boris Feld <boris.feld@octobus.net>
parents: 38763
diff changeset
   270
    if revs[endidx - 1] < len(revlog):
3730b779ed5b sparse-revlog: fix delta validity computation
Boris Feld <boris.feld@octobus.net>
parents: 38763
diff changeset
   271
        # Trim empty revs at the end, except the very first revision of a chain
3730b779ed5b sparse-revlog: fix delta validity computation
Boris Feld <boris.feld@octobus.net>
parents: 38763
diff changeset
   272
        while (endidx > 1
3730b779ed5b sparse-revlog: fix delta validity computation
Boris Feld <boris.feld@octobus.net>
parents: 38763
diff changeset
   273
                and endidx > startidx
3730b779ed5b sparse-revlog: fix delta validity computation
Boris Feld <boris.feld@octobus.net>
parents: 38763
diff changeset
   274
                and length(revs[endidx - 1]) == 0):
3730b779ed5b sparse-revlog: fix delta validity computation
Boris Feld <boris.feld@octobus.net>
parents: 38763
diff changeset
   275
            endidx -= 1
34898
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents: 34881
diff changeset
   276
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents: 34881
diff changeset
   277
    return revs[startidx:endidx]
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents: 34881
diff changeset
   278
39139
3730b779ed5b sparse-revlog: fix delta validity computation
Boris Feld <boris.feld@octobus.net>
parents: 38763
diff changeset
   279
def _segmentspan(revlog, revs, deltainfo=None):
38636
25534e0cf0d9 revlog: _segmentspan computes the byte span of a segment
Paul Morelle <paul.morelle@octobus.net>
parents: 38635
diff changeset
   280
    """Get the byte span of a segment of revisions
25534e0cf0d9 revlog: _segmentspan computes the byte span of a segment
Paul Morelle <paul.morelle@octobus.net>
parents: 38635
diff changeset
   281
25534e0cf0d9 revlog: _segmentspan computes the byte span of a segment
Paul Morelle <paul.morelle@octobus.net>
parents: 38635
diff changeset
   282
    revs is a sorted array of revision numbers
38639
2dd4cf273804 revlog: add a doctest for _segmentspan
Boris Feld <boris.feld@octobus.net>
parents: 38638
diff changeset
   283
2dd4cf273804 revlog: add a doctest for _segmentspan
Boris Feld <boris.feld@octobus.net>
parents: 38638
diff changeset
   284
    >>> revlog = _testrevlog([
2dd4cf273804 revlog: add a doctest for _segmentspan
Boris Feld <boris.feld@octobus.net>
parents: 38638
diff changeset
   285
    ...  5,  #0
2dd4cf273804 revlog: add a doctest for _segmentspan
Boris Feld <boris.feld@octobus.net>
parents: 38638
diff changeset
   286
    ...  10, #1
2dd4cf273804 revlog: add a doctest for _segmentspan
Boris Feld <boris.feld@octobus.net>
parents: 38638
diff changeset
   287
    ...  12, #2
2dd4cf273804 revlog: add a doctest for _segmentspan
Boris Feld <boris.feld@octobus.net>
parents: 38638
diff changeset
   288
    ...  12, #3 (empty)
2dd4cf273804 revlog: add a doctest for _segmentspan
Boris Feld <boris.feld@octobus.net>
parents: 38638
diff changeset
   289
    ...  17, #4
2dd4cf273804 revlog: add a doctest for _segmentspan
Boris Feld <boris.feld@octobus.net>
parents: 38638
diff changeset
   290
    ... ])
2dd4cf273804 revlog: add a doctest for _segmentspan
Boris Feld <boris.feld@octobus.net>
parents: 38638
diff changeset
   291
2dd4cf273804 revlog: add a doctest for _segmentspan
Boris Feld <boris.feld@octobus.net>
parents: 38638
diff changeset
   292
    >>> _segmentspan(revlog, [0, 1, 2, 3, 4])
2dd4cf273804 revlog: add a doctest for _segmentspan
Boris Feld <boris.feld@octobus.net>
parents: 38638
diff changeset
   293
    17
2dd4cf273804 revlog: add a doctest for _segmentspan
Boris Feld <boris.feld@octobus.net>
parents: 38638
diff changeset
   294
    >>> _segmentspan(revlog, [0, 4])
2dd4cf273804 revlog: add a doctest for _segmentspan
Boris Feld <boris.feld@octobus.net>
parents: 38638
diff changeset
   295
    17
2dd4cf273804 revlog: add a doctest for _segmentspan
Boris Feld <boris.feld@octobus.net>
parents: 38638
diff changeset
   296
    >>> _segmentspan(revlog, [3, 4])
2dd4cf273804 revlog: add a doctest for _segmentspan
Boris Feld <boris.feld@octobus.net>
parents: 38638
diff changeset
   297
    5
2dd4cf273804 revlog: add a doctest for _segmentspan
Boris Feld <boris.feld@octobus.net>
parents: 38638
diff changeset
   298
    >>> _segmentspan(revlog, [1, 2, 3,])
2dd4cf273804 revlog: add a doctest for _segmentspan
Boris Feld <boris.feld@octobus.net>
parents: 38638
diff changeset
   299
    7
2dd4cf273804 revlog: add a doctest for _segmentspan
Boris Feld <boris.feld@octobus.net>
parents: 38638
diff changeset
   300
    >>> _segmentspan(revlog, [1, 3])
2dd4cf273804 revlog: add a doctest for _segmentspan
Boris Feld <boris.feld@octobus.net>
parents: 38638
diff changeset
   301
    7
38636
25534e0cf0d9 revlog: _segmentspan computes the byte span of a segment
Paul Morelle <paul.morelle@octobus.net>
parents: 38635
diff changeset
   302
    """
25534e0cf0d9 revlog: _segmentspan computes the byte span of a segment
Paul Morelle <paul.morelle@octobus.net>
parents: 38635
diff changeset
   303
    if not revs:
25534e0cf0d9 revlog: _segmentspan computes the byte span of a segment
Paul Morelle <paul.morelle@octobus.net>
parents: 38635
diff changeset
   304
        return 0
39139
3730b779ed5b sparse-revlog: fix delta validity computation
Boris Feld <boris.feld@octobus.net>
parents: 38763
diff changeset
   305
    if deltainfo is not None and len(revlog) <= revs[-1]:
3730b779ed5b sparse-revlog: fix delta validity computation
Boris Feld <boris.feld@octobus.net>
parents: 38763
diff changeset
   306
        if len(revs) == 1:
3730b779ed5b sparse-revlog: fix delta validity computation
Boris Feld <boris.feld@octobus.net>
parents: 38763
diff changeset
   307
            return deltainfo.deltalen
3730b779ed5b sparse-revlog: fix delta validity computation
Boris Feld <boris.feld@octobus.net>
parents: 38763
diff changeset
   308
        offset = revlog.end(len(revlog) - 1)
3730b779ed5b sparse-revlog: fix delta validity computation
Boris Feld <boris.feld@octobus.net>
parents: 38763
diff changeset
   309
        end = deltainfo.deltalen + offset
3730b779ed5b sparse-revlog: fix delta validity computation
Boris Feld <boris.feld@octobus.net>
parents: 38763
diff changeset
   310
    else:
3730b779ed5b sparse-revlog: fix delta validity computation
Boris Feld <boris.feld@octobus.net>
parents: 38763
diff changeset
   311
        end = revlog.end(revs[-1])
3730b779ed5b sparse-revlog: fix delta validity computation
Boris Feld <boris.feld@octobus.net>
parents: 38763
diff changeset
   312
    return end - revlog.start(revs[0])
38636
25534e0cf0d9 revlog: _segmentspan computes the byte span of a segment
Paul Morelle <paul.morelle@octobus.net>
parents: 38635
diff changeset
   313
38718
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents: 38717
diff changeset
   314
def _slicechunk(revlog, revs, deltainfo=None, targetsize=None):
34824
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
   315
    """slice revs to reduce the amount of unrelated data to be read from disk.
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
   316
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
   317
    ``revs`` is sliced into groups that should be read in one time.
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
   318
    Assume that revs are sorted.
38640
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
   319
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
   320
    The initial chunk is sliced until the overall density (payload/chunks-span
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
   321
    ratio) is above `revlog._srdensitythreshold`. No gap smaller than
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
   322
    `revlog._srmingapsize` is skipped.
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
   323
38643
967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents: 38642
diff changeset
   324
    If `targetsize` is set, no chunk larger than `targetsize` will be yield.
967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents: 38642
diff changeset
   325
    For consistency with other slicing choice, this limit won't go lower than
967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents: 38642
diff changeset
   326
    `revlog._srmingapsize`.
967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents: 38642
diff changeset
   327
967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents: 38642
diff changeset
   328
    If individual revisions chunk are larger than this limit, they will still
967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents: 38642
diff changeset
   329
    be raised individually.
967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents: 38642
diff changeset
   330
38640
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
   331
    >>> revlog = _testrevlog([
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
   332
    ...  5,  #00 (5)
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
   333
    ...  10, #01 (5)
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
   334
    ...  12, #02 (2)
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
   335
    ...  12, #03 (empty)
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
   336
    ...  27, #04 (15)
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
   337
    ...  31, #05 (4)
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
   338
    ...  31, #06 (empty)
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
   339
    ...  42, #07 (11)
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
   340
    ...  47, #08 (5)
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
   341
    ...  47, #09 (empty)
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
   342
    ...  48, #10 (1)
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
   343
    ...  51, #11 (3)
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
   344
    ...  74, #12 (23)
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
   345
    ...  85, #13 (11)
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
   346
    ...  86, #14 (1)
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
   347
    ...  91, #15 (5)
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
   348
    ... ])
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
   349
38655
cd1c484e31e8 revlog: adjust doctest examples to be portable to Python 3
Augie Fackler <augie@google.com>
parents: 38644
diff changeset
   350
    >>> list(_slicechunk(revlog, list(range(16))))
38640
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
   351
    [[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]]
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
   352
    >>> list(_slicechunk(revlog, [0, 15]))
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
   353
    [[0], [15]]
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
   354
    >>> list(_slicechunk(revlog, [0, 11, 15]))
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
   355
    [[0], [11], [15]]
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
   356
    >>> list(_slicechunk(revlog, [0, 11, 13, 15]))
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
   357
    [[0], [11, 13, 15]]
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
   358
    >>> list(_slicechunk(revlog, [1, 2, 3, 5, 8, 10, 11, 14]))
f62b8fb0a484 revlog: document and test _slicechunk
Boris Feld <boris.feld@octobus.net>
parents: 38639
diff changeset
   359
    [[1, 2], [5, 8, 10, 11], [14]]
38643
967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents: 38642
diff changeset
   360
967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents: 38642
diff changeset
   361
    Slicing with a maximum chunk size
38718
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents: 38717
diff changeset
   362
    >>> list(_slicechunk(revlog, [0, 11, 13, 15], targetsize=15))
38643
967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents: 38642
diff changeset
   363
    [[0], [11], [13], [15]]
38718
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents: 38717
diff changeset
   364
    >>> list(_slicechunk(revlog, [0, 11, 13, 15], targetsize=20))
38643
967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents: 38642
diff changeset
   365
    [[0], [11], [13, 15]]
34824
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
   366
    """
38643
967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents: 38642
diff changeset
   367
    if targetsize is not None:
967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents: 38642
diff changeset
   368
        targetsize = max(targetsize, revlog._srmingapsize)
38718
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents: 38717
diff changeset
   369
    # targetsize should not be specified when evaluating delta candidates:
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents: 38717
diff changeset
   370
    # * targetsize is used to ensure we stay within specification when reading,
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents: 38717
diff changeset
   371
    # * deltainfo is used to pick are good delta chain when writing.
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents: 38717
diff changeset
   372
    if not (deltainfo is None or targetsize is None):
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents: 38717
diff changeset
   373
        msg = 'cannot use `targetsize` with a `deltainfo`'
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents: 38717
diff changeset
   374
        raise error.ProgrammingError(msg)
38641
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
   375
    for chunk in _slicechunktodensity(revlog, revs,
38718
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents: 38717
diff changeset
   376
                                      deltainfo,
38641
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
   377
                                      revlog._srdensitythreshold,
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
   378
                                      revlog._srmingapsize):
38643
967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents: 38642
diff changeset
   379
        for subchunk in _slicechunktosize(revlog, chunk, targetsize):
967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size
Boris Feld <boris.feld@octobus.net>
parents: 38642
diff changeset
   380
            yield subchunk
38641
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
   381
38718
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents: 38717
diff changeset
   382
def _slicechunktosize(revlog, revs, targetsize=None):
38642
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   383
    """slice revs to match the target size
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   384
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   385
    This is intended to be used on chunk that density slicing selected by that
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   386
    are still too large compared to the read garantee of revlog. This might
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   387
    happens when "minimal gap size" interrupted the slicing or when chain are
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   388
    built in a way that create large blocks next to each other.
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   389
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   390
    >>> revlog = _testrevlog([
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   391
    ...  3,  #0 (3)
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   392
    ...  5,  #1 (2)
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   393
    ...  6,  #2 (1)
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   394
    ...  8,  #3 (2)
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   395
    ...  8,  #4 (empty)
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   396
    ...  11, #5 (3)
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   397
    ...  12, #6 (1)
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   398
    ...  13, #7 (1)
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   399
    ...  14, #8 (1)
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   400
    ... ])
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   401
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   402
    Cases where chunk is already small enough
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   403
    >>> list(_slicechunktosize(revlog, [0], 3))
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   404
    [[0]]
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   405
    >>> list(_slicechunktosize(revlog, [6, 7], 3))
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   406
    [[6, 7]]
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   407
    >>> list(_slicechunktosize(revlog, [0], None))
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   408
    [[0]]
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   409
    >>> list(_slicechunktosize(revlog, [6, 7], None))
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   410
    [[6, 7]]
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   411
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   412
    cases where we need actual slicing
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   413
    >>> list(_slicechunktosize(revlog, [0, 1], 3))
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   414
    [[0], [1]]
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   415
    >>> list(_slicechunktosize(revlog, [1, 3], 3))
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   416
    [[1], [3]]
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   417
    >>> list(_slicechunktosize(revlog, [1, 2, 3], 3))
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   418
    [[1, 2], [3]]
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   419
    >>> list(_slicechunktosize(revlog, [3, 5], 3))
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   420
    [[3], [5]]
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   421
    >>> list(_slicechunktosize(revlog, [3, 4, 5], 3))
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   422
    [[3], [5]]
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   423
    >>> list(_slicechunktosize(revlog, [5, 6, 7, 8], 3))
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   424
    [[5], [6, 7, 8]]
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   425
    >>> list(_slicechunktosize(revlog, [0, 1, 2, 3, 4, 5, 6, 7, 8], 3))
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   426
    [[0], [1, 2], [3], [5], [6, 7, 8]]
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   427
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   428
    Case with too large individual chunk (must return valid chunk)
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   429
    >>> list(_slicechunktosize(revlog, [0, 1], 2))
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   430
    [[0], [1]]
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   431
    >>> list(_slicechunktosize(revlog, [1, 3], 1))
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   432
    [[1], [3]]
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   433
    >>> list(_slicechunktosize(revlog, [3, 4, 5], 2))
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   434
    [[3], [5]]
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   435
    """
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   436
    assert targetsize is None or 0 <= targetsize
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   437
    if targetsize is None or _segmentspan(revlog, revs) <= targetsize:
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   438
        yield revs
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   439
        return
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   440
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   441
    startrevidx = 0
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   442
    startdata = revlog.start(revs[0])
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   443
    endrevidx = 0
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   444
    iterrevs = enumerate(revs)
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   445
    next(iterrevs) # skip first rev.
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   446
    for idx, r in iterrevs:
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   447
        span = revlog.end(r) - startdata
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   448
        if span <= targetsize:
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   449
            endrevidx = idx
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   450
        else:
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   451
            chunk = _trimchunk(revlog, revs, startrevidx, endrevidx + 1)
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   452
            if chunk:
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   453
                yield chunk
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   454
            startrevidx = idx
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   455
            startdata = revlog.start(r)
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   456
            endrevidx = idx
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   457
    yield _trimchunk(revlog, revs, startrevidx)
e59e27e52297 revlog: add function to slice chunk down to a given size
Boris Feld <boris.feld@octobus.net>
parents: 38641
diff changeset
   458
38718
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents: 38717
diff changeset
   459
def _slicechunktodensity(revlog, revs, deltainfo=None, targetdensity=0.5,
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents: 38717
diff changeset
   460
                         mingapsize=0):
38641
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
   461
    """slice revs to reduce the amount of unrelated data to be read from disk.
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
   462
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
   463
    ``revs`` is sliced into groups that should be read in one time.
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
   464
    Assume that revs are sorted.
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
   465
38718
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents: 38717
diff changeset
   466
    ``deltainfo`` is a _deltainfo instance of a revision that we would append
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents: 38717
diff changeset
   467
    to the top of the revlog.
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents: 38717
diff changeset
   468
38641
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
   469
    The initial chunk is sliced until the overall density (payload/chunks-span
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
   470
    ratio) is above `targetdensity`. No gap smaller than `mingapsize` is
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
   471
    skipped.
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
   472
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
   473
    >>> revlog = _testrevlog([
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
   474
    ...  5,  #00 (5)
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
   475
    ...  10, #01 (5)
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
   476
    ...  12, #02 (2)
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
   477
    ...  12, #03 (empty)
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
   478
    ...  27, #04 (15)
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
   479
    ...  31, #05 (4)
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
   480
    ...  31, #06 (empty)
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
   481
    ...  42, #07 (11)
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
   482
    ...  47, #08 (5)
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
   483
    ...  47, #09 (empty)
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
   484
    ...  48, #10 (1)
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
   485
    ...  51, #11 (3)
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
   486
    ...  74, #12 (23)
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
   487
    ...  85, #13 (11)
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
   488
    ...  86, #14 (1)
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
   489
    ...  91, #15 (5)
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
   490
    ... ])
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
   491
38655
cd1c484e31e8 revlog: adjust doctest examples to be portable to Python 3
Augie Fackler <augie@google.com>
parents: 38644
diff changeset
   492
    >>> list(_slicechunktodensity(revlog, list(range(16))))
38641
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
   493
    [[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]]
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
   494
    >>> list(_slicechunktodensity(revlog, [0, 15]))
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
   495
    [[0], [15]]
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
   496
    >>> list(_slicechunktodensity(revlog, [0, 11, 15]))
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
   497
    [[0], [11], [15]]
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
   498
    >>> list(_slicechunktodensity(revlog, [0, 11, 13, 15]))
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
   499
    [[0], [11, 13, 15]]
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
   500
    >>> list(_slicechunktodensity(revlog, [1, 2, 3, 5, 8, 10, 11, 14]))
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
   501
    [[1, 2], [5, 8, 10, 11], [14]]
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
   502
    >>> list(_slicechunktodensity(revlog, [1, 2, 3, 5, 8, 10, 11, 14],
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
   503
    ...                           mingapsize=20))
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
   504
    [[1, 2, 3, 5, 8, 10, 11], [14]]
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
   505
    >>> list(_slicechunktodensity(revlog, [1, 2, 3, 5, 8, 10, 11, 14],
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
   506
    ...                           targetdensity=0.95))
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
   507
    [[1, 2], [5], [8, 10, 11], [14]]
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
   508
    >>> list(_slicechunktodensity(revlog, [1, 2, 3, 5, 8, 10, 11, 14],
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
   509
    ...                           targetdensity=0.95, mingapsize=12))
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
   510
    [[1, 2], [5, 8, 10, 11], [14]]
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
   511
    """
34824
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
   512
    start = revlog.start
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
   513
    length = revlog.length
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
   514
34880
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
   515
    if len(revs) <= 1:
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
   516
        yield revs
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
   517
        return
34824
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
   518
38718
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents: 38717
diff changeset
   519
    nextrev = len(revlog)
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents: 38717
diff changeset
   520
    nextoffset = revlog.end(nextrev - 1)
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents: 38717
diff changeset
   521
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents: 38717
diff changeset
   522
    if deltainfo is None:
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents: 38717
diff changeset
   523
        deltachainspan = _segmentspan(revlog, revs)
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents: 38717
diff changeset
   524
        chainpayload = sum(length(r) for r in revs)
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents: 38717
diff changeset
   525
    else:
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents: 38717
diff changeset
   526
        deltachainspan = deltainfo.distance
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents: 38717
diff changeset
   527
        chainpayload = deltainfo.compresseddeltalen
34880
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
   528
38641
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
   529
    if deltachainspan < mingapsize:
38635
d083ae26c325 revlog: early return in _slicechunk when span is already small enough
Boris Feld <boris.feld@octobus.net>
parents: 38634
diff changeset
   530
        yield revs
d083ae26c325 revlog: early return in _slicechunk when span is already small enough
Boris Feld <boris.feld@octobus.net>
parents: 38634
diff changeset
   531
        return
d083ae26c325 revlog: early return in _slicechunk when span is already small enough
Boris Feld <boris.feld@octobus.net>
parents: 38634
diff changeset
   532
38718
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents: 38717
diff changeset
   533
    readdata = deltachainspan
34824
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
   534
34880
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
   535
    if deltachainspan:
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
   536
        density = chainpayload / float(deltachainspan)
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
   537
    else:
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
   538
        density = 1.0
34824
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
   539
38641
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
   540
    if density >= targetdensity:
38634
f0ea8b847831 revlog: early return in _slicechunk when density is already good
Paul Morelle <paul.morelle@octobus.net>
parents: 38632
diff changeset
   541
        yield revs
f0ea8b847831 revlog: early return in _slicechunk when density is already good
Paul Morelle <paul.morelle@octobus.net>
parents: 38632
diff changeset
   542
        return
f0ea8b847831 revlog: early return in _slicechunk when density is already good
Paul Morelle <paul.morelle@octobus.net>
parents: 38632
diff changeset
   543
39139
3730b779ed5b sparse-revlog: fix delta validity computation
Boris Feld <boris.feld@octobus.net>
parents: 38763
diff changeset
   544
    if deltainfo is not None and deltainfo.deltalen:
38718
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents: 38717
diff changeset
   545
        revs = list(revs)
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents: 38717
diff changeset
   546
        revs.append(nextrev)
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents: 38717
diff changeset
   547
34880
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
   548
    # Store the gaps in a heap to have them sorted by decreasing size
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
   549
    gapsheap = []
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
   550
    heapq.heapify(gapsheap)
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
   551
    prevend = None
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
   552
    for i, rev in enumerate(revs):
38718
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents: 38717
diff changeset
   553
        if rev < nextrev:
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents: 38717
diff changeset
   554
            revstart = start(rev)
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents: 38717
diff changeset
   555
            revlen = length(rev)
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents: 38717
diff changeset
   556
        else:
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents: 38717
diff changeset
   557
            revstart = nextoffset
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents: 38717
diff changeset
   558
            revlen = deltainfo.deltalen
34824
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
   559
34898
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents: 34881
diff changeset
   560
        # Skip empty revisions to form larger holes
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents: 34881
diff changeset
   561
        if revlen == 0:
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents: 34881
diff changeset
   562
            continue
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents: 34881
diff changeset
   563
34880
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
   564
        if prevend is not None:
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
   565
            gapsize = revstart - prevend
34881
8c9b08a0c48c sparse-read: skip gaps too small to be worth splitting
Paul Morelle <paul.morelle@octobus.net>
parents: 34880
diff changeset
   566
            # only consider holes that are large enough
38641
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
   567
            if gapsize > mingapsize:
34880
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
   568
                heapq.heappush(gapsheap, (-gapsize, i))
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
   569
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
   570
        prevend = revstart + revlen
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
   571
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
   572
    # Collect the indices of the largest holes until the density is acceptable
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
   573
    indicesheap = []
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
   574
    heapq.heapify(indicesheap)
38641
feba6be0941b revlog: extract density based slicing into its own function
Boris Feld <boris.feld@octobus.net>
parents: 38640
diff changeset
   575
    while gapsheap and density < targetdensity:
34880
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
   576
        oppgapsize, gapidx = heapq.heappop(gapsheap)
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
   577
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
   578
        heapq.heappush(indicesheap, gapidx)
34824
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
   579
34880
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
   580
        # the gap sizes are stored as negatives to be sorted decreasingly
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
   581
        # by the heap
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
   582
        readdata -= (-oppgapsize)
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
   583
        if readdata > 0:
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
   584
            density = chainpayload / float(readdata)
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
   585
        else:
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
   586
            density = 1.0
34824
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
   587
34880
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
   588
    # Cut the revs at collected indices
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
   589
    previdx = 0
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
   590
    while indicesheap:
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
   591
        idx = heapq.heappop(indicesheap)
34898
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents: 34881
diff changeset
   592
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents: 34881
diff changeset
   593
        chunk = _trimchunk(revlog, revs, previdx, idx)
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents: 34881
diff changeset
   594
        if chunk:
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents: 34881
diff changeset
   595
            yield chunk
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents: 34881
diff changeset
   596
34880
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one
Boris Feld <boris.feld@octobus.net>
parents: 34825
diff changeset
   597
        previdx = idx
34898
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents: 34881
diff changeset
   598
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents: 34881
diff changeset
   599
    chunk = _trimchunk(revlog, revs, previdx)
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents: 34881
diff changeset
   600
    if chunk:
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk
Paul Morelle <paul.morelle@octobus.net>
parents: 34881
diff changeset
   601
        yield chunk
34824
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
   602
35638
edc9330acac1 revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents: 35637
diff changeset
   603
@attr.s(slots=True, frozen=True)
edc9330acac1 revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents: 35637
diff changeset
   604
class _deltainfo(object):
edc9330acac1 revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents: 35637
diff changeset
   605
    distance = attr.ib()
edc9330acac1 revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents: 35637
diff changeset
   606
    deltalen = attr.ib()
edc9330acac1 revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents: 35637
diff changeset
   607
    data = attr.ib()
edc9330acac1 revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents: 35637
diff changeset
   608
    base = attr.ib()
edc9330acac1 revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents: 35637
diff changeset
   609
    chainbase = attr.ib()
edc9330acac1 revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents: 35637
diff changeset
   610
    chainlen = attr.ib()
edc9330acac1 revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents: 35637
diff changeset
   611
    compresseddeltalen = attr.ib()
39154
e0da43e2f71f revlog: compute snapshot depth on delta info
Boris Feld <boris.feld@octobus.net>
parents: 39152
diff changeset
   612
    snapshotdepth = attr.ib()
35638
edc9330acac1 revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents: 35637
diff changeset
   613
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   614
class _deltacomputer(object):
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   615
    def __init__(self, revlog):
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   616
        self.revlog = revlog
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   617
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   618
    def _getcandidaterevs(self, p1, p2, cachedelta):
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   619
        """
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   620
        Provides revisions that present an interest to be diffed against,
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   621
        grouped by level of easiness.
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   622
        """
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   623
        revlog = self.revlog
38114
f79ba1d1b4b1 revlog: in _getcandidaterevs, shorten revlog._generaldelta to gdelta
Paul Morelle <paul.morelle@octobus.net>
parents: 38102
diff changeset
   624
        gdelta = revlog._generaldelta
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   625
        curr = len(revlog)
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   626
        prev = curr - 1
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   627
        p1r, p2r = revlog.rev(p1), revlog.rev(p2)
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   628
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   629
        # should we try to build a delta?
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   630
        if prev != nullrev and revlog.storedeltachains:
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   631
            tested = set()
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   632
            # This condition is true most of the time when processing
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   633
            # changegroup data into a generaldelta repo. The only time it
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   634
            # isn't true is if this is the first revision in a delta chain
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   635
            # or if ``format.generaldelta=true`` disabled ``lazydeltabase``.
38114
f79ba1d1b4b1 revlog: in _getcandidaterevs, shorten revlog._generaldelta to gdelta
Paul Morelle <paul.morelle@octobus.net>
parents: 38102
diff changeset
   636
            if cachedelta and gdelta and revlog._lazydeltabase:
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   637
                # Assume what we received from the server is a good choice
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   638
                # build delta will reuse the cache
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   639
                yield (cachedelta[0],)
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   640
                tested.add(cachedelta[0])
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   641
38114
f79ba1d1b4b1 revlog: in _getcandidaterevs, shorten revlog._generaldelta to gdelta
Paul Morelle <paul.morelle@octobus.net>
parents: 38102
diff changeset
   642
            if gdelta:
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   643
                # exclude already lazy tested base if any
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   644
                parents = [p for p in (p1r, p2r)
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   645
                           if p != nullrev and p not in tested]
38102
9bf0bd4d7a2e revlog: suggest other parent when a parent was refused for a delta (issue5481)
Paul Morelle <paul.morelle@octobus.net>
parents: 38090
diff changeset
   646
38736
93777d16a25d aggressivemergedeltas: rename variable internally
Boris Feld <boris.feld@octobus.net>
parents: 38718
diff changeset
   647
                if not revlog._deltabothparents and len(parents) == 2:
38102
9bf0bd4d7a2e revlog: suggest other parent when a parent was refused for a delta (issue5481)
Paul Morelle <paul.morelle@octobus.net>
parents: 38090
diff changeset
   648
                    parents.sort()
9bf0bd4d7a2e revlog: suggest other parent when a parent was refused for a delta (issue5481)
Paul Morelle <paul.morelle@octobus.net>
parents: 38090
diff changeset
   649
                    # To minimize the chance of having to build a fulltext,
9bf0bd4d7a2e revlog: suggest other parent when a parent was refused for a delta (issue5481)
Paul Morelle <paul.morelle@octobus.net>
parents: 38090
diff changeset
   650
                    # pick first whichever parent is closest to us (max rev)
9bf0bd4d7a2e revlog: suggest other parent when a parent was refused for a delta (issue5481)
Paul Morelle <paul.morelle@octobus.net>
parents: 38090
diff changeset
   651
                    yield (parents[1],)
9bf0bd4d7a2e revlog: suggest other parent when a parent was refused for a delta (issue5481)
Paul Morelle <paul.morelle@octobus.net>
parents: 38090
diff changeset
   652
                    # then the other one (min rev) if the first did not fit
9bf0bd4d7a2e revlog: suggest other parent when a parent was refused for a delta (issue5481)
Paul Morelle <paul.morelle@octobus.net>
parents: 38090
diff changeset
   653
                    yield (parents[0],)
9bf0bd4d7a2e revlog: suggest other parent when a parent was refused for a delta (issue5481)
Paul Morelle <paul.morelle@octobus.net>
parents: 38090
diff changeset
   654
                    tested.update(parents)
9bf0bd4d7a2e revlog: suggest other parent when a parent was refused for a delta (issue5481)
Paul Morelle <paul.morelle@octobus.net>
parents: 38090
diff changeset
   655
                elif len(parents) > 0:
9bf0bd4d7a2e revlog: suggest other parent when a parent was refused for a delta (issue5481)
Paul Morelle <paul.morelle@octobus.net>
parents: 38090
diff changeset
   656
                    # Test all parents (1 or 2), and keep the best candidate
9bf0bd4d7a2e revlog: suggest other parent when a parent was refused for a delta (issue5481)
Paul Morelle <paul.morelle@octobus.net>
parents: 38090
diff changeset
   657
                    yield parents
9bf0bd4d7a2e revlog: suggest other parent when a parent was refused for a delta (issue5481)
Paul Morelle <paul.morelle@octobus.net>
parents: 38090
diff changeset
   658
                    tested.update(parents)
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   659
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   660
            if prev not in tested:
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   661
                # other approach failed try against prev to hopefully save us a
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   662
                # fulltext.
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   663
                yield (prev,)
38168
faa015417348 revlog: make getcandidaterevs more consistent about updating tested revs set
Paul Morelle <paul.morelle@octobus.net>
parents: 38117
diff changeset
   664
                tested.add(prev)
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   665
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   666
    def buildtext(self, revinfo, fh):
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   667
        """Builds a fulltext version of a revision
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   668
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   669
        revinfo: _revisioninfo instance that contains all needed info
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   670
        fh:      file handle to either the .i or the .d revlog file,
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   671
                 depending on whether it is inlined or not
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   672
        """
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   673
        btext = revinfo.btext
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   674
        if btext[0] is not None:
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   675
            return btext[0]
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   676
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   677
        revlog = self.revlog
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   678
        cachedelta = revinfo.cachedelta
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   679
        flags = revinfo.flags
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   680
        node = revinfo.node
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   681
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   682
        baserev = cachedelta[0]
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   683
        delta = cachedelta[1]
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   684
        # special case deltas which replace entire base; no need to decode
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   685
        # base revision. this neatly avoids censored bases, which throw when
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   686
        # they're decoded.
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   687
        hlen = struct.calcsize(">lll")
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   688
        if delta[:hlen] == mdiff.replacediffheader(revlog.rawsize(baserev),
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   689
                                                   len(delta) - hlen):
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   690
            btext[0] = delta[hlen:]
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   691
        else:
36748
369aadf7a326 revlog: resolve lfs rawtext to vanilla rawtext before applying delta
Jun Wu <quark@fb.com>
parents: 36744
diff changeset
   692
            # deltabase is rawtext before changed by flag processors, which is
369aadf7a326 revlog: resolve lfs rawtext to vanilla rawtext before applying delta
Jun Wu <quark@fb.com>
parents: 36744
diff changeset
   693
            # equivalent to non-raw text
369aadf7a326 revlog: resolve lfs rawtext to vanilla rawtext before applying delta
Jun Wu <quark@fb.com>
parents: 36744
diff changeset
   694
            basetext = revlog.revision(baserev, _df=fh, raw=False)
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   695
            btext[0] = mdiff.patch(basetext, delta)
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   696
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   697
        try:
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   698
            res = revlog._processflags(btext[0], flags, 'read', raw=True)
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   699
            btext[0], validatehash = res
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   700
            if validatehash:
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   701
                revlog.checkhash(btext[0], node, p1=revinfo.p1, p2=revinfo.p2)
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   702
            if flags & REVIDX_ISCENSORED:
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   703
                raise RevlogError(_('node %s is not censored') % node)
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   704
        except CensoredNodeError:
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   705
            # must pass the censored index flag to add censored revisions
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   706
            if not flags & REVIDX_ISCENSORED:
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   707
                raise
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   708
        return btext[0]
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   709
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   710
    def _builddeltadiff(self, base, revinfo, fh):
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   711
        revlog = self.revlog
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   712
        t = self.buildtext(revinfo, fh)
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   713
        if revlog.iscensored(base):
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   714
            # deltas based on a censored revision must replace the
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   715
            # full content in one patch, so delta works everywhere
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   716
            header = mdiff.replacediffheader(revlog.rawsize(base), len(t))
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   717
            delta = header + t
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   718
        else:
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   719
            ptext = revlog.revision(base, _df=fh, raw=True)
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   720
            delta = mdiff.textdiff(ptext, t)
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   721
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   722
        return delta
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   723
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   724
    def _builddeltainfo(self, revinfo, base, fh):
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   725
        # can we use the cached delta?
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   726
        if revinfo.cachedelta and revinfo.cachedelta[0] == base:
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   727
            delta = revinfo.cachedelta[1]
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   728
        else:
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   729
            delta = self._builddeltadiff(base, revinfo, fh)
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   730
        revlog = self.revlog
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   731
        header, data = revlog.compress(delta)
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   732
        deltalen = len(header) + len(data)
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   733
        chainbase = revlog.chainbase(base)
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   734
        offset = revlog.end(len(revlog) - 1)
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   735
        dist = deltalen + offset - revlog.start(chainbase)
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   736
        if revlog._generaldelta:
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   737
            deltabase = base
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   738
        else:
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   739
            deltabase = chainbase
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   740
        chainlen, compresseddeltalen = revlog._chaininfo(base)
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   741
        chainlen += 1
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   742
        compresseddeltalen += deltalen
39154
e0da43e2f71f revlog: compute snapshot depth on delta info
Boris Feld <boris.feld@octobus.net>
parents: 39152
diff changeset
   743
e0da43e2f71f revlog: compute snapshot depth on delta info
Boris Feld <boris.feld@octobus.net>
parents: 39152
diff changeset
   744
        revlog = self.revlog
e0da43e2f71f revlog: compute snapshot depth on delta info
Boris Feld <boris.feld@octobus.net>
parents: 39152
diff changeset
   745
        snapshotdepth = None
e0da43e2f71f revlog: compute snapshot depth on delta info
Boris Feld <boris.feld@octobus.net>
parents: 39152
diff changeset
   746
        if deltabase == nullrev:
e0da43e2f71f revlog: compute snapshot depth on delta info
Boris Feld <boris.feld@octobus.net>
parents: 39152
diff changeset
   747
            snapshotdepth = 0
e0da43e2f71f revlog: compute snapshot depth on delta info
Boris Feld <boris.feld@octobus.net>
parents: 39152
diff changeset
   748
        elif revlog._sparserevlog and revlog.issnapshot(deltabase):
e0da43e2f71f revlog: compute snapshot depth on delta info
Boris Feld <boris.feld@octobus.net>
parents: 39152
diff changeset
   749
            # A delta chain should always be one full snapshot,
e0da43e2f71f revlog: compute snapshot depth on delta info
Boris Feld <boris.feld@octobus.net>
parents: 39152
diff changeset
   750
            # zero or more semi-snapshots, and zero or more deltas
e0da43e2f71f revlog: compute snapshot depth on delta info
Boris Feld <boris.feld@octobus.net>
parents: 39152
diff changeset
   751
            p1, p2 = revlog.rev(revinfo.p1), revlog.rev(revinfo.p2)
e0da43e2f71f revlog: compute snapshot depth on delta info
Boris Feld <boris.feld@octobus.net>
parents: 39152
diff changeset
   752
            if deltabase not in (p1, p2) and revlog.issnapshot(deltabase):
e0da43e2f71f revlog: compute snapshot depth on delta info
Boris Feld <boris.feld@octobus.net>
parents: 39152
diff changeset
   753
                snapshotdepth = len(revlog._deltachain(deltabase)[0])
e0da43e2f71f revlog: compute snapshot depth on delta info
Boris Feld <boris.feld@octobus.net>
parents: 39152
diff changeset
   754
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   755
        return _deltainfo(dist, deltalen, (header, data), deltabase,
39154
e0da43e2f71f revlog: compute snapshot depth on delta info
Boris Feld <boris.feld@octobus.net>
parents: 39152
diff changeset
   756
                          chainbase, chainlen, compresseddeltalen,
e0da43e2f71f revlog: compute snapshot depth on delta info
Boris Feld <boris.feld@octobus.net>
parents: 39152
diff changeset
   757
                          snapshotdepth)
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   758
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   759
    def finddeltainfo(self, revinfo, fh):
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   760
        """Find an acceptable delta against a candidate revision
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   761
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   762
        revinfo: information about the revision (instance of _revisioninfo)
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   763
        fh:      file handle to either the .i or the .d revlog file,
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   764
                 depending on whether it is inlined or not
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   765
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   766
        Returns the first acceptable candidate revision, as ordered by
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   767
        _getcandidaterevs
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   768
        """
39085
dbb3e9e44fce revlog: do not search for delta for empty content
Boris Feld <boris.feld@octobus.net>
parents: 39084
diff changeset
   769
        if not revinfo.textlen:
dbb3e9e44fce revlog: do not search for delta for empty content
Boris Feld <boris.feld@octobus.net>
parents: 39084
diff changeset
   770
            return None # empty file do not need delta
dbb3e9e44fce revlog: do not search for delta for empty content
Boris Feld <boris.feld@octobus.net>
parents: 39084
diff changeset
   771
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   772
        cachedelta = revinfo.cachedelta
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   773
        p1 = revinfo.p1
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   774
        p2 = revinfo.p2
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   775
        revlog = self.revlog
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   776
39083
8f83a953dddf revlog: skip over empty revision when looking for delta base
Boris Feld <boris.feld@octobus.net>
parents: 39079
diff changeset
   777
        deltalength = self.revlog.length
8f83a953dddf revlog: skip over empty revision when looking for delta base
Boris Feld <boris.feld@octobus.net>
parents: 39079
diff changeset
   778
        deltaparent = self.revlog.deltaparent
8f83a953dddf revlog: skip over empty revision when looking for delta base
Boris Feld <boris.feld@octobus.net>
parents: 39079
diff changeset
   779
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   780
        deltainfo = None
39087
f90b333e79cb revlog: filter out "invalid" delta base candidates
Boris Feld <boris.feld@octobus.net>
parents: 39086
diff changeset
   781
        deltas_limit = revinfo.textlen * LIMIT_DELTA2TEXT
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   782
        for candidaterevs in self._getcandidaterevs(p1, p2, cachedelta):
39087
f90b333e79cb revlog: filter out "invalid" delta base candidates
Boris Feld <boris.feld@octobus.net>
parents: 39086
diff changeset
   783
            # filter out delta base that will never produce good delta
f90b333e79cb revlog: filter out "invalid" delta base candidates
Boris Feld <boris.feld@octobus.net>
parents: 39086
diff changeset
   784
            candidaterevs = [r for r in candidaterevs
f90b333e79cb revlog: filter out "invalid" delta base candidates
Boris Feld <boris.feld@octobus.net>
parents: 39086
diff changeset
   785
                             if self.revlog.length(r) <= deltas_limit]
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   786
            nominateddeltas = []
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   787
            for candidaterev in candidaterevs:
39083
8f83a953dddf revlog: skip over empty revision when looking for delta base
Boris Feld <boris.feld@octobus.net>
parents: 39079
diff changeset
   788
                # skip over empty delta (no need to include them in a chain)
8f83a953dddf revlog: skip over empty revision when looking for delta base
Boris Feld <boris.feld@octobus.net>
parents: 39079
diff changeset
   789
                while candidaterev != nullrev and not deltalength(candidaterev):
8f83a953dddf revlog: skip over empty revision when looking for delta base
Boris Feld <boris.feld@octobus.net>
parents: 39079
diff changeset
   790
                    candidaterev = deltaparent(candidaterev)
8f83a953dddf revlog: skip over empty revision when looking for delta base
Boris Feld <boris.feld@octobus.net>
parents: 39079
diff changeset
   791
                # no need to try a delta against nullid, this will be handled
8f83a953dddf revlog: skip over empty revision when looking for delta base
Boris Feld <boris.feld@octobus.net>
parents: 39079
diff changeset
   792
                # by fulltext later.
8f83a953dddf revlog: skip over empty revision when looking for delta base
Boris Feld <boris.feld@octobus.net>
parents: 39079
diff changeset
   793
                if candidaterev == nullrev:
8f83a953dddf revlog: skip over empty revision when looking for delta base
Boris Feld <boris.feld@octobus.net>
parents: 39079
diff changeset
   794
                    continue
36744
33275ab5e837 revlog: do not use delta for lfs revisions
Jun Wu <quark@fb.com>
parents: 36743
diff changeset
   795
                # no delta for rawtext-changing revs (see "candelta" for why)
33275ab5e837 revlog: do not use delta for lfs revisions
Jun Wu <quark@fb.com>
parents: 36743
diff changeset
   796
                if revlog.flags(candidaterev) & REVIDX_RAWTEXT_CHANGING_FLAGS:
33275ab5e837 revlog: do not use delta for lfs revisions
Jun Wu <quark@fb.com>
parents: 36743
diff changeset
   797
                    continue
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   798
                candidatedelta = self._builddeltainfo(revinfo, candidaterev, fh)
38117
69ec6f98cfa6 revlog: isgooddeltainfo takes the whole revinfo object
Paul Morelle <paul.morelle@octobus.net>
parents: 38116
diff changeset
   799
                if revlog._isgooddeltainfo(candidatedelta, revinfo):
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   800
                    nominateddeltas.append(candidatedelta)
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   801
            if nominateddeltas:
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   802
                deltainfo = min(nominateddeltas, key=lambda x: x.deltalen)
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   803
                break
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   804
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   805
        return deltainfo
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
   806
35641
c41bb85c09ff revlog: group revision info into a dedicated structure
Paul Morelle <paul.morelle@octobus.net>
parents: 35640
diff changeset
   807
@attr.s(slots=True, frozen=True)
c41bb85c09ff revlog: group revision info into a dedicated structure
Paul Morelle <paul.morelle@octobus.net>
parents: 35640
diff changeset
   808
class _revisioninfo(object):
c41bb85c09ff revlog: group revision info into a dedicated structure
Paul Morelle <paul.morelle@octobus.net>
parents: 35640
diff changeset
   809
    """Information about a revision that allows building its fulltext
c41bb85c09ff revlog: group revision info into a dedicated structure
Paul Morelle <paul.morelle@octobus.net>
parents: 35640
diff changeset
   810
    node:       expected hash of the revision
c41bb85c09ff revlog: group revision info into a dedicated structure
Paul Morelle <paul.morelle@octobus.net>
parents: 35640
diff changeset
   811
    p1, p2:     parent revs of the revision
c41bb85c09ff revlog: group revision info into a dedicated structure
Paul Morelle <paul.morelle@octobus.net>
parents: 35640
diff changeset
   812
    btext:      built text cache consisting of a one-element list
c41bb85c09ff revlog: group revision info into a dedicated structure
Paul Morelle <paul.morelle@octobus.net>
parents: 35640
diff changeset
   813
    cachedelta: (baserev, uncompressed_delta) or None
c41bb85c09ff revlog: group revision info into a dedicated structure
Paul Morelle <paul.morelle@octobus.net>
parents: 35640
diff changeset
   814
    flags:      flags associated to the revision storage
c41bb85c09ff revlog: group revision info into a dedicated structure
Paul Morelle <paul.morelle@octobus.net>
parents: 35640
diff changeset
   815
c41bb85c09ff revlog: group revision info into a dedicated structure
Paul Morelle <paul.morelle@octobus.net>
parents: 35640
diff changeset
   816
    One of btext[0] or cachedelta must be set.
c41bb85c09ff revlog: group revision info into a dedicated structure
Paul Morelle <paul.morelle@octobus.net>
parents: 35640
diff changeset
   817
    """
c41bb85c09ff revlog: group revision info into a dedicated structure
Paul Morelle <paul.morelle@octobus.net>
parents: 35640
diff changeset
   818
    node = attr.ib()
c41bb85c09ff revlog: group revision info into a dedicated structure
Paul Morelle <paul.morelle@octobus.net>
parents: 35640
diff changeset
   819
    p1 = attr.ib()
c41bb85c09ff revlog: group revision info into a dedicated structure
Paul Morelle <paul.morelle@octobus.net>
parents: 35640
diff changeset
   820
    p2 = attr.ib()
c41bb85c09ff revlog: group revision info into a dedicated structure
Paul Morelle <paul.morelle@octobus.net>
parents: 35640
diff changeset
   821
    btext = attr.ib()
35737
d99b07bc69fb revlog: refactor out _finddeltainfo from _addrevision
Paul Morelle <paul.morelle@octobus.net>
parents: 35641
diff changeset
   822
    textlen = attr.ib()
35641
c41bb85c09ff revlog: group revision info into a dedicated structure
Paul Morelle <paul.morelle@octobus.net>
parents: 35640
diff changeset
   823
    cachedelta = attr.ib()
c41bb85c09ff revlog: group revision info into a dedicated structure
Paul Morelle <paul.morelle@octobus.net>
parents: 35640
diff changeset
   824
    flags = attr.ib()
c41bb85c09ff revlog: group revision info into a dedicated structure
Paul Morelle <paul.morelle@octobus.net>
parents: 35640
diff changeset
   825
39231
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
   826
@interfaceutil.implementer(repository.irevisiondelta)
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
   827
@attr.s(slots=True, frozen=True)
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
   828
class revlogrevisiondelta(object):
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
   829
    node = attr.ib()
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
   830
    p1node = attr.ib()
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
   831
    p2node = attr.ib()
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
   832
    basenode = attr.ib()
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
   833
    linknode = attr.ib()
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
   834
    flags = attr.ib()
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
   835
    baserevisionsize = attr.ib()
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
   836
    revision = attr.ib()
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
   837
    delta = attr.ib()
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
   838
18585
b280f3bfc8a0 revlog: document v0 format
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 18090
diff changeset
   839
# index v0:
b280f3bfc8a0 revlog: document v0 format
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 18090
diff changeset
   840
#  4 bytes: offset
b280f3bfc8a0 revlog: document v0 format
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 18090
diff changeset
   841
#  4 bytes: compressed length
b280f3bfc8a0 revlog: document v0 format
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 18090
diff changeset
   842
#  4 bytes: base rev
b280f3bfc8a0 revlog: document v0 format
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 18090
diff changeset
   843
#  4 bytes: link rev
25891
c73fada78589 revlog: correct comment about size of v0 index format
Yuya Nishihara <yuya@tcha.org>
parents: 25822
diff changeset
   844
# 20 bytes: parent 1 nodeid
c73fada78589 revlog: correct comment about size of v0 index format
Yuya Nishihara <yuya@tcha.org>
parents: 25822
diff changeset
   845
# 20 bytes: parent 2 nodeid
c73fada78589 revlog: correct comment about size of v0 index format
Yuya Nishihara <yuya@tcha.org>
parents: 25822
diff changeset
   846
# 20 bytes: nodeid
33392
ac6446611ad2 revlog: use struct.Struct instances for slight performance wins
Alex Gaynor <agaynor@mozilla.com>
parents: 33391
diff changeset
   847
indexformatv0 = struct.Struct(">4l20s20s20s")
ac6446611ad2 revlog: use struct.Struct instances for slight performance wins
Alex Gaynor <agaynor@mozilla.com>
parents: 33391
diff changeset
   848
indexformatv0_pack = indexformatv0.pack
ac6446611ad2 revlog: use struct.Struct instances for slight performance wins
Alex Gaynor <agaynor@mozilla.com>
parents: 33391
diff changeset
   849
indexformatv0_unpack = indexformatv0.unpack
4918
e017d3a82e1d revlog: raise offset/type helpers to global scope
Matt Mackall <mpm@selenic.com>
parents: 4746
diff changeset
   850
38849
da5a666f0f78 pure: create type for revlog v0 index
Martin von Zweigbergk <martinvonz@google.com>
parents: 38844
diff changeset
   851
class revlogoldindex(list):
da5a666f0f78 pure: create type for revlog v0 index
Martin von Zweigbergk <martinvonz@google.com>
parents: 38844
diff changeset
   852
    def __getitem__(self, i):
38852
a3dacabd476b index: don't allow index[len(index)] to mean nullid
Martin von Zweigbergk <martinvonz@google.com>
parents: 38851
diff changeset
   853
        if i == -1:
38849
da5a666f0f78 pure: create type for revlog v0 index
Martin von Zweigbergk <martinvonz@google.com>
parents: 38844
diff changeset
   854
            return (0, 0, 0, -1, -1, -1, -1, nullid)
da5a666f0f78 pure: create type for revlog v0 index
Martin von Zweigbergk <martinvonz@google.com>
parents: 38844
diff changeset
   855
        return list.__getitem__(self, i)
da5a666f0f78 pure: create type for revlog v0 index
Martin von Zweigbergk <martinvonz@google.com>
parents: 38844
diff changeset
   856
39086
64ddad2f26bb revlog: use a symbolic constant for the deltas/text ration
Boris Feld <boris.feld@octobus.net>
parents: 39085
diff changeset
   857
# maximum <delta-chain-data>/<revision-text-length> ratio
64ddad2f26bb revlog: use a symbolic constant for the deltas/text ration
Boris Feld <boris.feld@octobus.net>
parents: 39085
diff changeset
   858
LIMIT_DELTA2TEXT = 2
64ddad2f26bb revlog: use a symbolic constant for the deltas/text ration
Boris Feld <boris.feld@octobus.net>
parents: 39085
diff changeset
   859
4972
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   860
class revlogoldio(object):
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   861
    def __init__(self):
33392
ac6446611ad2 revlog: use struct.Struct instances for slight performance wins
Alex Gaynor <agaynor@mozilla.com>
parents: 33391
diff changeset
   862
        self.size = indexformatv0.size
4972
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   863
13264
8439526fb407 revlog/parseindex: no need to pass the file around
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 13259
diff changeset
   864
    def parseindex(self, data, inline):
4977
6cb30bc4ca32 revlog: parse revlogv0 indexes into v1 internally
Matt Mackall <mpm@selenic.com>
parents: 4976
diff changeset
   865
        s = self.size
4972
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   866
        index = []
27637
b502138f5faa cleanup: remove superfluous space after space after equals (python)
timeless <timeless@mozdev.org>
parents: 27475
diff changeset
   867
        nodemap = {nullid: nullrev}
4973
a386a6e4fe46 revlog: simplify the v0 parser
Matt Mackall <mpm@selenic.com>
parents: 4972
diff changeset
   868
        n = off = 0
a386a6e4fe46 revlog: simplify the v0 parser
Matt Mackall <mpm@selenic.com>
parents: 4972
diff changeset
   869
        l = len(data)
a386a6e4fe46 revlog: simplify the v0 parser
Matt Mackall <mpm@selenic.com>
parents: 4972
diff changeset
   870
        while off + s <= l:
a386a6e4fe46 revlog: simplify the v0 parser
Matt Mackall <mpm@selenic.com>
parents: 4972
diff changeset
   871
            cur = data[off:off + s]
a386a6e4fe46 revlog: simplify the v0 parser
Matt Mackall <mpm@selenic.com>
parents: 4972
diff changeset
   872
            off += s
33392
ac6446611ad2 revlog: use struct.Struct instances for slight performance wins
Alex Gaynor <agaynor@mozilla.com>
parents: 33391
diff changeset
   873
            e = indexformatv0_unpack(cur)
4977
6cb30bc4ca32 revlog: parse revlogv0 indexes into v1 internally
Matt Mackall <mpm@selenic.com>
parents: 4976
diff changeset
   874
            # transform to revlogv1 format
6cb30bc4ca32 revlog: parse revlogv0 indexes into v1 internally
Matt Mackall <mpm@selenic.com>
parents: 4976
diff changeset
   875
            e2 = (offset_type(e[0], 0), e[1], -1, e[2], e[3],
5544
686899a7de5b revlog: make revlogv0 loading more robust against corruption
Matt Mackall <mpm@selenic.com>
parents: 5453
diff changeset
   876
                  nodemap.get(e[4], nullrev), nodemap.get(e[5], nullrev), e[6])
4977
6cb30bc4ca32 revlog: parse revlogv0 indexes into v1 internally
Matt Mackall <mpm@selenic.com>
parents: 4976
diff changeset
   877
            index.append(e2)
6cb30bc4ca32 revlog: parse revlogv0 indexes into v1 internally
Matt Mackall <mpm@selenic.com>
parents: 4976
diff changeset
   878
            nodemap[e[6]] = n
4973
a386a6e4fe46 revlog: simplify the v0 parser
Matt Mackall <mpm@selenic.com>
parents: 4972
diff changeset
   879
            n += 1
4972
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   880
38849
da5a666f0f78 pure: create type for revlog v0 index
Martin von Zweigbergk <martinvonz@google.com>
parents: 38844
diff changeset
   881
        return revlogoldindex(index), nodemap, None
4972
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   882
5338
f87685355c9c revlog: fix revlogio.packentry corner case
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5325
diff changeset
   883
    def packentry(self, entry, node, version, rev):
10395
ea52a2d4f42c revlog: don't silently discard revlog flags on revlogv0
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 10329
diff changeset
   884
        if gettype(entry[0]):
32393
d47b62368f3a revlog: remove some revlogNG terminology
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32392
diff changeset
   885
            raise RevlogError(_('index entry flags need revlog version 1'))
4986
58cc017ec7e0 revlog: abstract out index entry packing
Matt Mackall <mpm@selenic.com>
parents: 4985
diff changeset
   886
        e2 = (getoffset(entry[0]), entry[1], entry[3], entry[4],
58cc017ec7e0 revlog: abstract out index entry packing
Matt Mackall <mpm@selenic.com>
parents: 4985
diff changeset
   887
              node(entry[5]), node(entry[6]), entry[7])
33392
ac6446611ad2 revlog: use struct.Struct instances for slight performance wins
Alex Gaynor <agaynor@mozilla.com>
parents: 33391
diff changeset
   888
        return indexformatv0_pack(*e2)
4986
58cc017ec7e0 revlog: abstract out index entry packing
Matt Mackall <mpm@selenic.com>
parents: 4985
diff changeset
   889
4987
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
   890
# index ng:
11323
d65b74106113 revlog: fix inconsistent comment formatting
Martin Geisler <mg@aragost.com>
parents: 11155
diff changeset
   891
#  6 bytes: offset
d65b74106113 revlog: fix inconsistent comment formatting
Martin Geisler <mg@aragost.com>
parents: 11155
diff changeset
   892
#  2 bytes: flags
d65b74106113 revlog: fix inconsistent comment formatting
Martin Geisler <mg@aragost.com>
parents: 11155
diff changeset
   893
#  4 bytes: compressed length
d65b74106113 revlog: fix inconsistent comment formatting
Martin Geisler <mg@aragost.com>
parents: 11155
diff changeset
   894
#  4 bytes: uncompressed length
d65b74106113 revlog: fix inconsistent comment formatting
Martin Geisler <mg@aragost.com>
parents: 11155
diff changeset
   895
#  4 bytes: base rev
d65b74106113 revlog: fix inconsistent comment formatting
Martin Geisler <mg@aragost.com>
parents: 11155
diff changeset
   896
#  4 bytes: link rev
d65b74106113 revlog: fix inconsistent comment formatting
Martin Geisler <mg@aragost.com>
parents: 11155
diff changeset
   897
#  4 bytes: parent 1 rev
d65b74106113 revlog: fix inconsistent comment formatting
Martin Geisler <mg@aragost.com>
parents: 11155
diff changeset
   898
#  4 bytes: parent 2 rev
4987
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
   899
# 32 bytes: nodeid
33392
ac6446611ad2 revlog: use struct.Struct instances for slight performance wins
Alex Gaynor <agaynor@mozilla.com>
parents: 33391
diff changeset
   900
indexformatng = struct.Struct(">Qiiiiii20s12x")
ac6446611ad2 revlog: use struct.Struct instances for slight performance wins
Alex Gaynor <agaynor@mozilla.com>
parents: 33391
diff changeset
   901
indexformatng_pack = indexformatng.pack
ac6446611ad2 revlog: use struct.Struct instances for slight performance wins
Alex Gaynor <agaynor@mozilla.com>
parents: 33391
diff changeset
   902
versionformat = struct.Struct(">I")
ac6446611ad2 revlog: use struct.Struct instances for slight performance wins
Alex Gaynor <agaynor@mozilla.com>
parents: 33391
diff changeset
   903
versionformat_pack = versionformat.pack
ac6446611ad2 revlog: use struct.Struct instances for slight performance wins
Alex Gaynor <agaynor@mozilla.com>
parents: 33391
diff changeset
   904
versionformat_unpack = versionformat.unpack
4987
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
   905
25410
eee88912db0a revlog: raise an exception earlier if an entry is too large (issue4675)
Jordi Gutiérrez Hermoso <jordigh@octave.org>
parents: 25361
diff changeset
   906
# corresponds to uncompressed length of indexformatng (2 gigs, 4-byte
eee88912db0a revlog: raise an exception earlier if an entry is too large (issue4675)
Jordi Gutiérrez Hermoso <jordigh@octave.org>
parents: 25361
diff changeset
   907
# signed integer)
eee88912db0a revlog: raise an exception earlier if an entry is too large (issue4675)
Jordi Gutiérrez Hermoso <jordigh@octave.org>
parents: 25361
diff changeset
   908
_maxentrysize = 0x7fffffff
eee88912db0a revlog: raise an exception earlier if an entry is too large (issue4675)
Jordi Gutiérrez Hermoso <jordigh@octave.org>
parents: 25361
diff changeset
   909
4972
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   910
class revlogio(object):
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   911
    def __init__(self):
33392
ac6446611ad2 revlog: use struct.Struct instances for slight performance wins
Alex Gaynor <agaynor@mozilla.com>
parents: 33391
diff changeset
   912
        self.size = indexformatng.size
4972
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   913
13264
8439526fb407 revlog/parseindex: no need to pass the file around
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 13259
diff changeset
   914
    def parseindex(self, data, inline):
7109
528b7fc1216c use the new parseindex implementation C in parsers
Bernhard Leiner <bleiner@gmail.com>
parents: 7089
diff changeset
   915
        # call the C implementation to parse the index data
13254
5ef5eb1f3515 revlog: only build the nodemap on demand
Matt Mackall <mpm@selenic.com>
parents: 13253
diff changeset
   916
        index, cache = parsers.parse_index2(data, inline)
16414
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16375
diff changeset
   917
        return index, getattr(index, 'nodemap', None), cache
4972
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   918
5338
f87685355c9c revlog: fix revlogio.packentry corner case
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5325
diff changeset
   919
    def packentry(self, entry, node, version, rev):
33392
ac6446611ad2 revlog: use struct.Struct instances for slight performance wins
Alex Gaynor <agaynor@mozilla.com>
parents: 33391
diff changeset
   920
        p = indexformatng_pack(*entry)
5338
f87685355c9c revlog: fix revlogio.packentry corner case
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5325
diff changeset
   921
        if rev == 0:
33392
ac6446611ad2 revlog: use struct.Struct instances for slight performance wins
Alex Gaynor <agaynor@mozilla.com>
parents: 33391
diff changeset
   922
            p = versionformat_pack(version) + p[4:]
4986
58cc017ec7e0 revlog: abstract out index entry packing
Matt Mackall <mpm@selenic.com>
parents: 4985
diff changeset
   923
        return p
58cc017ec7e0 revlog: abstract out index entry packing
Matt Mackall <mpm@selenic.com>
parents: 4985
diff changeset
   924
1559
59b3639df0a9 Convert all classes to new-style classes by deriving them from object.
Eric Hopper <hopper@omnifarious.org>
parents: 1551
diff changeset
   925
class revlog(object):
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   926
    """
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   927
    the underlying revision storage object
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   928
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   929
    A revlog consists of two parts, an index and the revision data.
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   930
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   931
    The index is a file with a fixed record size containing
6912
b92baef99ebf Fixed docstring typos
Martin Geisler <mg@daimi.au.dk>
parents: 6891
diff changeset
   932
    information on each revision, including its nodeid (hash), the
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   933
    nodeids of its parents, the position and offset of its data within
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   934
    the data file, and the revision it's based on. Finally, each entry
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   935
    contains a linkrev entry that can serve as a pointer to external
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   936
    data.
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   937
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   938
    The revision data itself is a linear collection of data chunks.
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   939
    Each chunk represents a revision and is usually represented as a
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   940
    delta against the previous chunk. To bound lookup time, runs of
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   941
    deltas are limited to about 2 times the length of the original
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   942
    version data. This makes retrieval of a version proportional to
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   943
    its size, or O(1) relative to the number of revisions.
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   944
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   945
    Both pieces of the revlog are written to in an append-only
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   946
    fashion, which means we never need to rewrite a file to insert or
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   947
    remove data, and can use some simple techniques to avoid the need
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   948
    for locking while reading.
29997
b5e5ddf48bd2 revlog: specify checkambig at writing to avoid file stat ambiguity
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29830
diff changeset
   949
b5e5ddf48bd2 revlog: specify checkambig at writing to avoid file stat ambiguity
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29830
diff changeset
   950
    If checkambig, indexfile is opened with checkambig=True at
b5e5ddf48bd2 revlog: specify checkambig at writing to avoid file stat ambiguity
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29830
diff changeset
   951
    writing, to avoid file stat ambiguity.
34296
3c9691728237 revlog: add option to mmap revlog index
Mark Thomas <mbthomas@fb.com>
parents: 34291
diff changeset
   952
3c9691728237 revlog: add option to mmap revlog index
Mark Thomas <mbthomas@fb.com>
parents: 34291
diff changeset
   953
    If mmaplargeindex is True, and an mmapindexthreshold is set, the
3c9691728237 revlog: add option to mmap revlog index
Mark Thomas <mbthomas@fb.com>
parents: 34291
diff changeset
   954
    index will be mmapped rather than read if it is larger than the
3c9691728237 revlog: add option to mmap revlog index
Mark Thomas <mbthomas@fb.com>
parents: 34291
diff changeset
   955
    configured threshold.
37443
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
   956
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
   957
    If censorable is True, the revlog can have censored revisions.
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   958
    """
34296
3c9691728237 revlog: add option to mmap revlog index
Mark Thomas <mbthomas@fb.com>
parents: 34291
diff changeset
   959
    def __init__(self, opener, indexfile, datafile=None, checkambig=False,
37443
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
   960
                 mmaplargeindex=False, censorable=False):
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   961
        """
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   962
        create a revlog object
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   963
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   964
        opener is a function that abstracts the file opening operation
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   965
        and can be used to implement COW semantics or the like.
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   966
        """
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
   967
        self.indexfile = indexfile
32307
3caec778774b changelog: make sure datafile is 00changelog.d (API)
Jun Wu <quark@fb.com>
parents: 32291
diff changeset
   968
        self.datafile = datafile or (indexfile[:-2] + ".d")
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
   969
        self.opener = opener
29997
b5e5ddf48bd2 revlog: specify checkambig at writing to avoid file stat ambiguity
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29830
diff changeset
   970
        #  When True, indexfile is opened with checkambig=True at writing, to
b5e5ddf48bd2 revlog: specify checkambig at writing to avoid file stat ambiguity
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29830
diff changeset
   971
        #  avoid file stat ambiguity.
b5e5ddf48bd2 revlog: specify checkambig at writing to avoid file stat ambiguity
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29830
diff changeset
   972
        self._checkambig = checkambig
37443
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
   973
        self._censorable = censorable
27070
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
   974
        # 3-tuple of (node, rev, text) for a raw revision.
4984
b4066fcbd6ba revlog: mark cache private
Matt Mackall <mpm@selenic.com>
parents: 4983
diff changeset
   975
        self._cache = None
29830
92ac2baaea86 revlog: use an LRU cache for delta chain bases
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29829
diff changeset
   976
        # Maps rev to chain base rev.
92ac2baaea86 revlog: use an LRU cache for delta chain bases
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29829
diff changeset
   977
        self._chainbasecache = util.lrucachedict(100)
27070
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
   978
        # 2-tuple of (offset, data) of raw data from the revlog at an offset.
8316
d593922cf480 revlog: clean up the chunk caching code
Matt Mackall <mpm@selenic.com>
parents: 8315
diff changeset
   979
        self._chunkcache = (0, '')
27070
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
   980
        # How much data to read and cache into the raw revlog data cache.
20180
969148b49fc6 revlog: allow tuning of the chunk cache size (via format.chunkcachesize)
Brodie Rao <brodie@sf.io>
parents: 20179
diff changeset
   981
        self._chunkcachesize = 65536
23255
76effa770ff9 revlog: add config variable for limiting delta-chain length
Mateusz Kwapich <mitrandir@fb.com>
parents: 23254
diff changeset
   982
        self._maxchainlen = None
38736
93777d16a25d aggressivemergedeltas: rename variable internally
Boris Feld <boris.feld@octobus.net>
parents: 38718
diff changeset
   983
        self._deltabothparents = True
4985
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
   984
        self.index = []
27070
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
   985
        # Mapping of partial identifiers to full nodes.
13258
c2661863f16f revlog: introduce a cache for partial lookups
Matt Mackall <mpm@selenic.com>
parents: 13254
diff changeset
   986
        self._pcache = {}
27070
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
   987
        # Mapping of revision integer to full node.
13275
68da048b4c88 revlog: incrementally build node cache with linear searches
Matt Mackall <mpm@selenic.com>
parents: 13268
diff changeset
   988
        self._nodecache = {nullid: nullrev}
68da048b4c88 revlog: incrementally build node cache with linear searches
Matt Mackall <mpm@selenic.com>
parents: 13268
diff changeset
   989
        self._nodepos = None
30818
4c0a5a256ae8 localrepo: experimental support for non-zlib revlog compression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30817
diff changeset
   990
        self._compengine = 'zlib'
33207
895ecec31c70 revlog: add an experimental option to mitigated delta issues (issue5480)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 33171
diff changeset
   991
        self._maxdeltachainspan = -1
34824
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
   992
        self._withsparseread = False
38717
aa21a9ad46ea sparse-revlog: new requirement enabled with format.sparse-revlog
Paul Morelle <paul.morelle@octobus.net>
parents: 38716
diff changeset
   993
        self._sparserevlog = False
38632
eb8504715071 sparse-read: target density of 50% instead of 25%
Paul Morelle <paul.morelle@octobus.net>
parents: 38602
diff changeset
   994
        self._srdensitythreshold = 0.50
34881
8c9b08a0c48c sparse-read: skip gaps too small to be worth splitting
Paul Morelle <paul.morelle@octobus.net>
parents: 34880
diff changeset
   995
        self._srmingapsize = 262144
4985
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
   996
34296
3c9691728237 revlog: add option to mmap revlog index
Mark Thomas <mbthomas@fb.com>
parents: 34291
diff changeset
   997
        mmapindexthreshold = None
4985
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
   998
        v = REVLOG_DEFAULT_VERSION
14960
497819817307 revlog: use getattr instead of hasattr
Augie Fackler <durin42@gmail.com>
parents: 14549
diff changeset
   999
        opts = getattr(opener, 'options', None)
497819817307 revlog: use getattr instead of hasattr
Augie Fackler <durin42@gmail.com>
parents: 14549
diff changeset
  1000
        if opts is not None:
32697
19b9fc40cc51 revlog: skeleton support for version 2 revlogs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32684
diff changeset
  1001
            if 'revlogv2' in opts:
19b9fc40cc51 revlog: skeleton support for version 2 revlogs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32684
diff changeset
  1002
                # version 2 revlogs always use generaldelta.
19b9fc40cc51 revlog: skeleton support for version 2 revlogs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32684
diff changeset
  1003
                v = REVLOGV2 | FLAG_GENERALDELTA | FLAG_INLINE_DATA
19b9fc40cc51 revlog: skeleton support for version 2 revlogs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32684
diff changeset
  1004
            elif 'revlogv1' in opts:
14960
497819817307 revlog: use getattr instead of hasattr
Augie Fackler <durin42@gmail.com>
parents: 14549
diff changeset
  1005
                if 'generaldelta' in opts:
32315
67026d65a4fc revlog: rename constants (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32307
diff changeset
  1006
                    v |= FLAG_GENERALDELTA
14333
31a5973fcf96 revlog: get rid of defversion
Sune Foldager <cryo@cyanite.org>
parents: 14325
diff changeset
  1007
            else:
31a5973fcf96 revlog: get rid of defversion
Sune Foldager <cryo@cyanite.org>
parents: 14325
diff changeset
  1008
                v = 0
20180
969148b49fc6 revlog: allow tuning of the chunk cache size (via format.chunkcachesize)
Brodie Rao <brodie@sf.io>
parents: 20179
diff changeset
  1009
            if 'chunkcachesize' in opts:
969148b49fc6 revlog: allow tuning of the chunk cache size (via format.chunkcachesize)
Brodie Rao <brodie@sf.io>
parents: 20179
diff changeset
  1010
                self._chunkcachesize = opts['chunkcachesize']
23255
76effa770ff9 revlog: add config variable for limiting delta-chain length
Mateusz Kwapich <mitrandir@fb.com>
parents: 23254
diff changeset
  1011
            if 'maxchainlen' in opts:
76effa770ff9 revlog: add config variable for limiting delta-chain length
Mateusz Kwapich <mitrandir@fb.com>
parents: 23254
diff changeset
  1012
                self._maxchainlen = opts['maxchainlen']
38736
93777d16a25d aggressivemergedeltas: rename variable internally
Boris Feld <boris.feld@octobus.net>
parents: 38718
diff changeset
  1013
            if 'deltabothparents' in opts:
93777d16a25d aggressivemergedeltas: rename variable internally
Boris Feld <boris.feld@octobus.net>
parents: 38718
diff changeset
  1014
                self._deltabothparents = opts['deltabothparents']
26907
dfab6edb98e3 format: introduce 'format.usegeneraldelta`
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26705
diff changeset
  1015
            self._lazydeltabase = bool(opts.get('lazydeltabase', False))
30818
4c0a5a256ae8 localrepo: experimental support for non-zlib revlog compression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30817
diff changeset
  1016
            if 'compengine' in opts:
4c0a5a256ae8 localrepo: experimental support for non-zlib revlog compression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30817
diff changeset
  1017
                self._compengine = opts['compengine']
33207
895ecec31c70 revlog: add an experimental option to mitigated delta issues (issue5480)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 33171
diff changeset
  1018
            if 'maxdeltachainspan' in opts:
895ecec31c70 revlog: add an experimental option to mitigated delta issues (issue5480)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 33171
diff changeset
  1019
                self._maxdeltachainspan = opts['maxdeltachainspan']
34296
3c9691728237 revlog: add option to mmap revlog index
Mark Thomas <mbthomas@fb.com>
parents: 34291
diff changeset
  1020
            if mmaplargeindex and 'mmapindexthreshold' in opts:
3c9691728237 revlog: add option to mmap revlog index
Mark Thomas <mbthomas@fb.com>
parents: 34291
diff changeset
  1021
                mmapindexthreshold = opts['mmapindexthreshold']
38717
aa21a9ad46ea sparse-revlog: new requirement enabled with format.sparse-revlog
Paul Morelle <paul.morelle@octobus.net>
parents: 38716
diff changeset
  1022
            self._sparserevlog = bool(opts.get('sparse-revlog', False))
aa21a9ad46ea sparse-revlog: new requirement enabled with format.sparse-revlog
Paul Morelle <paul.morelle@octobus.net>
parents: 38716
diff changeset
  1023
            withsparseread = bool(opts.get('with-sparse-read', False))
aa21a9ad46ea sparse-revlog: new requirement enabled with format.sparse-revlog
Paul Morelle <paul.morelle@octobus.net>
parents: 38716
diff changeset
  1024
            # sparse-revlog forces sparse-read
aa21a9ad46ea sparse-revlog: new requirement enabled with format.sparse-revlog
Paul Morelle <paul.morelle@octobus.net>
parents: 38716
diff changeset
  1025
            self._withsparseread = self._sparserevlog or withsparseread
34824
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
  1026
            if 'sparse-read-density-threshold' in opts:
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
  1027
                self._srdensitythreshold = opts['sparse-read-density-threshold']
34881
8c9b08a0c48c sparse-read: skip gaps too small to be worth splitting
Paul Morelle <paul.morelle@octobus.net>
parents: 34880
diff changeset
  1028
            if 'sparse-read-min-gap-size' in opts:
8c9b08a0c48c sparse-read: skip gaps too small to be worth splitting
Paul Morelle <paul.morelle@octobus.net>
parents: 34880
diff changeset
  1029
                self._srmingapsize = opts['sparse-read-min-gap-size']
20180
969148b49fc6 revlog: allow tuning of the chunk cache size (via format.chunkcachesize)
Brodie Rao <brodie@sf.io>
parents: 20179
diff changeset
  1030
969148b49fc6 revlog: allow tuning of the chunk cache size (via format.chunkcachesize)
Brodie Rao <brodie@sf.io>
parents: 20179
diff changeset
  1031
        if self._chunkcachesize <= 0:
969148b49fc6 revlog: allow tuning of the chunk cache size (via format.chunkcachesize)
Brodie Rao <brodie@sf.io>
parents: 20179
diff changeset
  1032
            raise RevlogError(_('revlog chunk cache size %r is not greater '
969148b49fc6 revlog: allow tuning of the chunk cache size (via format.chunkcachesize)
Brodie Rao <brodie@sf.io>
parents: 20179
diff changeset
  1033
                                'than 0') % self._chunkcachesize)
969148b49fc6 revlog: allow tuning of the chunk cache size (via format.chunkcachesize)
Brodie Rao <brodie@sf.io>
parents: 20179
diff changeset
  1034
        elif self._chunkcachesize & (self._chunkcachesize - 1):
969148b49fc6 revlog: allow tuning of the chunk cache size (via format.chunkcachesize)
Brodie Rao <brodie@sf.io>
parents: 20179
diff changeset
  1035
            raise RevlogError(_('revlog chunk cache size %r is not a power '
969148b49fc6 revlog: allow tuning of the chunk cache size (via format.chunkcachesize)
Brodie Rao <brodie@sf.io>
parents: 20179
diff changeset
  1036
                                'of 2') % self._chunkcachesize)
11928
b69899dbad40 revlog: parentdelta flags for revlog index
Pradeepkumar Gayam <in3xes@gmail.com>
parents: 11759
diff changeset
  1037
26241
eb97d49768cc revlog: rename generic "i" variable to "indexdata"
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26118
diff changeset
  1038
        indexdata = ''
14334
85c82ebc96a3 changelog: don't use generaldelta
Sune Foldager <cryo@cyanite.org>
parents: 14333
diff changeset
  1039
        self._initempty = True
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
  1040
        try:
35969
1f2b8a644dd4 revlog: use context manager for index file life time in __init__
Boris Feld <boris.feld@octobus.net>
parents: 35968
diff changeset
  1041
            with self._indexfp() as f:
1f2b8a644dd4 revlog: use context manager for index file life time in __init__
Boris Feld <boris.feld@octobus.net>
parents: 35968
diff changeset
  1042
                if (mmapindexthreshold is not None and
1f2b8a644dd4 revlog: use context manager for index file life time in __init__
Boris Feld <boris.feld@octobus.net>
parents: 35968
diff changeset
  1043
                        self.opener.fstat(f).st_size >= mmapindexthreshold):
1f2b8a644dd4 revlog: use context manager for index file life time in __init__
Boris Feld <boris.feld@octobus.net>
parents: 35968
diff changeset
  1044
                    indexdata = util.buffer(util.mmapread(f))
1f2b8a644dd4 revlog: use context manager for index file life time in __init__
Boris Feld <boris.feld@octobus.net>
parents: 35968
diff changeset
  1045
                else:
1f2b8a644dd4 revlog: use context manager for index file life time in __init__
Boris Feld <boris.feld@octobus.net>
parents: 35968
diff changeset
  1046
                    indexdata = f.read()
26241
eb97d49768cc revlog: rename generic "i" variable to "indexdata"
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26118
diff changeset
  1047
            if len(indexdata) > 0:
33392
ac6446611ad2 revlog: use struct.Struct instances for slight performance wins
Alex Gaynor <agaynor@mozilla.com>
parents: 33391
diff changeset
  1048
                v = versionformat_unpack(indexdata[:4])[0]
14334
85c82ebc96a3 changelog: don't use generaldelta
Sune Foldager <cryo@cyanite.org>
parents: 14333
diff changeset
  1049
                self._initempty = False
25660
328739ea70c3 global: mass rewrite to use modern exception syntax
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25459
diff changeset
  1050
        except IOError as inst:
1322
b3d44e9b3092 Make revlog constructor more discerning in its treatment of errors.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1232
diff changeset
  1051
            if inst.errno != errno.ENOENT:
b3d44e9b3092 Make revlog constructor more discerning in its treatment of errors.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1232
diff changeset
  1052
                raise
4985
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
  1053
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
  1054
        self.version = v
32315
67026d65a4fc revlog: rename constants (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32307
diff changeset
  1055
        self._inline = v & FLAG_INLINE_DATA
67026d65a4fc revlog: rename constants (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32307
diff changeset
  1056
        self._generaldelta = v & FLAG_GENERALDELTA
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
  1057
        flags = v & ~0xFFFF
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
  1058
        fmt = v & 0xFFFF
32392
36d3559c69a6 revlog: tweak wording and logic for flags validation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32372
diff changeset
  1059
        if fmt == REVLOGV0:
36d3559c69a6 revlog: tweak wording and logic for flags validation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32372
diff changeset
  1060
            if flags:
36d3559c69a6 revlog: tweak wording and logic for flags validation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32372
diff changeset
  1061
                raise RevlogError(_('unknown flags (%#04x) in version %d '
36d3559c69a6 revlog: tweak wording and logic for flags validation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32372
diff changeset
  1062
                                    'revlog %s') %
36d3559c69a6 revlog: tweak wording and logic for flags validation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32372
diff changeset
  1063
                                  (flags >> 16, fmt, self.indexfile))
36d3559c69a6 revlog: tweak wording and logic for flags validation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32372
diff changeset
  1064
        elif fmt == REVLOGV1:
36d3559c69a6 revlog: tweak wording and logic for flags validation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32372
diff changeset
  1065
            if flags & ~REVLOGV1_FLAGS:
36d3559c69a6 revlog: tweak wording and logic for flags validation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32372
diff changeset
  1066
                raise RevlogError(_('unknown flags (%#04x) in version %d '
36d3559c69a6 revlog: tweak wording and logic for flags validation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32372
diff changeset
  1067
                                    'revlog %s') %
36d3559c69a6 revlog: tweak wording and logic for flags validation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32372
diff changeset
  1068
                                  (flags >> 16, fmt, self.indexfile))
32697
19b9fc40cc51 revlog: skeleton support for version 2 revlogs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32684
diff changeset
  1069
        elif fmt == REVLOGV2:
19b9fc40cc51 revlog: skeleton support for version 2 revlogs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32684
diff changeset
  1070
            if flags & ~REVLOGV2_FLAGS:
19b9fc40cc51 revlog: skeleton support for version 2 revlogs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32684
diff changeset
  1071
                raise RevlogError(_('unknown flags (%#04x) in version %d '
19b9fc40cc51 revlog: skeleton support for version 2 revlogs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32684
diff changeset
  1072
                                    'revlog %s') %
19b9fc40cc51 revlog: skeleton support for version 2 revlogs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32684
diff changeset
  1073
                                  (flags >> 16, fmt, self.indexfile))
32392
36d3559c69a6 revlog: tweak wording and logic for flags validation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32372
diff changeset
  1074
        else:
36d3559c69a6 revlog: tweak wording and logic for flags validation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32372
diff changeset
  1075
            raise RevlogError(_('unknown version (%d) in revlog %s') %
36d3559c69a6 revlog: tweak wording and logic for flags validation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32372
diff changeset
  1076
                              (fmt, self.indexfile))
4985
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
  1077
30210
5e4f16874a9f revlog: make 'storedeltachains' a "public" attribute
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 30154
diff changeset
  1078
        self.storedeltachains = True
30154
5e72129d75ed revlog: add instance variable controlling delta chain use
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30012
diff changeset
  1079
4972
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
  1080
        self._io = revlogio()
4971
3e6dae278c99 revlog: regroup parsing code
Matt Mackall <mpm@selenic.com>
parents: 4920
diff changeset
  1081
        if self.version == REVLOGV0:
4972
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
  1082
            self._io = revlogoldio()
13265
04b302ce2781 revlog: always add the magic nullid/nullrev entry in parseindex
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 13264
diff changeset
  1083
        try:
26241
eb97d49768cc revlog: rename generic "i" variable to "indexdata"
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26118
diff changeset
  1084
            d = self._io.parseindex(indexdata, self._inline)
13265
04b302ce2781 revlog: always add the magic nullid/nullrev entry in parseindex
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 13264
diff changeset
  1085
        except (ValueError, IndexError):
04b302ce2781 revlog: always add the magic nullid/nullrev entry in parseindex
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 13264
diff changeset
  1086
            raise RevlogError(_("index %s is corrupted") % (self.indexfile))
13268
fff12b3d953a revlog: explicit test and explicit variable names
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 13267
diff changeset
  1087
        self.index, nodemap, self._chunkcache = d
fff12b3d953a revlog: explicit test and explicit variable names
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 13267
diff changeset
  1088
        if nodemap is not None:
13275
68da048b4c88 revlog: incrementally build node cache with linear searches
Matt Mackall <mpm@selenic.com>
parents: 13268
diff changeset
  1089
            self.nodemap = self._nodecache = nodemap
13265
04b302ce2781 revlog: always add the magic nullid/nullrev entry in parseindex
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 13264
diff changeset
  1090
        if not self._chunkcache:
04b302ce2781 revlog: always add the magic nullid/nullrev entry in parseindex
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 13264
diff changeset
  1091
            self._chunkclear()
23306
f7a42f8e82bd revlog: cache chain info after calculating it for a rev (issue4452)
Siddharth Agarwal <sid0@fb.com>
parents: 23288
diff changeset
  1092
        # revnum -> (chain-length, sum-delta-length)
f7a42f8e82bd revlog: cache chain info after calculating it for a rev (issue4452)
Siddharth Agarwal <sid0@fb.com>
parents: 23288
diff changeset
  1093
        self._chaininfocache = {}
30817
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
  1094
        # revlog header -> revlog compressor
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
  1095
        self._decompressors = {}
116
e484cd5ec282 Only use lazy indexing for big indices and avoid the overhead of the
mpm@selenic.com
parents: 115
diff changeset
  1096
30795
78ac56aebab6 revlog: use compression engine API for compression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30793
diff changeset
  1097
    @util.propertycache
78ac56aebab6 revlog: use compression engine API for compression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30793
diff changeset
  1098
    def _compressor(self):
30818
4c0a5a256ae8 localrepo: experimental support for non-zlib revlog compression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30817
diff changeset
  1099
        return util.compengines[self._compengine].revlogcompressor()
30795
78ac56aebab6 revlog: use compression engine API for compression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30793
diff changeset
  1100
35968
91b3b11565e5 revlog: move index file opening in a method
Boris Feld <boris.feld@octobus.net>
parents: 35967
diff changeset
  1101
    def _indexfp(self, mode='r'):
91b3b11565e5 revlog: move index file opening in a method
Boris Feld <boris.feld@octobus.net>
parents: 35967
diff changeset
  1102
        """file object for the revlog's index file"""
91b3b11565e5 revlog: move index file opening in a method
Boris Feld <boris.feld@octobus.net>
parents: 35967
diff changeset
  1103
        args = {r'mode': mode}
91b3b11565e5 revlog: move index file opening in a method
Boris Feld <boris.feld@octobus.net>
parents: 35967
diff changeset
  1104
        if mode != 'r':
91b3b11565e5 revlog: move index file opening in a method
Boris Feld <boris.feld@octobus.net>
parents: 35967
diff changeset
  1105
            args[r'checkambig'] = self._checkambig
91b3b11565e5 revlog: move index file opening in a method
Boris Feld <boris.feld@octobus.net>
parents: 35967
diff changeset
  1106
        if mode == 'w':
91b3b11565e5 revlog: move index file opening in a method
Boris Feld <boris.feld@octobus.net>
parents: 35967
diff changeset
  1107
            args[r'atomictemp'] = True
91b3b11565e5 revlog: move index file opening in a method
Boris Feld <boris.feld@octobus.net>
parents: 35967
diff changeset
  1108
        return self.opener(self.indexfile, **args)
91b3b11565e5 revlog: move index file opening in a method
Boris Feld <boris.feld@octobus.net>
parents: 35967
diff changeset
  1109
35967
61326dd7cb8d revlog: move datafile opening in a method
Boris Feld <boris.feld@octobus.net>
parents: 35845
diff changeset
  1110
    def _datafp(self, mode='r'):
61326dd7cb8d revlog: move datafile opening in a method
Boris Feld <boris.feld@octobus.net>
parents: 35845
diff changeset
  1111
        """file object for the revlog's data file"""
61326dd7cb8d revlog: move datafile opening in a method
Boris Feld <boris.feld@octobus.net>
parents: 35845
diff changeset
  1112
        return self.opener(self.datafile, mode=mode)
61326dd7cb8d revlog: move datafile opening in a method
Boris Feld <boris.feld@octobus.net>
parents: 35845
diff changeset
  1113
35973
4d66993bdcff revlog: add a _datareadfp context manager for data access needs
Boris Feld <boris.feld@octobus.net>
parents: 35972
diff changeset
  1114
    @contextlib.contextmanager
4d66993bdcff revlog: add a _datareadfp context manager for data access needs
Boris Feld <boris.feld@octobus.net>
parents: 35972
diff changeset
  1115
    def _datareadfp(self, existingfp=None):
4d66993bdcff revlog: add a _datareadfp context manager for data access needs
Boris Feld <boris.feld@octobus.net>
parents: 35972
diff changeset
  1116
        """file object suitable to read data"""
4d66993bdcff revlog: add a _datareadfp context manager for data access needs
Boris Feld <boris.feld@octobus.net>
parents: 35972
diff changeset
  1117
        if existingfp is not None:
4d66993bdcff revlog: add a _datareadfp context manager for data access needs
Boris Feld <boris.feld@octobus.net>
parents: 35972
diff changeset
  1118
            yield existingfp
4d66993bdcff revlog: add a _datareadfp context manager for data access needs
Boris Feld <boris.feld@octobus.net>
parents: 35972
diff changeset
  1119
        else:
4d66993bdcff revlog: add a _datareadfp context manager for data access needs
Boris Feld <boris.feld@octobus.net>
parents: 35972
diff changeset
  1120
            if self._inline:
4d66993bdcff revlog: add a _datareadfp context manager for data access needs
Boris Feld <boris.feld@octobus.net>
parents: 35972
diff changeset
  1121
                func = self._indexfp
4d66993bdcff revlog: add a _datareadfp context manager for data access needs
Boris Feld <boris.feld@octobus.net>
parents: 35972
diff changeset
  1122
            else:
4d66993bdcff revlog: add a _datareadfp context manager for data access needs
Boris Feld <boris.feld@octobus.net>
parents: 35972
diff changeset
  1123
                func = self._datafp
4d66993bdcff revlog: add a _datareadfp context manager for data access needs
Boris Feld <boris.feld@octobus.net>
parents: 35972
diff changeset
  1124
            with func() as fp:
4d66993bdcff revlog: add a _datareadfp context manager for data access needs
Boris Feld <boris.feld@octobus.net>
parents: 35972
diff changeset
  1125
                yield fp
4d66993bdcff revlog: add a _datareadfp context manager for data access needs
Boris Feld <boris.feld@octobus.net>
parents: 35972
diff changeset
  1126
4980
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
  1127
    def tip(self):
38851
781b2720d2ac index: don't include nullid in len()
Martin von Zweigbergk <martinvonz@google.com>
parents: 38850
diff changeset
  1128
        return self.node(len(self.index) - 1)
24030
828dc8db5515 revlog: add __contains__ for fast membership test
Yuya Nishihara <yuya@tcha.org>
parents: 23857
diff changeset
  1129
    def __contains__(self, rev):
828dc8db5515 revlog: add __contains__ for fast membership test
Yuya Nishihara <yuya@tcha.org>
parents: 23857
diff changeset
  1130
        return 0 <= rev < len(self)
6750
fb42030d79d6 add __len__ and __iter__ methods to repo and revlog
Matt Mackall <mpm@selenic.com>
parents: 6703
diff changeset
  1131
    def __len__(self):
38851
781b2720d2ac index: don't include nullid in len()
Martin von Zweigbergk <martinvonz@google.com>
parents: 38850
diff changeset
  1132
        return len(self.index)
6750
fb42030d79d6 add __len__ and __iter__ methods to repo and revlog
Matt Mackall <mpm@selenic.com>
parents: 6703
diff changeset
  1133
    def __iter__(self):
38783
e7aa113b14f7 global: use pycompat.xrange()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38777
diff changeset
  1134
        return iter(pycompat.xrange(len(self)))
17672
474047947b8f clfilter: make the revlog class responsible of all its iteration
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17537
diff changeset
  1135
    def revs(self, start=0, stop=None):
474047947b8f clfilter: make the revlog class responsible of all its iteration
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17537
diff changeset
  1136
        """iterate over all rev in this revlog (from start to stop)"""
17975
c56b5b65430d revlog: allow reverse iteration with revlog.revs
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17972
diff changeset
  1137
        step = 1
38777
83a505b5cf85 revlog: don't include trailing nullrev in revlog.revs(stop=len(revlog))
Martin von Zweigbergk <martinvonz@google.com>
parents: 38763
diff changeset
  1138
        length = len(self)
17975
c56b5b65430d revlog: allow reverse iteration with revlog.revs
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17972
diff changeset
  1139
        if stop is not None:
c56b5b65430d revlog: allow reverse iteration with revlog.revs
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17972
diff changeset
  1140
            if start > stop:
c56b5b65430d revlog: allow reverse iteration with revlog.revs
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17972
diff changeset
  1141
                step = -1
c56b5b65430d revlog: allow reverse iteration with revlog.revs
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17972
diff changeset
  1142
            stop += step
38777
83a505b5cf85 revlog: don't include trailing nullrev in revlog.revs(stop=len(revlog))
Martin von Zweigbergk <martinvonz@google.com>
parents: 38763
diff changeset
  1143
            if stop > length:
83a505b5cf85 revlog: don't include trailing nullrev in revlog.revs(stop=len(revlog))
Martin von Zweigbergk <martinvonz@google.com>
parents: 38763
diff changeset
  1144
                stop = length
17672
474047947b8f clfilter: make the revlog class responsible of all its iteration
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17537
diff changeset
  1145
        else:
38777
83a505b5cf85 revlog: don't include trailing nullrev in revlog.revs(stop=len(revlog))
Martin von Zweigbergk <martinvonz@google.com>
parents: 38763
diff changeset
  1146
            stop = length
38783
e7aa113b14f7 global: use pycompat.xrange()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38777
diff changeset
  1147
        return pycompat.xrange(start, stop, step)
13275
68da048b4c88 revlog: incrementally build node cache with linear searches
Matt Mackall <mpm@selenic.com>
parents: 13268
diff changeset
  1148
68da048b4c88 revlog: incrementally build node cache with linear searches
Matt Mackall <mpm@selenic.com>
parents: 13268
diff changeset
  1149
    @util.propertycache
68da048b4c88 revlog: incrementally build node cache with linear searches
Matt Mackall <mpm@selenic.com>
parents: 13268
diff changeset
  1150
    def nodemap(self):
39143
65d5de1169dd revlog: fix pure nodemap to not access missing index entry
Yuya Nishihara <yuya@tcha.org>
parents: 39087
diff changeset
  1151
        if self.index:
65d5de1169dd revlog: fix pure nodemap to not access missing index entry
Yuya Nishihara <yuya@tcha.org>
parents: 39087
diff changeset
  1152
            # populate mapping down to the initial node
39144
5f924aa0567c revlog: obtain the first node at the lowest layer while building pure nodemap
Yuya Nishihara <yuya@tcha.org>
parents: 39143
diff changeset
  1153
            node0 = self.index[0][7]  # get around changelog filtering
5f924aa0567c revlog: obtain the first node at the lowest layer while building pure nodemap
Yuya Nishihara <yuya@tcha.org>
parents: 39143
diff changeset
  1154
            self.rev(node0)
13275
68da048b4c88 revlog: incrementally build node cache with linear searches
Matt Mackall <mpm@selenic.com>
parents: 13268
diff changeset
  1155
        return self._nodecache
13259
3b616dfa4b17 revlog: do revlog node->rev mapping by scanning
Matt Mackall <mpm@selenic.com>
parents: 13258
diff changeset
  1156
16374
29c2ff719715 revlog: add hasnode helper method
Matt Mackall <mpm@selenic.com>
parents: 15890
diff changeset
  1157
    def hasnode(self, node):
29c2ff719715 revlog: add hasnode helper method
Matt Mackall <mpm@selenic.com>
parents: 15890
diff changeset
  1158
        try:
29c2ff719715 revlog: add hasnode helper method
Matt Mackall <mpm@selenic.com>
parents: 15890
diff changeset
  1159
            self.rev(node)
29c2ff719715 revlog: add hasnode helper method
Matt Mackall <mpm@selenic.com>
parents: 15890
diff changeset
  1160
            return True
29c2ff719715 revlog: add hasnode helper method
Matt Mackall <mpm@selenic.com>
parents: 15890
diff changeset
  1161
        except KeyError:
29c2ff719715 revlog: add hasnode helper method
Matt Mackall <mpm@selenic.com>
parents: 15890
diff changeset
  1162
            return False
29c2ff719715 revlog: add hasnode helper method
Matt Mackall <mpm@selenic.com>
parents: 15890
diff changeset
  1163
36743
d031609b3cb7 changegroup: do not delta lfs revisions
Jun Wu <quark@fb.com>
parents: 35738
diff changeset
  1164
    def candelta(self, baserev, rev):
d031609b3cb7 changegroup: do not delta lfs revisions
Jun Wu <quark@fb.com>
parents: 35738
diff changeset
  1165
        """whether two revisions (baserev, rev) can be delta-ed or not"""
d031609b3cb7 changegroup: do not delta lfs revisions
Jun Wu <quark@fb.com>
parents: 35738
diff changeset
  1166
        # Disable delta if either rev requires a content-changing flag
d031609b3cb7 changegroup: do not delta lfs revisions
Jun Wu <quark@fb.com>
parents: 35738
diff changeset
  1167
        # processor (ex. LFS). This is because such flag processor can alter
d031609b3cb7 changegroup: do not delta lfs revisions
Jun Wu <quark@fb.com>
parents: 35738
diff changeset
  1168
        # the rawtext content that the delta will be based on, and two clients
d031609b3cb7 changegroup: do not delta lfs revisions
Jun Wu <quark@fb.com>
parents: 35738
diff changeset
  1169
        # could have a same revlog node with different flags (i.e. different
d031609b3cb7 changegroup: do not delta lfs revisions
Jun Wu <quark@fb.com>
parents: 35738
diff changeset
  1170
        # rawtext contents) and the delta could be incompatible.
d031609b3cb7 changegroup: do not delta lfs revisions
Jun Wu <quark@fb.com>
parents: 35738
diff changeset
  1171
        if ((self.flags(baserev) & REVIDX_RAWTEXT_CHANGING_FLAGS)
d031609b3cb7 changegroup: do not delta lfs revisions
Jun Wu <quark@fb.com>
parents: 35738
diff changeset
  1172
            or (self.flags(rev) & REVIDX_RAWTEXT_CHANGING_FLAGS)):
d031609b3cb7 changegroup: do not delta lfs revisions
Jun Wu <quark@fb.com>
parents: 35738
diff changeset
  1173
            return False
d031609b3cb7 changegroup: do not delta lfs revisions
Jun Wu <quark@fb.com>
parents: 35738
diff changeset
  1174
        return True
d031609b3cb7 changegroup: do not delta lfs revisions
Jun Wu <quark@fb.com>
parents: 35738
diff changeset
  1175
16414
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16375
diff changeset
  1176
    def clearcaches(self):
27465
072a675c51f2 revlog: make clearcaches() more effective
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27449
diff changeset
  1177
        self._cache = None
29830
92ac2baaea86 revlog: use an LRU cache for delta chain bases
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29829
diff changeset
  1178
        self._chainbasecache.clear()
27465
072a675c51f2 revlog: make clearcaches() more effective
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27449
diff changeset
  1179
        self._chunkcache = (0, '')
072a675c51f2 revlog: make clearcaches() more effective
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27449
diff changeset
  1180
        self._pcache = {}
072a675c51f2 revlog: make clearcaches() more effective
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27449
diff changeset
  1181
16414
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16375
diff changeset
  1182
        try:
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16375
diff changeset
  1183
            self._nodecache.clearcaches()
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16375
diff changeset
  1184
        except AttributeError:
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16375
diff changeset
  1185
            self._nodecache = {nullid: nullrev}
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16375
diff changeset
  1186
            self._nodepos = None
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16375
diff changeset
  1187
13259
3b616dfa4b17 revlog: do revlog node->rev mapping by scanning
Matt Mackall <mpm@selenic.com>
parents: 13258
diff changeset
  1188
    def rev(self, node):
13275
68da048b4c88 revlog: incrementally build node cache with linear searches
Matt Mackall <mpm@selenic.com>
parents: 13268
diff changeset
  1189
        try:
68da048b4c88 revlog: incrementally build node cache with linear searches
Matt Mackall <mpm@selenic.com>
parents: 13268
diff changeset
  1190
            return self._nodecache[node]
22282
4092d12ba18a repoview: fix 0L with pack/unpack for 2.4
Matt Mackall <mpm@selenic.com>
parents: 21752
diff changeset
  1191
        except TypeError:
4092d12ba18a repoview: fix 0L with pack/unpack for 2.4
Matt Mackall <mpm@selenic.com>
parents: 21752
diff changeset
  1192
            raise
16414
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16375
diff changeset
  1193
        except RevlogError:
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16375
diff changeset
  1194
            # parsers.c radix tree lookup failed
37449
a0d71618074f revlog: detect pseudo file nodeids to raise WdirUnsupported exception
Yuya Nishihara <yuya@tcha.org>
parents: 37443
diff changeset
  1195
            if node == wdirid or node in wdirfilenodeids:
32659
7b17f9de6d3e revlog: map rev(wdirid) to WdirUnsupported exception
Yuya Nishihara <yuya@tcha.org>
parents: 32443
diff changeset
  1196
                raise error.WdirUnsupported
16414
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16375
diff changeset
  1197
            raise LookupError(node, self.indexfile, _('no node'))
13275
68da048b4c88 revlog: incrementally build node cache with linear searches
Matt Mackall <mpm@selenic.com>
parents: 13268
diff changeset
  1198
        except KeyError:
16414
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16375
diff changeset
  1199
            # pure python cache lookup failed
13275
68da048b4c88 revlog: incrementally build node cache with linear searches
Matt Mackall <mpm@selenic.com>
parents: 13268
diff changeset
  1200
            n = self._nodecache
68da048b4c88 revlog: incrementally build node cache with linear searches
Matt Mackall <mpm@selenic.com>
parents: 13268
diff changeset
  1201
            i = self.index
68da048b4c88 revlog: incrementally build node cache with linear searches
Matt Mackall <mpm@selenic.com>
parents: 13268
diff changeset
  1202
            p = self._nodepos
68da048b4c88 revlog: incrementally build node cache with linear searches
Matt Mackall <mpm@selenic.com>
parents: 13268
diff changeset
  1203
            if p is None:
38851
781b2720d2ac index: don't include nullid in len()
Martin von Zweigbergk <martinvonz@google.com>
parents: 38850
diff changeset
  1204
                p = len(i) - 1
37494
1ce7a55b09d1 revlog: reset _nodepos after strip
Joerg Sonnenberger <joerg@bec.de>
parents: 37449
diff changeset
  1205
            else:
1ce7a55b09d1 revlog: reset _nodepos after strip
Joerg Sonnenberger <joerg@bec.de>
parents: 37449
diff changeset
  1206
                assert p < len(i)
38783
e7aa113b14f7 global: use pycompat.xrange()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38777
diff changeset
  1207
            for r in pycompat.xrange(p, -1, -1):
13275
68da048b4c88 revlog: incrementally build node cache with linear searches
Matt Mackall <mpm@selenic.com>
parents: 13268
diff changeset
  1208
                v = i[r][7]
68da048b4c88 revlog: incrementally build node cache with linear searches
Matt Mackall <mpm@selenic.com>
parents: 13268
diff changeset
  1209
                n[v] = r
68da048b4c88 revlog: incrementally build node cache with linear searches
Matt Mackall <mpm@selenic.com>
parents: 13268
diff changeset
  1210
                if v == node:
68da048b4c88 revlog: incrementally build node cache with linear searches
Matt Mackall <mpm@selenic.com>
parents: 13268
diff changeset
  1211
                    self._nodepos = r - 1
68da048b4c88 revlog: incrementally build node cache with linear searches
Matt Mackall <mpm@selenic.com>
parents: 13268
diff changeset
  1212
                    return r
37449
a0d71618074f revlog: detect pseudo file nodeids to raise WdirUnsupported exception
Yuya Nishihara <yuya@tcha.org>
parents: 37443
diff changeset
  1213
            if node == wdirid or node in wdirfilenodeids:
32659
7b17f9de6d3e revlog: map rev(wdirid) to WdirUnsupported exception
Yuya Nishihara <yuya@tcha.org>
parents: 32443
diff changeset
  1214
                raise error.WdirUnsupported
13275
68da048b4c88 revlog: incrementally build node cache with linear searches
Matt Mackall <mpm@selenic.com>
parents: 13268
diff changeset
  1215
            raise LookupError(node, self.indexfile, _('no node'))
68da048b4c88 revlog: incrementally build node cache with linear searches
Matt Mackall <mpm@selenic.com>
parents: 13268
diff changeset
  1216
30287
0986f225c149 revlog: reorder index accessors to match data structure order
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30210
diff changeset
  1217
    # Accessors for index entries.
0986f225c149 revlog: reorder index accessors to match data structure order
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30210
diff changeset
  1218
0986f225c149 revlog: reorder index accessors to match data structure order
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30210
diff changeset
  1219
    # First tuple entry is 8 bytes. First 6 bytes are offset. Last 2 bytes
0986f225c149 revlog: reorder index accessors to match data structure order
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30210
diff changeset
  1220
    # are flags.
2072
74d3f5336b66 Implement revlogng.
mason@suse.com
parents: 2002
diff changeset
  1221
    def start(self, rev):
5006
c2febf5420e9 revlog: minor chunk speed-up
Matt Mackall <mpm@selenic.com>
parents: 5005
diff changeset
  1222
        return int(self.index[rev][0] >> 16)
30287
0986f225c149 revlog: reorder index accessors to match data structure order
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30210
diff changeset
  1223
0986f225c149 revlog: reorder index accessors to match data structure order
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30210
diff changeset
  1224
    def flags(self, rev):
0986f225c149 revlog: reorder index accessors to match data structure order
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30210
diff changeset
  1225
        return self.index[rev][0] & 0xFFFF
0986f225c149 revlog: reorder index accessors to match data structure order
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30210
diff changeset
  1226
4980
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
  1227
    def length(self, rev):
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
  1228
        return self.index[rev][1]
30287
0986f225c149 revlog: reorder index accessors to match data structure order
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30210
diff changeset
  1229
0986f225c149 revlog: reorder index accessors to match data structure order
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30210
diff changeset
  1230
    def rawsize(self, rev):
0986f225c149 revlog: reorder index accessors to match data structure order
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30210
diff changeset
  1231
        """return the length of the uncompressed text for a given revision"""
0986f225c149 revlog: reorder index accessors to match data structure order
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30210
diff changeset
  1232
        l = self.index[rev][2]
38177
7fa3408f83ef revlog: disallow setting uncompressed length to None
Yuya Nishihara <yuya@tcha.org>
parents: 38169
diff changeset
  1233
        if l >= 0:
30287
0986f225c149 revlog: reorder index accessors to match data structure order
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30210
diff changeset
  1234
            return l
0986f225c149 revlog: reorder index accessors to match data structure order
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30210
diff changeset
  1235
31801
d22f29abeb42 revlog: use raw revision for rawsize
Jun Wu <quark@fb.com>
parents: 31756
diff changeset
  1236
        t = self.revision(rev, raw=True)
30287
0986f225c149 revlog: reorder index accessors to match data structure order
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30210
diff changeset
  1237
        return len(t)
31856
0ab7f469d386 revlog: make "size" diverge from "rawsize"
Jun Wu <quark@fb.com>
parents: 31804
diff changeset
  1238
0ab7f469d386 revlog: make "size" diverge from "rawsize"
Jun Wu <quark@fb.com>
parents: 31804
diff changeset
  1239
    def size(self, rev):
0ab7f469d386 revlog: make "size" diverge from "rawsize"
Jun Wu <quark@fb.com>
parents: 31804
diff changeset
  1240
        """length of non-raw text (processed by a "read" flag processor)"""
0ab7f469d386 revlog: make "size" diverge from "rawsize"
Jun Wu <quark@fb.com>
parents: 31804
diff changeset
  1241
        # fast path: if no "read" flag processor could change the content,
0ab7f469d386 revlog: make "size" diverge from "rawsize"
Jun Wu <quark@fb.com>
parents: 31804
diff changeset
  1242
        # size is rawsize. note: ELLIPSIS is known to not change the content.
0ab7f469d386 revlog: make "size" diverge from "rawsize"
Jun Wu <quark@fb.com>
parents: 31804
diff changeset
  1243
        flags = self.flags(rev)
0ab7f469d386 revlog: make "size" diverge from "rawsize"
Jun Wu <quark@fb.com>
parents: 31804
diff changeset
  1244
        if flags & (REVIDX_KNOWN_FLAGS ^ REVIDX_ELLIPSIS) == 0:
0ab7f469d386 revlog: make "size" diverge from "rawsize"
Jun Wu <quark@fb.com>
parents: 31804
diff changeset
  1245
            return self.rawsize(rev)
0ab7f469d386 revlog: make "size" diverge from "rawsize"
Jun Wu <quark@fb.com>
parents: 31804
diff changeset
  1246
0ab7f469d386 revlog: make "size" diverge from "rawsize"
Jun Wu <quark@fb.com>
parents: 31804
diff changeset
  1247
        return len(self.revision(rev, raw=False))
30287
0986f225c149 revlog: reorder index accessors to match data structure order
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30210
diff changeset
  1248
14252
19067884c5f5 revlog: calculate base revisions iteratively
Sune Foldager <cryo@cyanite.org>
parents: 14251
diff changeset
  1249
    def chainbase(self, rev):
29830
92ac2baaea86 revlog: use an LRU cache for delta chain bases
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29829
diff changeset
  1250
        base = self._chainbasecache.get(rev)
92ac2baaea86 revlog: use an LRU cache for delta chain bases
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29829
diff changeset
  1251
        if base is not None:
92ac2baaea86 revlog: use an LRU cache for delta chain bases
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29829
diff changeset
  1252
            return base
92ac2baaea86 revlog: use an LRU cache for delta chain bases
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29829
diff changeset
  1253
14252
19067884c5f5 revlog: calculate base revisions iteratively
Sune Foldager <cryo@cyanite.org>
parents: 14251
diff changeset
  1254
        index = self.index
38169
fc72beec2a1a revlog: make chainbase cache its result for the correct revision
Paul Morelle <paul.morelle@octobus.net>
parents: 38168
diff changeset
  1255
        iterrev = rev
fc72beec2a1a revlog: make chainbase cache its result for the correct revision
Paul Morelle <paul.morelle@octobus.net>
parents: 38168
diff changeset
  1256
        base = index[iterrev][3]
fc72beec2a1a revlog: make chainbase cache its result for the correct revision
Paul Morelle <paul.morelle@octobus.net>
parents: 38168
diff changeset
  1257
        while base != iterrev:
fc72beec2a1a revlog: make chainbase cache its result for the correct revision
Paul Morelle <paul.morelle@octobus.net>
parents: 38168
diff changeset
  1258
            iterrev = base
fc72beec2a1a revlog: make chainbase cache its result for the correct revision
Paul Morelle <paul.morelle@octobus.net>
parents: 38168
diff changeset
  1259
            base = index[iterrev][3]
29830
92ac2baaea86 revlog: use an LRU cache for delta chain bases
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29829
diff changeset
  1260
92ac2baaea86 revlog: use an LRU cache for delta chain bases
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29829
diff changeset
  1261
        self._chainbasecache[rev] = base
14252
19067884c5f5 revlog: calculate base revisions iteratively
Sune Foldager <cryo@cyanite.org>
parents: 14251
diff changeset
  1262
        return base
30287
0986f225c149 revlog: reorder index accessors to match data structure order
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30210
diff changeset
  1263
0986f225c149 revlog: reorder index accessors to match data structure order
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30210
diff changeset
  1264
    def linkrev(self, rev):
0986f225c149 revlog: reorder index accessors to match data structure order
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30210
diff changeset
  1265
        return self.index[rev][4]
0986f225c149 revlog: reorder index accessors to match data structure order
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30210
diff changeset
  1266
0986f225c149 revlog: reorder index accessors to match data structure order
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30210
diff changeset
  1267
    def parentrevs(self, rev):
32403
a28c76e1cea9 revlog: raise WdirUnsupported when wdirrev is passed
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32393
diff changeset
  1268
        try:
35521
a0fab647a8f1 revlog: don't use slicing to return parents
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35345
diff changeset
  1269
            entry = self.index[rev]
32403
a28c76e1cea9 revlog: raise WdirUnsupported when wdirrev is passed
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32393
diff changeset
  1270
        except IndexError:
a28c76e1cea9 revlog: raise WdirUnsupported when wdirrev is passed
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32393
diff changeset
  1271
            if rev == wdirrev:
a28c76e1cea9 revlog: raise WdirUnsupported when wdirrev is passed
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32393
diff changeset
  1272
                raise error.WdirUnsupported
a28c76e1cea9 revlog: raise WdirUnsupported when wdirrev is passed
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32393
diff changeset
  1273
            raise
30287
0986f225c149 revlog: reorder index accessors to match data structure order
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30210
diff changeset
  1274
35521
a0fab647a8f1 revlog: don't use slicing to return parents
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35345
diff changeset
  1275
        return entry[5], entry[6]
a0fab647a8f1 revlog: don't use slicing to return parents
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35345
diff changeset
  1276
30287
0986f225c149 revlog: reorder index accessors to match data structure order
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30210
diff changeset
  1277
    def node(self, rev):
32443
34e9b8b94f66 revlog: raise error.WdirUnsupported from revlog.node() if wdirrev is passed
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32403
diff changeset
  1278
        try:
34e9b8b94f66 revlog: raise error.WdirUnsupported from revlog.node() if wdirrev is passed
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32403
diff changeset
  1279
            return self.index[rev][7]
34e9b8b94f66 revlog: raise error.WdirUnsupported from revlog.node() if wdirrev is passed
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32403
diff changeset
  1280
        except IndexError:
34e9b8b94f66 revlog: raise error.WdirUnsupported from revlog.node() if wdirrev is passed
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32403
diff changeset
  1281
            if rev == wdirrev:
34e9b8b94f66 revlog: raise error.WdirUnsupported from revlog.node() if wdirrev is passed
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32403
diff changeset
  1282
                raise error.WdirUnsupported
34e9b8b94f66 revlog: raise error.WdirUnsupported from revlog.node() if wdirrev is passed
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32403
diff changeset
  1283
            raise
30287
0986f225c149 revlog: reorder index accessors to match data structure order
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30210
diff changeset
  1284
0986f225c149 revlog: reorder index accessors to match data structure order
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30210
diff changeset
  1285
    # Derived from index values.
0986f225c149 revlog: reorder index accessors to match data structure order
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30210
diff changeset
  1286
0986f225c149 revlog: reorder index accessors to match data structure order
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30210
diff changeset
  1287
    def end(self, rev):
0986f225c149 revlog: reorder index accessors to match data structure order
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30210
diff changeset
  1288
        return self.start(rev) + self.length(rev)
0986f225c149 revlog: reorder index accessors to match data structure order
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30210
diff changeset
  1289
0986f225c149 revlog: reorder index accessors to match data structure order
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30210
diff changeset
  1290
    def parents(self, node):
0986f225c149 revlog: reorder index accessors to match data structure order
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30210
diff changeset
  1291
        i = self.index
0986f225c149 revlog: reorder index accessors to match data structure order
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30210
diff changeset
  1292
        d = i[self.rev(node)]
0986f225c149 revlog: reorder index accessors to match data structure order
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30210
diff changeset
  1293
        return i[d[5]][7], i[d[6]][7] # map revisions to nodes inline
0986f225c149 revlog: reorder index accessors to match data structure order
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30210
diff changeset
  1294
23254
d23834b871ac debugrevlog: fix computing chain length in debugrevlog -d
Mateusz Kwapich <mitrandir@fb.com>
parents: 22934
diff changeset
  1295
    def chainlen(self, rev):
23286
40e0067899d4 revlog: compute length of compressed deltas along with chain length
Siddharth Agarwal <sid0@fb.com>
parents: 23285
diff changeset
  1296
        return self._chaininfo(rev)[0]
23306
f7a42f8e82bd revlog: cache chain info after calculating it for a rev (issue4452)
Siddharth Agarwal <sid0@fb.com>
parents: 23288
diff changeset
  1297
23286
40e0067899d4 revlog: compute length of compressed deltas along with chain length
Siddharth Agarwal <sid0@fb.com>
parents: 23285
diff changeset
  1298
    def _chaininfo(self, rev):
23306
f7a42f8e82bd revlog: cache chain info after calculating it for a rev (issue4452)
Siddharth Agarwal <sid0@fb.com>
parents: 23288
diff changeset
  1299
        chaininfocache = self._chaininfocache
f7a42f8e82bd revlog: cache chain info after calculating it for a rev (issue4452)
Siddharth Agarwal <sid0@fb.com>
parents: 23288
diff changeset
  1300
        if rev in chaininfocache:
f7a42f8e82bd revlog: cache chain info after calculating it for a rev (issue4452)
Siddharth Agarwal <sid0@fb.com>
parents: 23288
diff changeset
  1301
            return chaininfocache[rev]
23254
d23834b871ac debugrevlog: fix computing chain length in debugrevlog -d
Mateusz Kwapich <mitrandir@fb.com>
parents: 22934
diff changeset
  1302
        index = self.index
d23834b871ac debugrevlog: fix computing chain length in debugrevlog -d
Mateusz Kwapich <mitrandir@fb.com>
parents: 22934
diff changeset
  1303
        generaldelta = self._generaldelta
d23834b871ac debugrevlog: fix computing chain length in debugrevlog -d
Mateusz Kwapich <mitrandir@fb.com>
parents: 22934
diff changeset
  1304
        iterrev = rev
d23834b871ac debugrevlog: fix computing chain length in debugrevlog -d
Mateusz Kwapich <mitrandir@fb.com>
parents: 22934
diff changeset
  1305
        e = index[iterrev]
d23834b871ac debugrevlog: fix computing chain length in debugrevlog -d
Mateusz Kwapich <mitrandir@fb.com>
parents: 22934
diff changeset
  1306
        clen = 0
23286
40e0067899d4 revlog: compute length of compressed deltas along with chain length
Siddharth Agarwal <sid0@fb.com>
parents: 23285
diff changeset
  1307
        compresseddeltalen = 0
23254
d23834b871ac debugrevlog: fix computing chain length in debugrevlog -d
Mateusz Kwapich <mitrandir@fb.com>
parents: 22934
diff changeset
  1308
        while iterrev != e[3]:
d23834b871ac debugrevlog: fix computing chain length in debugrevlog -d
Mateusz Kwapich <mitrandir@fb.com>
parents: 22934
diff changeset
  1309
            clen += 1
23286
40e0067899d4 revlog: compute length of compressed deltas along with chain length
Siddharth Agarwal <sid0@fb.com>
parents: 23285
diff changeset
  1310
            compresseddeltalen += e[1]
23254
d23834b871ac debugrevlog: fix computing chain length in debugrevlog -d
Mateusz Kwapich <mitrandir@fb.com>
parents: 22934
diff changeset
  1311
            if generaldelta:
d23834b871ac debugrevlog: fix computing chain length in debugrevlog -d
Mateusz Kwapich <mitrandir@fb.com>
parents: 22934
diff changeset
  1312
                iterrev = e[3]
d23834b871ac debugrevlog: fix computing chain length in debugrevlog -d
Mateusz Kwapich <mitrandir@fb.com>
parents: 22934
diff changeset
  1313
            else:
d23834b871ac debugrevlog: fix computing chain length in debugrevlog -d
Mateusz Kwapich <mitrandir@fb.com>
parents: 22934
diff changeset
  1314
                iterrev -= 1
23306
f7a42f8e82bd revlog: cache chain info after calculating it for a rev (issue4452)
Siddharth Agarwal <sid0@fb.com>
parents: 23288
diff changeset
  1315
            if iterrev in chaininfocache:
f7a42f8e82bd revlog: cache chain info after calculating it for a rev (issue4452)
Siddharth Agarwal <sid0@fb.com>
parents: 23288
diff changeset
  1316
                t = chaininfocache[iterrev]
f7a42f8e82bd revlog: cache chain info after calculating it for a rev (issue4452)
Siddharth Agarwal <sid0@fb.com>
parents: 23288
diff changeset
  1317
                clen += t[0]
f7a42f8e82bd revlog: cache chain info after calculating it for a rev (issue4452)
Siddharth Agarwal <sid0@fb.com>
parents: 23288
diff changeset
  1318
                compresseddeltalen += t[1]
f7a42f8e82bd revlog: cache chain info after calculating it for a rev (issue4452)
Siddharth Agarwal <sid0@fb.com>
parents: 23288
diff changeset
  1319
                break
23254
d23834b871ac debugrevlog: fix computing chain length in debugrevlog -d
Mateusz Kwapich <mitrandir@fb.com>
parents: 22934
diff changeset
  1320
            e = index[iterrev]
23306
f7a42f8e82bd revlog: cache chain info after calculating it for a rev (issue4452)
Siddharth Agarwal <sid0@fb.com>
parents: 23288
diff changeset
  1321
        else:
f7a42f8e82bd revlog: cache chain info after calculating it for a rev (issue4452)
Siddharth Agarwal <sid0@fb.com>
parents: 23288
diff changeset
  1322
            # Add text length of base since decompressing that also takes
f7a42f8e82bd revlog: cache chain info after calculating it for a rev (issue4452)
Siddharth Agarwal <sid0@fb.com>
parents: 23288
diff changeset
  1323
            # work. For cache hits the length is already included.
f7a42f8e82bd revlog: cache chain info after calculating it for a rev (issue4452)
Siddharth Agarwal <sid0@fb.com>
parents: 23288
diff changeset
  1324
            compresseddeltalen += e[1]
f7a42f8e82bd revlog: cache chain info after calculating it for a rev (issue4452)
Siddharth Agarwal <sid0@fb.com>
parents: 23288
diff changeset
  1325
        r = (clen, compresseddeltalen)
f7a42f8e82bd revlog: cache chain info after calculating it for a rev (issue4452)
Siddharth Agarwal <sid0@fb.com>
parents: 23288
diff changeset
  1326
        chaininfocache[rev] = r
f7a42f8e82bd revlog: cache chain info after calculating it for a rev (issue4452)
Siddharth Agarwal <sid0@fb.com>
parents: 23288
diff changeset
  1327
        return r
f7a42f8e82bd revlog: cache chain info after calculating it for a rev (issue4452)
Siddharth Agarwal <sid0@fb.com>
parents: 23288
diff changeset
  1328
27468
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
  1329
    def _deltachain(self, rev, stoprev=None):
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
  1330
        """Obtain the delta chain for a revision.
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
  1331
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
  1332
        ``stoprev`` specifies a revision to stop at. If not specified, we
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
  1333
        stop at the base of the chain.
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
  1334
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
  1335
        Returns a 2-tuple of (chain, stopped) where ``chain`` is a list of
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
  1336
        revs in ascending order and ``stopped`` is a bool indicating whether
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
  1337
        ``stoprev`` was hit.
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
  1338
        """
33171
6d678ab1b10d revlog: C implementation of delta chain resolution
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32969
diff changeset
  1339
        # Try C implementation.
6d678ab1b10d revlog: C implementation of delta chain resolution
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32969
diff changeset
  1340
        try:
6d678ab1b10d revlog: C implementation of delta chain resolution
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32969
diff changeset
  1341
            return self.index.deltachain(rev, stoprev, self._generaldelta)
6d678ab1b10d revlog: C implementation of delta chain resolution
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32969
diff changeset
  1342
        except AttributeError:
6d678ab1b10d revlog: C implementation of delta chain resolution
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32969
diff changeset
  1343
            pass
6d678ab1b10d revlog: C implementation of delta chain resolution
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32969
diff changeset
  1344
27468
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
  1345
        chain = []
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
  1346
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
  1347
        # Alias to prevent attribute lookup in tight loop.
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
  1348
        index = self.index
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
  1349
        generaldelta = self._generaldelta
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
  1350
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
  1351
        iterrev = rev
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
  1352
        e = index[iterrev]
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
  1353
        while iterrev != e[3] and iterrev != stoprev:
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
  1354
            chain.append(iterrev)
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
  1355
            if generaldelta:
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
  1356
                iterrev = e[3]
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
  1357
            else:
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
  1358
                iterrev -= 1
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
  1359
            e = index[iterrev]
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
  1360
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
  1361
        if iterrev == stoprev:
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
  1362
            stopped = True
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
  1363
        else:
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
  1364
            chain.append(iterrev)
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
  1365
            stopped = False
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
  1366
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
  1367
        chain.reverse()
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
  1368
        return chain, stopped
93ac15f03331 revlog: refactor delta chain computation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27465
diff changeset
  1369
18081
f88c60e740a1 revlog.ancestors: add support for including revs
Siddharth Agarwal <sid0@fb.com>
parents: 17975
diff changeset
  1370
    def ancestors(self, revs, stoprev=0, inclusive=False):
10047
27267b1f68b4 revlog: rewrite several method docstrings
Greg Ward <greg-hg@gerg.ca>
parents: 9679
diff changeset
  1371
        """Generate the ancestors of 'revs' in reverse topological order.
16868
eb88ed4269c5 revlog: add optional stoprev arg to revlog.ancestors()
Joshua Redstone <joshua.redstone@fb.com>
parents: 16867
diff changeset
  1372
        Does not generate revs lower than stoprev.
10047
27267b1f68b4 revlog: rewrite several method docstrings
Greg Ward <greg-hg@gerg.ca>
parents: 9679
diff changeset
  1373
18090
9abc55ef85b5 revlog: move ancestor generation out to a new class
Siddharth Agarwal <sid0@fb.com>
parents: 18083
diff changeset
  1374
        See the documentation for ancestor.lazyancestors for more details."""
18081
f88c60e740a1 revlog.ancestors: add support for including revs
Siddharth Agarwal <sid0@fb.com>
parents: 17975
diff changeset
  1375
23328
3a7d9c0c57a5 ancestor.lazyancestors: take parentrevs function rather than changelog
Siddharth Agarwal <sid0@fb.com>
parents: 23306
diff changeset
  1376
        return ancestor.lazyancestors(self.parentrevs, revs, stoprev=stoprev,
18090
9abc55ef85b5 revlog: move ancestor generation out to a new class
Siddharth Agarwal <sid0@fb.com>
parents: 18083
diff changeset
  1377
                                      inclusive=inclusive)
6872
c7cc40fd74f6 Add ancestors and descendants to revlog
Stefano Tortarolo <stefano.tortarolo@gmail.com>
parents: 6750
diff changeset
  1378
16867
1093ad1e8903 revlog: descendants(*revs) becomes descendants(revs) (API)
Bryan O'Sullivan <bryano@fb.com>
parents: 16866
diff changeset
  1379
    def descendants(self, revs):
10047
27267b1f68b4 revlog: rewrite several method docstrings
Greg Ward <greg-hg@gerg.ca>
parents: 9679
diff changeset
  1380
        """Generate the descendants of 'revs' in revision order.
27267b1f68b4 revlog: rewrite several method docstrings
Greg Ward <greg-hg@gerg.ca>
parents: 9679
diff changeset
  1381
27267b1f68b4 revlog: rewrite several method docstrings
Greg Ward <greg-hg@gerg.ca>
parents: 9679
diff changeset
  1382
        Yield a sequence of revision numbers starting with a child of
27267b1f68b4 revlog: rewrite several method docstrings
Greg Ward <greg-hg@gerg.ca>
parents: 9679
diff changeset
  1383
        some rev in revs, i.e., each revision is *not* considered a
27267b1f68b4 revlog: rewrite several method docstrings
Greg Ward <greg-hg@gerg.ca>
parents: 9679
diff changeset
  1384
        descendant of itself.  Results are ordered by revision number (a
27267b1f68b4 revlog: rewrite several method docstrings
Greg Ward <greg-hg@gerg.ca>
parents: 9679
diff changeset
  1385
        topological sort)."""
12950
2405b4a5964a revlog: fix descendants() if nullrev is in revs
Nicolas Dumazet <nicdumz.commits@gmail.com>
parents: 12949
diff changeset
  1386
        first = min(revs)
2405b4a5964a revlog: fix descendants() if nullrev is in revs
Nicolas Dumazet <nicdumz.commits@gmail.com>
parents: 12949
diff changeset
  1387
        if first == nullrev:
2405b4a5964a revlog: fix descendants() if nullrev is in revs
Nicolas Dumazet <nicdumz.commits@gmail.com>
parents: 12949
diff changeset
  1388
            for i in self:
2405b4a5964a revlog: fix descendants() if nullrev is in revs
Nicolas Dumazet <nicdumz.commits@gmail.com>
parents: 12949
diff changeset
  1389
                yield i
2405b4a5964a revlog: fix descendants() if nullrev is in revs
Nicolas Dumazet <nicdumz.commits@gmail.com>
parents: 12949
diff changeset
  1390
            return
2405b4a5964a revlog: fix descendants() if nullrev is in revs
Nicolas Dumazet <nicdumz.commits@gmail.com>
parents: 12949
diff changeset
  1391
8150
bbc24c0753a0 util: use built-in set and frozenset
Martin Geisler <mg@lazybytes.net>
parents: 8073
diff changeset
  1392
        seen = set(revs)
17672
474047947b8f clfilter: make the revlog class responsible of all its iteration
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17537
diff changeset
  1393
        for i in self.revs(start=first + 1):
6872
c7cc40fd74f6 Add ancestors and descendants to revlog
Stefano Tortarolo <stefano.tortarolo@gmail.com>
parents: 6750
diff changeset
  1394
            for x in self.parentrevs(i):
c7cc40fd74f6 Add ancestors and descendants to revlog
Stefano Tortarolo <stefano.tortarolo@gmail.com>
parents: 6750
diff changeset
  1395
                if x != nullrev and x in seen:
c7cc40fd74f6 Add ancestors and descendants to revlog
Stefano Tortarolo <stefano.tortarolo@gmail.com>
parents: 6750
diff changeset
  1396
                    seen.add(i)
c7cc40fd74f6 Add ancestors and descendants to revlog
Stefano Tortarolo <stefano.tortarolo@gmail.com>
parents: 6750
diff changeset
  1397
                    yield i
c7cc40fd74f6 Add ancestors and descendants to revlog
Stefano Tortarolo <stefano.tortarolo@gmail.com>
parents: 6750
diff changeset
  1398
                    break
c7cc40fd74f6 Add ancestors and descendants to revlog
Stefano Tortarolo <stefano.tortarolo@gmail.com>
parents: 6750
diff changeset
  1399
13741
b51bf961b3cb wireproto: add getbundle() function
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13400
diff changeset
  1400
    def findcommonmissing(self, common=None, heads=None):
b51bf961b3cb wireproto: add getbundle() function
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13400
diff changeset
  1401
        """Return a tuple of the ancestors of common and the ancestors of heads
15835
fa15869bf95c revlog: improve docstring for findcommonmissing
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 15827
diff changeset
  1402
        that are not ancestors of common. In revset terminology, we return the
fa15869bf95c revlog: improve docstring for findcommonmissing
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 15827
diff changeset
  1403
        tuple:
10047
27267b1f68b4 revlog: rewrite several method docstrings
Greg Ward <greg-hg@gerg.ca>
parents: 9679
diff changeset
  1404
15835
fa15869bf95c revlog: improve docstring for findcommonmissing
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 15827
diff changeset
  1405
          ::common, (::heads) - (::common)
7233
9f0e52e1df77 fix pull racing with push/commit (issue1320)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 7109
diff changeset
  1406
10047
27267b1f68b4 revlog: rewrite several method docstrings
Greg Ward <greg-hg@gerg.ca>
parents: 9679
diff changeset
  1407
        The list is sorted by revision number, meaning it is
27267b1f68b4 revlog: rewrite several method docstrings
Greg Ward <greg-hg@gerg.ca>
parents: 9679
diff changeset
  1408
        topologically sorted.
27267b1f68b4 revlog: rewrite several method docstrings
Greg Ward <greg-hg@gerg.ca>
parents: 9679
diff changeset
  1409
27267b1f68b4 revlog: rewrite several method docstrings
Greg Ward <greg-hg@gerg.ca>
parents: 9679
diff changeset
  1410
        'heads' and 'common' are both lists of node IDs.  If heads is
27267b1f68b4 revlog: rewrite several method docstrings
Greg Ward <greg-hg@gerg.ca>
parents: 9679
diff changeset
  1411
        not supplied, uses all of the revlog's heads.  If common is not
27267b1f68b4 revlog: rewrite several method docstrings
Greg Ward <greg-hg@gerg.ca>
parents: 9679
diff changeset
  1412
        supplied, uses nullid."""
7233
9f0e52e1df77 fix pull racing with push/commit (issue1320)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 7109
diff changeset
  1413
        if common is None:
9f0e52e1df77 fix pull racing with push/commit (issue1320)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 7109
diff changeset
  1414
            common = [nullid]
9f0e52e1df77 fix pull racing with push/commit (issue1320)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 7109
diff changeset
  1415
        if heads is None:
9f0e52e1df77 fix pull racing with push/commit (issue1320)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 7109
diff changeset
  1416
            heads = self.heads()
9f0e52e1df77 fix pull racing with push/commit (issue1320)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 7109
diff changeset
  1417
9f0e52e1df77 fix pull racing with push/commit (issue1320)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 7109
diff changeset
  1418
        common = [self.rev(n) for n in common]
9f0e52e1df77 fix pull racing with push/commit (issue1320)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 7109
diff changeset
  1419
        heads = [self.rev(n) for n in heads]
9f0e52e1df77 fix pull racing with push/commit (issue1320)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 7109
diff changeset
  1420
9f0e52e1df77 fix pull racing with push/commit (issue1320)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 7109
diff changeset
  1421
        # we want the ancestors, but inclusive
20073
eeba4eaf0716 revlog: return lazy set from findcommonmissing
Durham Goode <durham@fb.com>
parents: 19776
diff changeset
  1422
        class lazyset(object):
eeba4eaf0716 revlog: return lazy set from findcommonmissing
Durham Goode <durham@fb.com>
parents: 19776
diff changeset
  1423
            def __init__(self, lazyvalues):
eeba4eaf0716 revlog: return lazy set from findcommonmissing
Durham Goode <durham@fb.com>
parents: 19776
diff changeset
  1424
                self.addedvalues = set()
eeba4eaf0716 revlog: return lazy set from findcommonmissing
Durham Goode <durham@fb.com>
parents: 19776
diff changeset
  1425
                self.lazyvalues = lazyvalues
eeba4eaf0716 revlog: return lazy set from findcommonmissing
Durham Goode <durham@fb.com>
parents: 19776
diff changeset
  1426
eeba4eaf0716 revlog: return lazy set from findcommonmissing
Durham Goode <durham@fb.com>
parents: 19776
diff changeset
  1427
            def __contains__(self, value):
eeba4eaf0716 revlog: return lazy set from findcommonmissing
Durham Goode <durham@fb.com>
parents: 19776
diff changeset
  1428
                return value in self.addedvalues or value in self.lazyvalues
eeba4eaf0716 revlog: return lazy set from findcommonmissing
Durham Goode <durham@fb.com>
parents: 19776
diff changeset
  1429
eeba4eaf0716 revlog: return lazy set from findcommonmissing
Durham Goode <durham@fb.com>
parents: 19776
diff changeset
  1430
            def __iter__(self):
eeba4eaf0716 revlog: return lazy set from findcommonmissing
Durham Goode <durham@fb.com>
parents: 19776
diff changeset
  1431
                added = self.addedvalues
eeba4eaf0716 revlog: return lazy set from findcommonmissing
Durham Goode <durham@fb.com>
parents: 19776
diff changeset
  1432
                for r in added:
eeba4eaf0716 revlog: return lazy set from findcommonmissing
Durham Goode <durham@fb.com>
parents: 19776
diff changeset
  1433
                    yield r
eeba4eaf0716 revlog: return lazy set from findcommonmissing
Durham Goode <durham@fb.com>
parents: 19776
diff changeset
  1434
                for r in self.lazyvalues:
eeba4eaf0716 revlog: return lazy set from findcommonmissing
Durham Goode <durham@fb.com>
parents: 19776
diff changeset
  1435
                    if not r in added:
eeba4eaf0716 revlog: return lazy set from findcommonmissing
Durham Goode <durham@fb.com>
parents: 19776
diff changeset
  1436
                        yield r
eeba4eaf0716 revlog: return lazy set from findcommonmissing
Durham Goode <durham@fb.com>
parents: 19776
diff changeset
  1437
eeba4eaf0716 revlog: return lazy set from findcommonmissing
Durham Goode <durham@fb.com>
parents: 19776
diff changeset
  1438
            def add(self, value):
eeba4eaf0716 revlog: return lazy set from findcommonmissing
Durham Goode <durham@fb.com>
parents: 19776
diff changeset
  1439
                self.addedvalues.add(value)
eeba4eaf0716 revlog: return lazy set from findcommonmissing
Durham Goode <durham@fb.com>
parents: 19776
diff changeset
  1440
eeba4eaf0716 revlog: return lazy set from findcommonmissing
Durham Goode <durham@fb.com>
parents: 19776
diff changeset
  1441
            def update(self, values):
eeba4eaf0716 revlog: return lazy set from findcommonmissing
Durham Goode <durham@fb.com>
parents: 19776
diff changeset
  1442
                self.addedvalues.update(values)
eeba4eaf0716 revlog: return lazy set from findcommonmissing
Durham Goode <durham@fb.com>
parents: 19776
diff changeset
  1443
eeba4eaf0716 revlog: return lazy set from findcommonmissing
Durham Goode <durham@fb.com>
parents: 19776
diff changeset
  1444
        has = lazyset(self.ancestors(common))
8152
08e1baf924ca replace set-like dictionaries with real sets
Martin Geisler <mg@lazybytes.net>
parents: 8150
diff changeset
  1445
        has.add(nullrev)
08e1baf924ca replace set-like dictionaries with real sets
Martin Geisler <mg@lazybytes.net>
parents: 8150
diff changeset
  1446
        has.update(common)
7233
9f0e52e1df77 fix pull racing with push/commit (issue1320)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 7109
diff changeset
  1447
9f0e52e1df77 fix pull racing with push/commit (issue1320)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 7109
diff changeset
  1448
        # take all ancestors from heads that aren't in has
8453
d1ca637b0773 revlog.missing(): use sets instead of a dict
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 8391
diff changeset
  1449
        missing = set()
25113
0ca8410ea345 util: drop alias for collections.deque
Martin von Zweigbergk <martinvonz@google.com>
parents: 24454
diff changeset
  1450
        visit = collections.deque(r for r in heads if r not in has)
7233
9f0e52e1df77 fix pull racing with push/commit (issue1320)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 7109
diff changeset
  1451
        while visit:
16803
107a3270a24a cleanup: use the deque type where appropriate
Bryan O'Sullivan <bryano@fb.com>
parents: 16786
diff changeset
  1452
            r = visit.popleft()
7233
9f0e52e1df77 fix pull racing with push/commit (issue1320)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 7109
diff changeset
  1453
            if r in missing:
9f0e52e1df77 fix pull racing with push/commit (issue1320)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 7109
diff changeset
  1454
                continue
9f0e52e1df77 fix pull racing with push/commit (issue1320)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 7109
diff changeset
  1455
            else:
8453
d1ca637b0773 revlog.missing(): use sets instead of a dict
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 8391
diff changeset
  1456
                missing.add(r)
7233
9f0e52e1df77 fix pull racing with push/commit (issue1320)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 7109
diff changeset
  1457
                for p in self.parentrevs(r):
9f0e52e1df77 fix pull racing with push/commit (issue1320)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 7109
diff changeset
  1458
                    if p not in has:
9f0e52e1df77 fix pull racing with push/commit (issue1320)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 7109
diff changeset
  1459
                        visit.append(p)
8453
d1ca637b0773 revlog.missing(): use sets instead of a dict
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 8391
diff changeset
  1460
        missing = list(missing)
7233
9f0e52e1df77 fix pull racing with push/commit (issue1320)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 7109
diff changeset
  1461
        missing.sort()
30391
2ded17b64f09 revlog: avoid shadowing several variables using list comprehensions
Augie Fackler <augie@google.com>
parents: 30289
diff changeset
  1462
        return has, [self.node(miss) for miss in missing]
13741
b51bf961b3cb wireproto: add getbundle() function
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13400
diff changeset
  1463
23337
3a8a763f4197 revlog: add a method to get missing revs incrementally
Siddharth Agarwal <sid0@fb.com>
parents: 23328
diff changeset
  1464
    def incrementalmissingrevs(self, common=None):
3a8a763f4197 revlog: add a method to get missing revs incrementally
Siddharth Agarwal <sid0@fb.com>
parents: 23328
diff changeset
  1465
        """Return an object that can be used to incrementally compute the
3a8a763f4197 revlog: add a method to get missing revs incrementally
Siddharth Agarwal <sid0@fb.com>
parents: 23328
diff changeset
  1466
        revision numbers of the ancestors of arbitrary sets that are not
3a8a763f4197 revlog: add a method to get missing revs incrementally
Siddharth Agarwal <sid0@fb.com>
parents: 23328
diff changeset
  1467
        ancestors of common. This is an ancestor.incrementalmissingancestors
3a8a763f4197 revlog: add a method to get missing revs incrementally
Siddharth Agarwal <sid0@fb.com>
parents: 23328
diff changeset
  1468
        object.
3a8a763f4197 revlog: add a method to get missing revs incrementally
Siddharth Agarwal <sid0@fb.com>
parents: 23328
diff changeset
  1469
3a8a763f4197 revlog: add a method to get missing revs incrementally
Siddharth Agarwal <sid0@fb.com>
parents: 23328
diff changeset
  1470
        'common' is a list of revision numbers. If common is not supplied, uses
3a8a763f4197 revlog: add a method to get missing revs incrementally
Siddharth Agarwal <sid0@fb.com>
parents: 23328
diff changeset
  1471
        nullrev.
3a8a763f4197 revlog: add a method to get missing revs incrementally
Siddharth Agarwal <sid0@fb.com>
parents: 23328
diff changeset
  1472
        """
3a8a763f4197 revlog: add a method to get missing revs incrementally
Siddharth Agarwal <sid0@fb.com>
parents: 23328
diff changeset
  1473
        if common is None:
3a8a763f4197 revlog: add a method to get missing revs incrementally
Siddharth Agarwal <sid0@fb.com>
parents: 23328
diff changeset
  1474
            common = [nullrev]
3a8a763f4197 revlog: add a method to get missing revs incrementally
Siddharth Agarwal <sid0@fb.com>
parents: 23328
diff changeset
  1475
3a8a763f4197 revlog: add a method to get missing revs incrementally
Siddharth Agarwal <sid0@fb.com>
parents: 23328
diff changeset
  1476
        return ancestor.incrementalmissingancestors(self.parentrevs, common)
3a8a763f4197 revlog: add a method to get missing revs incrementally
Siddharth Agarwal <sid0@fb.com>
parents: 23328
diff changeset
  1477
17972
7ef00d09ef35 revlog: add rev-specific variant of findmissing
Siddharth Agarwal <sid0@fb.com>
parents: 17971
diff changeset
  1478
    def findmissingrevs(self, common=None, heads=None):
7ef00d09ef35 revlog: add rev-specific variant of findmissing
Siddharth Agarwal <sid0@fb.com>
parents: 17971
diff changeset
  1479
        """Return the revision numbers of the ancestors of heads that
7ef00d09ef35 revlog: add rev-specific variant of findmissing
Siddharth Agarwal <sid0@fb.com>
parents: 17971
diff changeset
  1480
        are not ancestors of common.
7ef00d09ef35 revlog: add rev-specific variant of findmissing
Siddharth Agarwal <sid0@fb.com>
parents: 17971
diff changeset
  1481
7ef00d09ef35 revlog: add rev-specific variant of findmissing
Siddharth Agarwal <sid0@fb.com>
parents: 17971
diff changeset
  1482
        More specifically, return a list of revision numbers corresponding to
7ef00d09ef35 revlog: add rev-specific variant of findmissing
Siddharth Agarwal <sid0@fb.com>
parents: 17971
diff changeset
  1483
        nodes N such that every N satisfies the following constraints:
7ef00d09ef35 revlog: add rev-specific variant of findmissing
Siddharth Agarwal <sid0@fb.com>
parents: 17971
diff changeset
  1484
7ef00d09ef35 revlog: add rev-specific variant of findmissing
Siddharth Agarwal <sid0@fb.com>
parents: 17971
diff changeset
  1485
          1. N is an ancestor of some node in 'heads'
7ef00d09ef35 revlog: add rev-specific variant of findmissing
Siddharth Agarwal <sid0@fb.com>
parents: 17971
diff changeset
  1486
          2. N is not an ancestor of any node in 'common'
7ef00d09ef35 revlog: add rev-specific variant of findmissing
Siddharth Agarwal <sid0@fb.com>
parents: 17971
diff changeset
  1487
7ef00d09ef35 revlog: add rev-specific variant of findmissing
Siddharth Agarwal <sid0@fb.com>
parents: 17971
diff changeset
  1488
        The list is sorted by revision number, meaning it is
7ef00d09ef35 revlog: add rev-specific variant of findmissing
Siddharth Agarwal <sid0@fb.com>
parents: 17971
diff changeset
  1489
        topologically sorted.
7ef00d09ef35 revlog: add rev-specific variant of findmissing
Siddharth Agarwal <sid0@fb.com>
parents: 17971
diff changeset
  1490
7ef00d09ef35 revlog: add rev-specific variant of findmissing
Siddharth Agarwal <sid0@fb.com>
parents: 17971
diff changeset
  1491
        'heads' and 'common' are both lists of revision numbers.  If heads is
7ef00d09ef35 revlog: add rev-specific variant of findmissing
Siddharth Agarwal <sid0@fb.com>
parents: 17971
diff changeset
  1492
        not supplied, uses all of the revlog's heads.  If common is not
7ef00d09ef35 revlog: add rev-specific variant of findmissing
Siddharth Agarwal <sid0@fb.com>
parents: 17971
diff changeset
  1493
        supplied, uses nullid."""
7ef00d09ef35 revlog: add rev-specific variant of findmissing
Siddharth Agarwal <sid0@fb.com>
parents: 17971
diff changeset
  1494
        if common is None:
7ef00d09ef35 revlog: add rev-specific variant of findmissing
Siddharth Agarwal <sid0@fb.com>
parents: 17971
diff changeset
  1495
            common = [nullrev]
7ef00d09ef35 revlog: add rev-specific variant of findmissing
Siddharth Agarwal <sid0@fb.com>
parents: 17971
diff changeset
  1496
        if heads is None:
7ef00d09ef35 revlog: add rev-specific variant of findmissing
Siddharth Agarwal <sid0@fb.com>
parents: 17971
diff changeset
  1497
            heads = self.headrevs()
7ef00d09ef35 revlog: add rev-specific variant of findmissing
Siddharth Agarwal <sid0@fb.com>
parents: 17971
diff changeset
  1498
23338
d8f5b2f50f41 revlog: switch findmissing* methods to incrementalmissingrevs
Siddharth Agarwal <sid0@fb.com>
parents: 23337
diff changeset
  1499
        inc = self.incrementalmissingrevs(common=common)
d8f5b2f50f41 revlog: switch findmissing* methods to incrementalmissingrevs
Siddharth Agarwal <sid0@fb.com>
parents: 23337
diff changeset
  1500
        return inc.missingancestors(heads)
17972
7ef00d09ef35 revlog: add rev-specific variant of findmissing
Siddharth Agarwal <sid0@fb.com>
parents: 17971
diff changeset
  1501
13741
b51bf961b3cb wireproto: add getbundle() function
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13400
diff changeset
  1502
    def findmissing(self, common=None, heads=None):
b51bf961b3cb wireproto: add getbundle() function
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13400
diff changeset
  1503
        """Return the ancestors of heads that are not ancestors of common.
b51bf961b3cb wireproto: add getbundle() function
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13400
diff changeset
  1504
b51bf961b3cb wireproto: add getbundle() function
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13400
diff changeset
  1505
        More specifically, return a list of nodes N such that every N
b51bf961b3cb wireproto: add getbundle() function
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13400
diff changeset
  1506
        satisfies the following constraints:
b51bf961b3cb wireproto: add getbundle() function
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13400
diff changeset
  1507
b51bf961b3cb wireproto: add getbundle() function
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13400
diff changeset
  1508
          1. N is an ancestor of some node in 'heads'
b51bf961b3cb wireproto: add getbundle() function
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13400
diff changeset
  1509
          2. N is not an ancestor of any node in 'common'
b51bf961b3cb wireproto: add getbundle() function
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13400
diff changeset
  1510
b51bf961b3cb wireproto: add getbundle() function
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13400
diff changeset
  1511
        The list is sorted by revision number, meaning it is
b51bf961b3cb wireproto: add getbundle() function
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13400
diff changeset
  1512
        topologically sorted.
b51bf961b3cb wireproto: add getbundle() function
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13400
diff changeset
  1513
b51bf961b3cb wireproto: add getbundle() function
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13400
diff changeset
  1514
        'heads' and 'common' are both lists of node IDs.  If heads is
b51bf961b3cb wireproto: add getbundle() function
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13400
diff changeset
  1515
        not supplied, uses all of the revlog's heads.  If common is not
b51bf961b3cb wireproto: add getbundle() function
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13400
diff changeset
  1516
        supplied, uses nullid."""
17971
e1b9a78a7aed revlog: switch findmissing to use ancestor.missingancestors
Siddharth Agarwal <sid0@fb.com>
parents: 17951
diff changeset
  1517
        if common is None:
e1b9a78a7aed revlog: switch findmissing to use ancestor.missingancestors
Siddharth Agarwal <sid0@fb.com>
parents: 17951
diff changeset
  1518
            common = [nullid]
e1b9a78a7aed revlog: switch findmissing to use ancestor.missingancestors
Siddharth Agarwal <sid0@fb.com>
parents: 17951
diff changeset
  1519
        if heads is None:
e1b9a78a7aed revlog: switch findmissing to use ancestor.missingancestors
Siddharth Agarwal <sid0@fb.com>
parents: 17951
diff changeset
  1520
            heads = self.heads()
e1b9a78a7aed revlog: switch findmissing to use ancestor.missingancestors
Siddharth Agarwal <sid0@fb.com>
parents: 17951
diff changeset
  1521
e1b9a78a7aed revlog: switch findmissing to use ancestor.missingancestors
Siddharth Agarwal <sid0@fb.com>
parents: 17951
diff changeset
  1522
        common = [self.rev(n) for n in common]
e1b9a78a7aed revlog: switch findmissing to use ancestor.missingancestors
Siddharth Agarwal <sid0@fb.com>
parents: 17951
diff changeset
  1523
        heads = [self.rev(n) for n in heads]
e1b9a78a7aed revlog: switch findmissing to use ancestor.missingancestors
Siddharth Agarwal <sid0@fb.com>
parents: 17951
diff changeset
  1524
23338
d8f5b2f50f41 revlog: switch findmissing* methods to incrementalmissingrevs
Siddharth Agarwal <sid0@fb.com>
parents: 23337
diff changeset
  1525
        inc = self.incrementalmissingrevs(common=common)
d8f5b2f50f41 revlog: switch findmissing* methods to incrementalmissingrevs
Siddharth Agarwal <sid0@fb.com>
parents: 23337
diff changeset
  1526
        return [self.node(r) for r in inc.missingancestors(heads)]
7233
9f0e52e1df77 fix pull racing with push/commit (issue1320)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 7109
diff changeset
  1527
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1528
    def nodesbetween(self, roots=None, heads=None):
10047
27267b1f68b4 revlog: rewrite several method docstrings
Greg Ward <greg-hg@gerg.ca>
parents: 9679
diff changeset
  1529
        """Return a topological path from 'roots' to 'heads'.
27267b1f68b4 revlog: rewrite several method docstrings
Greg Ward <greg-hg@gerg.ca>
parents: 9679
diff changeset
  1530
27267b1f68b4 revlog: rewrite several method docstrings
Greg Ward <greg-hg@gerg.ca>
parents: 9679
diff changeset
  1531
        Return a tuple (nodes, outroots, outheads) where 'nodes' is a
27267b1f68b4 revlog: rewrite several method docstrings
Greg Ward <greg-hg@gerg.ca>
parents: 9679
diff changeset
  1532
        topologically sorted list of all nodes N that satisfy both of
27267b1f68b4 revlog: rewrite several method docstrings
Greg Ward <greg-hg@gerg.ca>
parents: 9679
diff changeset
  1533
        these constraints:
27267b1f68b4 revlog: rewrite several method docstrings
Greg Ward <greg-hg@gerg.ca>
parents: 9679
diff changeset
  1534
27267b1f68b4 revlog: rewrite several method docstrings
Greg Ward <greg-hg@gerg.ca>
parents: 9679
diff changeset
  1535
          1. N is a descendant of some node in 'roots'
27267b1f68b4 revlog: rewrite several method docstrings
Greg Ward <greg-hg@gerg.ca>
parents: 9679
diff changeset
  1536
          2. N is an ancestor of some node in 'heads'
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1537
10047
27267b1f68b4 revlog: rewrite several method docstrings
Greg Ward <greg-hg@gerg.ca>
parents: 9679
diff changeset
  1538
        Every node is considered to be both a descendant and an ancestor
27267b1f68b4 revlog: rewrite several method docstrings
Greg Ward <greg-hg@gerg.ca>
parents: 9679
diff changeset
  1539
        of itself, so every reachable node in 'roots' and 'heads' will be
27267b1f68b4 revlog: rewrite several method docstrings
Greg Ward <greg-hg@gerg.ca>
parents: 9679
diff changeset
  1540
        included in 'nodes'.
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1541
10047
27267b1f68b4 revlog: rewrite several method docstrings
Greg Ward <greg-hg@gerg.ca>
parents: 9679
diff changeset
  1542
        'outroots' is the list of reachable nodes in 'roots', i.e., the
27267b1f68b4 revlog: rewrite several method docstrings
Greg Ward <greg-hg@gerg.ca>
parents: 9679
diff changeset
  1543
        subset of 'roots' that is returned in 'nodes'.  Likewise,
27267b1f68b4 revlog: rewrite several method docstrings
Greg Ward <greg-hg@gerg.ca>
parents: 9679
diff changeset
  1544
        'outheads' is the subset of 'heads' that is also in 'nodes'.
27267b1f68b4 revlog: rewrite several method docstrings
Greg Ward <greg-hg@gerg.ca>
parents: 9679
diff changeset
  1545
27267b1f68b4 revlog: rewrite several method docstrings
Greg Ward <greg-hg@gerg.ca>
parents: 9679
diff changeset
  1546
        'roots' and 'heads' are both lists of node IDs.  If 'roots' is
27267b1f68b4 revlog: rewrite several method docstrings
Greg Ward <greg-hg@gerg.ca>
parents: 9679
diff changeset
  1547
        unspecified, uses nullid as the only root.  If 'heads' is
27267b1f68b4 revlog: rewrite several method docstrings
Greg Ward <greg-hg@gerg.ca>
parents: 9679
diff changeset
  1548
        unspecified, uses list of all of the revlog's heads."""
1463
26e73acc0cdf Fix to handle case of empty list for roots or heads in nodesbetween.
Eric Hopper <hopper@omnifarious.org>
parents: 1459
diff changeset
  1549
        nonodes = ([], [], [])
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1550
        if roots is not None:
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1551
            roots = list(roots)
1463
26e73acc0cdf Fix to handle case of empty list for roots or heads in nodesbetween.
Eric Hopper <hopper@omnifarious.org>
parents: 1459
diff changeset
  1552
            if not roots:
26e73acc0cdf Fix to handle case of empty list for roots or heads in nodesbetween.
Eric Hopper <hopper@omnifarious.org>
parents: 1459
diff changeset
  1553
                return nonodes
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1554
            lowestrev = min([self.rev(n) for n in roots])
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1555
        else:
14549
48ec0763afbb check-code: catch misspellings of descendant
Matt Mackall <mpm@selenic.com>
parents: 14523
diff changeset
  1556
            roots = [nullid] # Everybody's a descendant of nullid
3578
3b4e00cba57a Define and use nullrev (revision of nullid) instead of -1.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 3508
diff changeset
  1557
            lowestrev = nullrev
3b4e00cba57a Define and use nullrev (revision of nullid) instead of -1.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 3508
diff changeset
  1558
        if (lowestrev == nullrev) and (heads is None):
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1559
            # We want _all_ the nodes!
6750
fb42030d79d6 add __len__ and __iter__ methods to repo and revlog
Matt Mackall <mpm@selenic.com>
parents: 6703
diff changeset
  1560
            return ([self.node(r) for r in self], [nullid], list(self.heads()))
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1561
        if heads is None:
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1562
            # All nodes are ancestors, so the latest ancestor is the last
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1563
            # node.
6750
fb42030d79d6 add __len__ and __iter__ methods to repo and revlog
Matt Mackall <mpm@selenic.com>
parents: 6703
diff changeset
  1564
            highestrev = len(self) - 1
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1565
            # Set ancestors to None to signal that every node is an ancestor.
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1566
            ancestors = None
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1567
            # Set heads to an empty dictionary for later discovery of heads
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1568
            heads = {}
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1569
        else:
1463
26e73acc0cdf Fix to handle case of empty list for roots or heads in nodesbetween.
Eric Hopper <hopper@omnifarious.org>
parents: 1459
diff changeset
  1570
            heads = list(heads)
26e73acc0cdf Fix to handle case of empty list for roots or heads in nodesbetween.
Eric Hopper <hopper@omnifarious.org>
parents: 1459
diff changeset
  1571
            if not heads:
26e73acc0cdf Fix to handle case of empty list for roots or heads in nodesbetween.
Eric Hopper <hopper@omnifarious.org>
parents: 1459
diff changeset
  1572
                return nonodes
8464
7af92e70bb25 revlog: use set instead of dict
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 8453
diff changeset
  1573
            ancestors = set()
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1574
            # Turn heads into a dictionary so we can remove 'fake' heads.
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1575
            # Also, later we will be using it to filter out the heads we can't
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1576
            # find from roots.
14219
c33427080671 revlog: use real Booleans instead of 0/1 in nodesbetween
Martin Geisler <mg@aragost.com>
parents: 14208
diff changeset
  1577
            heads = dict.fromkeys(heads, False)
3360
ef8307585b41 nodesbetween: fix a bug with duplicate heads
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 3335
diff changeset
  1578
            # Start at the top and keep marking parents until we're done.
8163
62d7287fe6b0 rebase, revlog: use set(x) instead of set(x.keys())
Martin Geisler <mg@lazybytes.net>
parents: 8153
diff changeset
  1579
            nodestotag = set(heads)
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1580
            # Remember where the top was so we can use it as a limit later.
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1581
            highestrev = max([self.rev(n) for n in nodestotag])
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1582
            while nodestotag:
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1583
                # grab a node to tag
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1584
                n = nodestotag.pop()
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1585
                # Never tag nullid
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1586
                if n == nullid:
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1587
                    continue
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1588
                # A node's revision number represents its place in a
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1589
                # topologically sorted list of nodes.
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1590
                r = self.rev(n)
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1591
                if r >= lowestrev:
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1592
                    if n not in ancestors:
14549
48ec0763afbb check-code: catch misspellings of descendant
Matt Mackall <mpm@selenic.com>
parents: 14523
diff changeset
  1593
                        # If we are possibly a descendant of one of the roots
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1594
                        # and we haven't already been marked as an ancestor
8464
7af92e70bb25 revlog: use set instead of dict
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 8453
diff changeset
  1595
                        ancestors.add(n) # Mark as ancestor
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1596
                        # Add non-nullid parents to list of nodes to tag.
8153
616f20e1004a revlog: let nodestotag be a set instead of a list
Martin Geisler <mg@lazybytes.net>
parents: 8152
diff changeset
  1597
                        nodestotag.update([p for p in self.parents(n) if
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1598
                                           p != nullid])
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1599
                    elif n in heads: # We've seen it before, is it a fake head?
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1600
                        # So it is, real heads should not be the ancestors of
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1601
                        # any other heads.
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1602
                        heads.pop(n)
1459
106fdec8e1fb Fix small bug in nodesbetween if heads is [nullid].
Eric Hopper <hopper@omnifarious.org>
parents: 1458
diff changeset
  1603
            if not ancestors:
1463
26e73acc0cdf Fix to handle case of empty list for roots or heads in nodesbetween.
Eric Hopper <hopper@omnifarious.org>
parents: 1459
diff changeset
  1604
                return nonodes
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1605
            # Now that we have our set of ancestors, we want to remove any
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1606
            # roots that are not ancestors.
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1607
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1608
            # If one of the roots was nullid, everything is included anyway.
3578
3b4e00cba57a Define and use nullrev (revision of nullid) instead of -1.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 3508
diff changeset
  1609
            if lowestrev > nullrev:
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1610
                # But, since we weren't, let's recompute the lowest rev to not
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1611
                # include roots that aren't ancestors.
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1612
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1613
                # Filter out roots that aren't ancestors of heads
30391
2ded17b64f09 revlog: avoid shadowing several variables using list comprehensions
Augie Fackler <augie@google.com>
parents: 30289
diff changeset
  1614
                roots = [root for root in roots if root in ancestors]
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1615
                # Recompute the lowest revision
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1616
                if roots:
30391
2ded17b64f09 revlog: avoid shadowing several variables using list comprehensions
Augie Fackler <augie@google.com>
parents: 30289
diff changeset
  1617
                    lowestrev = min([self.rev(root) for root in roots])
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1618
                else:
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1619
                    # No more roots?  Return empty list
1463
26e73acc0cdf Fix to handle case of empty list for roots or heads in nodesbetween.
Eric Hopper <hopper@omnifarious.org>
parents: 1459
diff changeset
  1620
                    return nonodes
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1621
            else:
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1622
                # We are descending from nullid, and don't need to care about
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1623
                # any other roots.
3578
3b4e00cba57a Define and use nullrev (revision of nullid) instead of -1.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 3508
diff changeset
  1624
                lowestrev = nullrev
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1625
                roots = [nullid]
8152
08e1baf924ca replace set-like dictionaries with real sets
Martin Geisler <mg@lazybytes.net>
parents: 8150
diff changeset
  1626
        # Transform our roots list into a set.
14549
48ec0763afbb check-code: catch misspellings of descendant
Matt Mackall <mpm@selenic.com>
parents: 14523
diff changeset
  1627
        descendants = set(roots)
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1628
        # Also, keep the original roots so we can filter out roots that aren't
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1629
        # 'real' roots (i.e. are descended from other roots).
14549
48ec0763afbb check-code: catch misspellings of descendant
Matt Mackall <mpm@selenic.com>
parents: 14523
diff changeset
  1630
        roots = descendants.copy()
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1631
        # Our topologically sorted list of output nodes.
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1632
        orderedout = []
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1633
        # Don't start at nullid since we don't want nullid in our output list,
17483
fe3b26048140 spelling: descendants
timeless@mozdev.org
parents: 17150
diff changeset
  1634
        # and if nullid shows up in descendants, empty parents will look like
14549
48ec0763afbb check-code: catch misspellings of descendant
Matt Mackall <mpm@selenic.com>
parents: 14523
diff changeset
  1635
        # they're descendants.
17672
474047947b8f clfilter: make the revlog class responsible of all its iteration
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17537
diff changeset
  1636
        for r in self.revs(start=max(lowestrev, 0), stop=highestrev + 1):
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1637
            n = self.node(r)
14549
48ec0763afbb check-code: catch misspellings of descendant
Matt Mackall <mpm@selenic.com>
parents: 14523
diff changeset
  1638
            isdescendant = False
48ec0763afbb check-code: catch misspellings of descendant
Matt Mackall <mpm@selenic.com>
parents: 14523
diff changeset
  1639
            if lowestrev == nullrev:  # Everybody is a descendant of nullid
48ec0763afbb check-code: catch misspellings of descendant
Matt Mackall <mpm@selenic.com>
parents: 14523
diff changeset
  1640
                isdescendant = True
48ec0763afbb check-code: catch misspellings of descendant
Matt Mackall <mpm@selenic.com>
parents: 14523
diff changeset
  1641
            elif n in descendants:
48ec0763afbb check-code: catch misspellings of descendant
Matt Mackall <mpm@selenic.com>
parents: 14523
diff changeset
  1642
                # n is already a descendant
48ec0763afbb check-code: catch misspellings of descendant
Matt Mackall <mpm@selenic.com>
parents: 14523
diff changeset
  1643
                isdescendant = True
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1644
                # This check only needs to be done here because all the roots
14549
48ec0763afbb check-code: catch misspellings of descendant
Matt Mackall <mpm@selenic.com>
parents: 14523
diff changeset
  1645
                # will start being marked is descendants before the loop.
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1646
                if n in roots:
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1647
                    # If n was a root, check if it's a 'real' root.
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1648
                    p = tuple(self.parents(n))
14549
48ec0763afbb check-code: catch misspellings of descendant
Matt Mackall <mpm@selenic.com>
parents: 14523
diff changeset
  1649
                    # If any of its parents are descendants, it's not a root.
48ec0763afbb check-code: catch misspellings of descendant
Matt Mackall <mpm@selenic.com>
parents: 14523
diff changeset
  1650
                    if (p[0] in descendants) or (p[1] in descendants):
8152
08e1baf924ca replace set-like dictionaries with real sets
Martin Geisler <mg@lazybytes.net>
parents: 8150
diff changeset
  1651
                        roots.remove(n)
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1652
            else:
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1653
                p = tuple(self.parents(n))
14549
48ec0763afbb check-code: catch misspellings of descendant
Matt Mackall <mpm@selenic.com>
parents: 14523
diff changeset
  1654
                # A node is a descendant if either of its parents are
48ec0763afbb check-code: catch misspellings of descendant
Matt Mackall <mpm@selenic.com>
parents: 14523
diff changeset
  1655
                # descendants.  (We seeded the dependents list with the roots
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1656
                # up there, remember?)
14549
48ec0763afbb check-code: catch misspellings of descendant
Matt Mackall <mpm@selenic.com>
parents: 14523
diff changeset
  1657
                if (p[0] in descendants) or (p[1] in descendants):
48ec0763afbb check-code: catch misspellings of descendant
Matt Mackall <mpm@selenic.com>
parents: 14523
diff changeset
  1658
                    descendants.add(n)
48ec0763afbb check-code: catch misspellings of descendant
Matt Mackall <mpm@selenic.com>
parents: 14523
diff changeset
  1659
                    isdescendant = True
48ec0763afbb check-code: catch misspellings of descendant
Matt Mackall <mpm@selenic.com>
parents: 14523
diff changeset
  1660
            if isdescendant and ((ancestors is None) or (n in ancestors)):
48ec0763afbb check-code: catch misspellings of descendant
Matt Mackall <mpm@selenic.com>
parents: 14523
diff changeset
  1661
                # Only include nodes that are both descendants and ancestors.
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1662
                orderedout.append(n)
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1663
                if (ancestors is not None) and (n in heads):
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1664
                    # We're trying to figure out which heads are reachable
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1665
                    # from roots.
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1666
                    # Mark this head as having been reached
14219
c33427080671 revlog: use real Booleans instead of 0/1 in nodesbetween
Martin Geisler <mg@aragost.com>
parents: 14208
diff changeset
  1667
                    heads[n] = True
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1668
                elif ancestors is None:
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1669
                    # Otherwise, we're trying to discover the heads.
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1670
                    # Assume this is a head because if it isn't, the next step
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1671
                    # will eventually remove it.
14219
c33427080671 revlog: use real Booleans instead of 0/1 in nodesbetween
Martin Geisler <mg@aragost.com>
parents: 14208
diff changeset
  1672
                    heads[n] = True
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1673
                    # But, obviously its parents aren't.
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1674
                    for p in self.parents(n):
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1675
                        heads.pop(p, None)
30391
2ded17b64f09 revlog: avoid shadowing several variables using list comprehensions
Augie Fackler <augie@google.com>
parents: 30289
diff changeset
  1676
        heads = [head for head, flag in heads.iteritems() if flag]
8152
08e1baf924ca replace set-like dictionaries with real sets
Martin Geisler <mg@lazybytes.net>
parents: 8150
diff changeset
  1677
        roots = list(roots)
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1678
        assert orderedout
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1679
        assert roots
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1680
        assert heads
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1681
        return (orderedout, roots, heads)
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
  1682
14164
cb98fed52495 discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14144
diff changeset
  1683
    def headrevs(self):
16786
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16762
diff changeset
  1684
        try:
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16762
diff changeset
  1685
            return self.index.headrevs()
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16762
diff changeset
  1686
        except AttributeError:
17674
e69274f8d444 clfilter: split `revlog.headrevs` C call from python code
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17673
diff changeset
  1687
            return self._headrevs()
e69274f8d444 clfilter: split `revlog.headrevs` C call from python code
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17673
diff changeset
  1688
24444
27e3ba73fbb1 phase: default to C implementation for phase computation
Laurent Charignon <lcharignon@fb.com>
parents: 24255
diff changeset
  1689
    def computephases(self, roots):
25361
1635579f9baf phases: fix bug where native phase computation wasn't called
Laurent Charignon <lcharignon@fb.com>
parents: 25113
diff changeset
  1690
        return self.index.computephasesmapsets(roots)
24444
27e3ba73fbb1 phase: default to C implementation for phase computation
Laurent Charignon <lcharignon@fb.com>
parents: 24255
diff changeset
  1691
17674
e69274f8d444 clfilter: split `revlog.headrevs` C call from python code
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17673
diff changeset
  1692
    def _headrevs(self):
14164
cb98fed52495 discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14144
diff changeset
  1693
        count = len(self)
cb98fed52495 discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14144
diff changeset
  1694
        if not count:
cb98fed52495 discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14144
diff changeset
  1695
            return [nullrev]
17673
d686c6876ef6 clfilter: handle non contiguous iteration in `revlov.headrevs`
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17672
diff changeset
  1696
        # we won't iter over filtered rev so nobody is a head at start
d686c6876ef6 clfilter: handle non contiguous iteration in `revlov.headrevs`
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17672
diff changeset
  1697
        ishead = [0] * (count + 1)
14164
cb98fed52495 discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14144
diff changeset
  1698
        index = self.index
17672
474047947b8f clfilter: make the revlog class responsible of all its iteration
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17537
diff changeset
  1699
        for r in self:
17673
d686c6876ef6 clfilter: handle non contiguous iteration in `revlov.headrevs`
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17672
diff changeset
  1700
            ishead[r] = 1  # I may be an head
14164
cb98fed52495 discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14144
diff changeset
  1701
            e = index[r]
17673
d686c6876ef6 clfilter: handle non contiguous iteration in `revlov.headrevs`
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17672
diff changeset
  1702
            ishead[e[5]] = ishead[e[6]] = 0  # my parent are not
d686c6876ef6 clfilter: handle non contiguous iteration in `revlov.headrevs`
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17672
diff changeset
  1703
        return [r for r, val in enumerate(ishead) if val]
14164
cb98fed52495 discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14144
diff changeset
  1704
3923
27230c29bfec fix calculation of new heads added during push with -r
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 3755
diff changeset
  1705
    def heads(self, start=None, stop=None):
1550
ccb9b62de892 add a -r/--rev option to heads to show only heads descendant from rev
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1535
diff changeset
  1706
        """return the list of all nodes that have no children
1551
e793cbc8be00 Fixes to "hg heads -r FOO":
Thomas Arendsen Hein <thomas@intevation.de>
parents: 1550
diff changeset
  1707
e793cbc8be00 Fixes to "hg heads -r FOO":
Thomas Arendsen Hein <thomas@intevation.de>
parents: 1550
diff changeset
  1708
        if start is specified, only heads that are descendants of
e793cbc8be00 Fixes to "hg heads -r FOO":
Thomas Arendsen Hein <thomas@intevation.de>
parents: 1550
diff changeset
  1709
        start will be returned
3923
27230c29bfec fix calculation of new heads added during push with -r
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 3755
diff changeset
  1710
        if stop is specified, it will consider all the revs from stop
27230c29bfec fix calculation of new heads added during push with -r
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 3755
diff changeset
  1711
        as if they had no children
1551
e793cbc8be00 Fixes to "hg heads -r FOO":
Thomas Arendsen Hein <thomas@intevation.de>
parents: 1550
diff changeset
  1712
        """
4991
9c8c42bcf17a revlog: implement a fast path for heads
Matt Mackall <mpm@selenic.com>
parents: 4990
diff changeset
  1713
        if start is None and stop is None:
14164
cb98fed52495 discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14144
diff changeset
  1714
            if not len(self):
4991
9c8c42bcf17a revlog: implement a fast path for heads
Matt Mackall <mpm@selenic.com>
parents: 4990
diff changeset
  1715
                return [nullid]
14164
cb98fed52495 discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14144
diff changeset
  1716
            return [self.node(r) for r in self.headrevs()]
4991
9c8c42bcf17a revlog: implement a fast path for heads
Matt Mackall <mpm@selenic.com>
parents: 4990
diff changeset
  1717
1551
e793cbc8be00 Fixes to "hg heads -r FOO":
Thomas Arendsen Hein <thomas@intevation.de>
parents: 1550
diff changeset
  1718
        if start is None:
e793cbc8be00 Fixes to "hg heads -r FOO":
Thomas Arendsen Hein <thomas@intevation.de>
parents: 1550
diff changeset
  1719
            start = nullid
3923
27230c29bfec fix calculation of new heads added during push with -r
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 3755
diff changeset
  1720
        if stop is None:
27230c29bfec fix calculation of new heads added during push with -r
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 3755
diff changeset
  1721
            stop = []
8152
08e1baf924ca replace set-like dictionaries with real sets
Martin Geisler <mg@lazybytes.net>
parents: 8150
diff changeset
  1722
        stoprevs = set([self.rev(n) for n in stop])
1550
ccb9b62de892 add a -r/--rev option to heads to show only heads descendant from rev
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1535
diff changeset
  1723
        startrev = self.rev(start)
32291
bd872f64a8ba cleanup: use set literals
Martin von Zweigbergk <martinvonz@google.com>
parents: 32286
diff changeset
  1724
        reachable = {startrev}
bd872f64a8ba cleanup: use set literals
Martin von Zweigbergk <martinvonz@google.com>
parents: 32286
diff changeset
  1725
        heads = {startrev}
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
  1726
2490
6ff82ec1f4b8 Change revlog.heads to walk the revision graph using revision numbers
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 2489
diff changeset
  1727
        parentrevs = self.parentrevs
17672
474047947b8f clfilter: make the revlog class responsible of all its iteration
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17537
diff changeset
  1728
        for r in self.revs(start=startrev + 1):
2490
6ff82ec1f4b8 Change revlog.heads to walk the revision graph using revision numbers
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 2489
diff changeset
  1729
            for p in parentrevs(r):
6ff82ec1f4b8 Change revlog.heads to walk the revision graph using revision numbers
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 2489
diff changeset
  1730
                if p in reachable:
3923
27230c29bfec fix calculation of new heads added during push with -r
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 3755
diff changeset
  1731
                    if r not in stoprevs:
8464
7af92e70bb25 revlog: use set instead of dict
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 8453
diff changeset
  1732
                        reachable.add(r)
7af92e70bb25 revlog: use set instead of dict
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 8453
diff changeset
  1733
                    heads.add(r)
3923
27230c29bfec fix calculation of new heads added during push with -r
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 3755
diff changeset
  1734
                if p in heads and p not in stoprevs:
8464
7af92e70bb25 revlog: use set instead of dict
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 8453
diff changeset
  1735
                    heads.remove(p)
3923
27230c29bfec fix calculation of new heads added during push with -r
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 3755
diff changeset
  1736
2490
6ff82ec1f4b8 Change revlog.heads to walk the revision graph using revision numbers
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 2489
diff changeset
  1737
        return [self.node(r) for r in heads]
370
c90385d82aec revlog: add a children function
mpm@selenic.com
parents: 330
diff changeset
  1738
c90385d82aec revlog: add a children function
mpm@selenic.com
parents: 330
diff changeset
  1739
    def children(self, node):
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
  1740
        """find the children of a given node"""
370
c90385d82aec revlog: add a children function
mpm@selenic.com
parents: 330
diff changeset
  1741
        c = []
c90385d82aec revlog: add a children function
mpm@selenic.com
parents: 330
diff changeset
  1742
        p = self.rev(node)
17672
474047947b8f clfilter: make the revlog class responsible of all its iteration
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17537
diff changeset
  1743
        for r in self.revs(start=p + 1):
4746
62c56d8f368b Fix revlog.children so the real children of the null revision can be calculated.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4635
diff changeset
  1744
            prevs = [pr for pr in self.parentrevs(r) if pr != nullrev]
62c56d8f368b Fix revlog.children so the real children of the null revision can be calculated.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4635
diff changeset
  1745
            if prevs:
62c56d8f368b Fix revlog.children so the real children of the null revision can be calculated.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4635
diff changeset
  1746
                for pr in prevs:
62c56d8f368b Fix revlog.children so the real children of the null revision can be calculated.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4635
diff changeset
  1747
                    if pr == p:
62c56d8f368b Fix revlog.children so the real children of the null revision can be calculated.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4635
diff changeset
  1748
                        c.append(self.node(r))
62c56d8f368b Fix revlog.children so the real children of the null revision can be calculated.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4635
diff changeset
  1749
            elif p == nullrev:
62c56d8f368b Fix revlog.children so the real children of the null revision can be calculated.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4635
diff changeset
  1750
                c.append(self.node(r))
370
c90385d82aec revlog: add a children function
mpm@selenic.com
parents: 330
diff changeset
  1751
        return c
515
03f27b1381f9 Whitespace cleanups
mpm@selenic.com
parents: 484
diff changeset
  1752
21104
40ace21cb3a1 revlog: introduce commonancestorsheads method
Mads Kiilerich <madski@unity3d.com>
parents: 20965
diff changeset
  1753
    def commonancestorsheads(self, a, b):
40ace21cb3a1 revlog: introduce commonancestorsheads method
Mads Kiilerich <madski@unity3d.com>
parents: 20965
diff changeset
  1754
        """calculate all the heads of the common ancestors of nodes a and b"""
40ace21cb3a1 revlog: introduce commonancestorsheads method
Mads Kiilerich <madski@unity3d.com>
parents: 20965
diff changeset
  1755
        a, b = self.rev(a), self.rev(b)
38512
99f864b34451 revlog: refactor out the rev-oriented part of commonancestorheads
Boris Feld <boris.feld@octobus.net>
parents: 38511
diff changeset
  1756
        ancs = self._commonancestorsheads(a, b)
99f864b34451 revlog: refactor out the rev-oriented part of commonancestorheads
Boris Feld <boris.feld@octobus.net>
parents: 38511
diff changeset
  1757
        return pycompat.maplist(self.node, ancs)
99f864b34451 revlog: refactor out the rev-oriented part of commonancestorheads
Boris Feld <boris.feld@octobus.net>
parents: 38511
diff changeset
  1758
99f864b34451 revlog: refactor out the rev-oriented part of commonancestorheads
Boris Feld <boris.feld@octobus.net>
parents: 38511
diff changeset
  1759
    def _commonancestorsheads(self, *revs):
99f864b34451 revlog: refactor out the rev-oriented part of commonancestorheads
Boris Feld <boris.feld@octobus.net>
parents: 38511
diff changeset
  1760
        """calculate all the heads of the common ancestors of revs"""
21104
40ace21cb3a1 revlog: introduce commonancestorsheads method
Mads Kiilerich <madski@unity3d.com>
parents: 20965
diff changeset
  1761
        try:
38512
99f864b34451 revlog: refactor out the rev-oriented part of commonancestorheads
Boris Feld <boris.feld@octobus.net>
parents: 38511
diff changeset
  1762
            ancs = self.index.commonancestorsheads(*revs)
21104
40ace21cb3a1 revlog: introduce commonancestorsheads method
Mads Kiilerich <madski@unity3d.com>
parents: 20965
diff changeset
  1763
        except (AttributeError, OverflowError): # C implementation failed
38512
99f864b34451 revlog: refactor out the rev-oriented part of commonancestorheads
Boris Feld <boris.feld@octobus.net>
parents: 38511
diff changeset
  1764
            ancs = ancestor.commonancestorsheads(self.parentrevs, *revs)
99f864b34451 revlog: refactor out the rev-oriented part of commonancestorheads
Boris Feld <boris.feld@octobus.net>
parents: 38511
diff changeset
  1765
        return ancs
21104
40ace21cb3a1 revlog: introduce commonancestorsheads method
Mads Kiilerich <madski@unity3d.com>
parents: 20965
diff changeset
  1766
22381
392ae5cb8d62 revlog: introduce isancestor method for efficiently determining node lineage
Mads Kiilerich <madski@unity3d.com>
parents: 22282
diff changeset
  1767
    def isancestor(self, a, b):
38664
160da69ba1bf revlog: replace descendant(b, a) by isdescendantrev(a, b) (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 38663
diff changeset
  1768
        """return True if node a is an ancestor of node b
160da69ba1bf revlog: replace descendant(b, a) by isdescendantrev(a, b) (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 38663
diff changeset
  1769
160da69ba1bf revlog: replace descendant(b, a) by isdescendantrev(a, b) (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 38663
diff changeset
  1770
        A revision is considered an ancestor of itself."""
38514
cc3543c87de5 revlog: reuse 'descendant' implemention in 'isancestor'
Boris Feld <boris.feld@octobus.net>
parents: 38513
diff changeset
  1771
        a, b = self.rev(a), self.rev(b)
38666
a06b2b032557 revlog: introduce a isancestorrev() and use it in rebase
Martin von Zweigbergk <martinvonz@google.com>
parents: 38665
diff changeset
  1772
        return self.isancestorrev(a, b)
a06b2b032557 revlog: introduce a isancestorrev() and use it in rebase
Martin von Zweigbergk <martinvonz@google.com>
parents: 38665
diff changeset
  1773
a06b2b032557 revlog: introduce a isancestorrev() and use it in rebase
Martin von Zweigbergk <martinvonz@google.com>
parents: 38665
diff changeset
  1774
    def isancestorrev(self, a, b):
a06b2b032557 revlog: introduce a isancestorrev() and use it in rebase
Martin von Zweigbergk <martinvonz@google.com>
parents: 38665
diff changeset
  1775
        """return True if revision a is an ancestor of revision b
a06b2b032557 revlog: introduce a isancestorrev() and use it in rebase
Martin von Zweigbergk <martinvonz@google.com>
parents: 38665
diff changeset
  1776
38668
21846c94e605 revlog: delete isdescendantrev() in favor of isancestorrev()
Martin von Zweigbergk <martinvonz@google.com>
parents: 38666
diff changeset
  1777
        A revision is considered an ancestor of itself.
21846c94e605 revlog: delete isdescendantrev() in favor of isancestorrev()
Martin von Zweigbergk <martinvonz@google.com>
parents: 38666
diff changeset
  1778
21846c94e605 revlog: delete isdescendantrev() in favor of isancestorrev()
Martin von Zweigbergk <martinvonz@google.com>
parents: 38666
diff changeset
  1779
        The implementation of this is trivial but the use of
21846c94e605 revlog: delete isdescendantrev() in favor of isancestorrev()
Martin von Zweigbergk <martinvonz@google.com>
parents: 38666
diff changeset
  1780
        commonancestorsheads is not."""
21846c94e605 revlog: delete isdescendantrev() in favor of isancestorrev()
Martin von Zweigbergk <martinvonz@google.com>
parents: 38666
diff changeset
  1781
        if a == nullrev:
21846c94e605 revlog: delete isdescendantrev() in favor of isancestorrev()
Martin von Zweigbergk <martinvonz@google.com>
parents: 38666
diff changeset
  1782
            return True
21846c94e605 revlog: delete isdescendantrev() in favor of isancestorrev()
Martin von Zweigbergk <martinvonz@google.com>
parents: 38666
diff changeset
  1783
        elif a == b:
21846c94e605 revlog: delete isdescendantrev() in favor of isancestorrev()
Martin von Zweigbergk <martinvonz@google.com>
parents: 38666
diff changeset
  1784
            return True
21846c94e605 revlog: delete isdescendantrev() in favor of isancestorrev()
Martin von Zweigbergk <martinvonz@google.com>
parents: 38666
diff changeset
  1785
        elif a > b:
21846c94e605 revlog: delete isdescendantrev() in favor of isancestorrev()
Martin von Zweigbergk <martinvonz@google.com>
parents: 38666
diff changeset
  1786
            return False
21846c94e605 revlog: delete isdescendantrev() in favor of isancestorrev()
Martin von Zweigbergk <martinvonz@google.com>
parents: 38666
diff changeset
  1787
        return a in self._commonancestorsheads(a, b)
22381
392ae5cb8d62 revlog: introduce isancestor method for efficiently determining node lineage
Mads Kiilerich <madski@unity3d.com>
parents: 22282
diff changeset
  1788
21107
4a6c8b6b10d3 revlog: backout 514d32de6646 - commonancestors
Mads Kiilerich <madski@unity3d.com>
parents: 21104
diff changeset
  1789
    def ancestor(self, a, b):
22389
94f77624dbb5 comments: describe ancestor consistently - avoid 'least common ancestor'
Mads Kiilerich <madski@unity3d.com>
parents: 22381
diff changeset
  1790
        """calculate the "best" common ancestor of nodes a and b"""
21107
4a6c8b6b10d3 revlog: backout 514d32de6646 - commonancestors
Mads Kiilerich <madski@unity3d.com>
parents: 21104
diff changeset
  1791
10897
adb6a291bbdb revlog: put graph related functions together
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 10404
diff changeset
  1792
        a, b = self.rev(a), self.rev(b)
18988
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18987
diff changeset
  1793
        try:
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18987
diff changeset
  1794
            ancs = self.index.ancestors(a, b)
21107
4a6c8b6b10d3 revlog: backout 514d32de6646 - commonancestors
Mads Kiilerich <madski@unity3d.com>
parents: 21104
diff changeset
  1795
        except (AttributeError, OverflowError):
18988
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18987
diff changeset
  1796
            ancs = ancestor.ancestors(self.parentrevs, a, b)
18987
3605d4e7e618 revlog: choose a consistent ancestor when there's a tie
Bryan O'Sullivan <bryano@fb.com>
parents: 18986
diff changeset
  1797
        if ancs:
3605d4e7e618 revlog: choose a consistent ancestor when there's a tie
Bryan O'Sullivan <bryano@fb.com>
parents: 18986
diff changeset
  1798
            # choose a consistent winner when there's a tie
21107
4a6c8b6b10d3 revlog: backout 514d32de6646 - commonancestors
Mads Kiilerich <madski@unity3d.com>
parents: 21104
diff changeset
  1799
            return min(map(self.node, ancs))
18987
3605d4e7e618 revlog: choose a consistent ancestor when there's a tie
Bryan O'Sullivan <bryano@fb.com>
parents: 18986
diff changeset
  1800
        return nullid
10897
adb6a291bbdb revlog: put graph related functions together
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 10404
diff changeset
  1801
3453
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
  1802
    def _match(self, id):
16762
93f8b9565257 revlog: don't handle long for revision matching
Matt Mackall <mpm@selenic.com>
parents: 16686
diff changeset
  1803
        if isinstance(id, int):
3156
d01e4cb2f5f2 cleanups in revlog.lookup
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 3139
diff changeset
  1804
            # rev
2641
156fb1feab62 lookup should allow -1 to represent nullid (if passed an int as arg)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 2600
diff changeset
  1805
            return self.node(id)
3438
b17f9d3eda74 revlog.lookup tweaks
Matt Mackall <mpm@selenic.com>
parents: 3390
diff changeset
  1806
        if len(id) == 20:
b17f9d3eda74 revlog.lookup tweaks
Matt Mackall <mpm@selenic.com>
parents: 3390
diff changeset
  1807
            # possibly a binary node
b17f9d3eda74 revlog.lookup tweaks
Matt Mackall <mpm@selenic.com>
parents: 3390
diff changeset
  1808
            # odds of a binary node being all hex in ASCII are 1 in 10**25
b17f9d3eda74 revlog.lookup tweaks
Matt Mackall <mpm@selenic.com>
parents: 3390
diff changeset
  1809
            try:
b17f9d3eda74 revlog.lookup tweaks
Matt Mackall <mpm@selenic.com>
parents: 3390
diff changeset
  1810
                node = id
7874
d812029cda85 cleanup: drop variables for unused return values
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 7873
diff changeset
  1811
                self.rev(node) # quick search the index
3438
b17f9d3eda74 revlog.lookup tweaks
Matt Mackall <mpm@selenic.com>
parents: 3390
diff changeset
  1812
                return node
3930
01d98d68d697 Add revlog.LookupError exception, and use it instead of RevlogError.
Brendan Cully <brendan@kublai.com>
parents: 3928
diff changeset
  1813
            except LookupError:
3438
b17f9d3eda74 revlog.lookup tweaks
Matt Mackall <mpm@selenic.com>
parents: 3390
diff changeset
  1814
                pass # may be partial hex id
36
da28286bf6b7 Add smart node lookup by substring or by rev number
mpm@selenic.com
parents: 26
diff changeset
  1815
        try:
3156
d01e4cb2f5f2 cleanups in revlog.lookup
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 3139
diff changeset
  1816
            # str(rev)
36
da28286bf6b7 Add smart node lookup by substring or by rev number
mpm@selenic.com
parents: 26
diff changeset
  1817
            rev = int(id)
36721
981f328d6d16 py3: use b"%d" instead of str() to convert integers to bytes
Pulkit Goyal <7895pulkit@gmail.com>
parents: 36702
diff changeset
  1818
            if "%d" % rev != id:
4980
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
  1819
                raise ValueError
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
  1820
            if rev < 0:
6750
fb42030d79d6 add __len__ and __iter__ methods to repo and revlog
Matt Mackall <mpm@selenic.com>
parents: 6703
diff changeset
  1821
                rev = len(self) + rev
fb42030d79d6 add __len__ and __iter__ methods to repo and revlog
Matt Mackall <mpm@selenic.com>
parents: 6703
diff changeset
  1822
            if rev < 0 or rev >= len(self):
4980
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
  1823
                raise ValueError
36
da28286bf6b7 Add smart node lookup by substring or by rev number
mpm@selenic.com
parents: 26
diff changeset
  1824
            return self.node(rev)
469
e205194ca7ef Various node id lookup tweaks
mpm@selenic.com
parents: 451
diff changeset
  1825
        except (ValueError, OverflowError):
3156
d01e4cb2f5f2 cleanups in revlog.lookup
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 3139
diff changeset
  1826
            pass
3453
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
  1827
        if len(id) == 40:
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
  1828
            try:
3438
b17f9d3eda74 revlog.lookup tweaks
Matt Mackall <mpm@selenic.com>
parents: 3390
diff changeset
  1829
                # a full hex nodeid?
b17f9d3eda74 revlog.lookup tweaks
Matt Mackall <mpm@selenic.com>
parents: 3390
diff changeset
  1830
                node = bin(id)
7874
d812029cda85 cleanup: drop variables for unused return values
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 7873
diff changeset
  1831
                self.rev(node)
3157
4fe41a9e4591 optimize revlog.lookup when passed hex(node)[:...]
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 3156
diff changeset
  1832
                return node
7062
efc579fdaf69 provide nicer feedback when an unknown node id is passed to a command
Sune Foldager <cryo@cyanite.org>
parents: 6912
diff changeset
  1833
            except (TypeError, LookupError):
3453
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
  1834
                pass
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
  1835
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
  1836
    def _partialmatch(self, id):
37449
a0d71618074f revlog: detect pseudo file nodeids to raise WdirUnsupported exception
Yuya Nishihara <yuya@tcha.org>
parents: 37443
diff changeset
  1837
        # we don't care wdirfilenodeids as they should be always full hash
32684
af854b1b36f8 revlog: add support for partial matching of wdir node id
Yuya Nishihara <yuya@tcha.org>
parents: 32659
diff changeset
  1838
        maybewdir = wdirhex.startswith(id)
16665
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16533
diff changeset
  1839
        try:
30391
2ded17b64f09 revlog: avoid shadowing several variables using list comprehensions
Augie Fackler <augie@google.com>
parents: 30289
diff changeset
  1840
            partial = self.index.partialmatch(id)
2ded17b64f09 revlog: avoid shadowing several variables using list comprehensions
Augie Fackler <augie@google.com>
parents: 30289
diff changeset
  1841
            if partial and self.hasnode(partial):
32684
af854b1b36f8 revlog: add support for partial matching of wdir node id
Yuya Nishihara <yuya@tcha.org>
parents: 32659
diff changeset
  1842
                if maybewdir:
af854b1b36f8 revlog: add support for partial matching of wdir node id
Yuya Nishihara <yuya@tcha.org>
parents: 32659
diff changeset
  1843
                    # single 'ff...' match in radix tree, ambiguous with wdir
af854b1b36f8 revlog: add support for partial matching of wdir node id
Yuya Nishihara <yuya@tcha.org>
parents: 32659
diff changeset
  1844
                    raise RevlogError
30391
2ded17b64f09 revlog: avoid shadowing several variables using list comprehensions
Augie Fackler <augie@google.com>
parents: 30289
diff changeset
  1845
                return partial
32684
af854b1b36f8 revlog: add support for partial matching of wdir node id
Yuya Nishihara <yuya@tcha.org>
parents: 32659
diff changeset
  1846
            if maybewdir:
af854b1b36f8 revlog: add support for partial matching of wdir node id
Yuya Nishihara <yuya@tcha.org>
parents: 32659
diff changeset
  1847
                # no 'ff...' match in radix tree, wdir identified
af854b1b36f8 revlog: add support for partial matching of wdir node id
Yuya Nishihara <yuya@tcha.org>
parents: 32659
diff changeset
  1848
                raise error.WdirUnsupported
19471
fd1bb7c1be78 revlog: handle hidden revs in _partialmatch (issue3979)
Matt Mackall <mpm@selenic.com>
parents: 19326
diff changeset
  1849
            return None
16665
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16533
diff changeset
  1850
        except RevlogError:
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16533
diff changeset
  1851
            # parsers.c radix tree lookup gave multiple matches
29396
d0ae5b8f80dc revlog: add a fast path for "ambiguous identifier"
Jun Wu <quark@fb.com>
parents: 29339
diff changeset
  1852
            # fast path: for unfiltered changelog, radix tree is accurate
d0ae5b8f80dc revlog: add a fast path for "ambiguous identifier"
Jun Wu <quark@fb.com>
parents: 29339
diff changeset
  1853
            if not getattr(self, 'filteredrevs', None):
38841
df0873ab5c14 revlog: use specialized exception for ambiguous prefix lookup
Martin von Zweigbergk <martinvonz@google.com>
parents: 38783
diff changeset
  1854
                raise AmbiguousPrefixLookupError(id, self.indexfile,
df0873ab5c14 revlog: use specialized exception for ambiguous prefix lookup
Martin von Zweigbergk <martinvonz@google.com>
parents: 38783
diff changeset
  1855
                                                 _('ambiguous identifier'))
19471
fd1bb7c1be78 revlog: handle hidden revs in _partialmatch (issue3979)
Matt Mackall <mpm@selenic.com>
parents: 19326
diff changeset
  1856
            # fall through to slow path that filters hidden revisions
16665
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16533
diff changeset
  1857
        except (AttributeError, ValueError):
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16533
diff changeset
  1858
            # we are pure python, or key was too short to search radix tree
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16533
diff changeset
  1859
            pass
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16533
diff changeset
  1860
13258
c2661863f16f revlog: introduce a cache for partial lookups
Matt Mackall <mpm@selenic.com>
parents: 13254
diff changeset
  1861
        if id in self._pcache:
c2661863f16f revlog: introduce a cache for partial lookups
Matt Mackall <mpm@selenic.com>
parents: 13254
diff changeset
  1862
            return self._pcache[id]
c2661863f16f revlog: introduce a cache for partial lookups
Matt Mackall <mpm@selenic.com>
parents: 13254
diff changeset
  1863
37819
ee3d58b4a47f revlog: make pure version of _partialmatch() support 40-byte hex nodeids
Martin von Zweigbergk <martinvonz@google.com>
parents: 37767
diff changeset
  1864
        if len(id) <= 40:
3453
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
  1865
            try:
3438
b17f9d3eda74 revlog.lookup tweaks
Matt Mackall <mpm@selenic.com>
parents: 3390
diff changeset
  1866
                # hex(node)[:...]
9029
0001e49f1c11 compat: use // for integer division
Alejandro Santos <alejolp@alejolp.com>
parents: 8658
diff changeset
  1867
                l = len(id) // 2  # grab an even number of digits
13259
3b616dfa4b17 revlog: do revlog node->rev mapping by scanning
Matt Mackall <mpm@selenic.com>
parents: 13258
diff changeset
  1868
                prefix = bin(id[:l * 2])
3b616dfa4b17 revlog: do revlog node->rev mapping by scanning
Matt Mackall <mpm@selenic.com>
parents: 13258
diff changeset
  1869
                nl = [e[7] for e in self.index if e[7].startswith(prefix)]
19471
fd1bb7c1be78 revlog: handle hidden revs in _partialmatch (issue3979)
Matt Mackall <mpm@selenic.com>
parents: 19326
diff changeset
  1870
                nl = [n for n in nl if hex(n).startswith(id) and
fd1bb7c1be78 revlog: handle hidden revs in _partialmatch (issue3979)
Matt Mackall <mpm@selenic.com>
parents: 19326
diff changeset
  1871
                      self.hasnode(n)]
39191
5517d62c1bcc revlog: fix pure version of _partialmatch() to include nullid
Martin von Zweigbergk <martinvonz@google.com>
parents: 39157
diff changeset
  1872
                if nullhex.startswith(id):
5517d62c1bcc revlog: fix pure version of _partialmatch() to include nullid
Martin von Zweigbergk <martinvonz@google.com>
parents: 39157
diff changeset
  1873
                    nl.append(nullid)
7365
ec3aafa84d44 lookup: speed up partial lookup
Matt Mackall <mpm@selenic.com>
parents: 7363
diff changeset
  1874
                if len(nl) > 0:
32684
af854b1b36f8 revlog: add support for partial matching of wdir node id
Yuya Nishihara <yuya@tcha.org>
parents: 32659
diff changeset
  1875
                    if len(nl) == 1 and not maybewdir:
13258
c2661863f16f revlog: introduce a cache for partial lookups
Matt Mackall <mpm@selenic.com>
parents: 13254
diff changeset
  1876
                        self._pcache[id] = nl[0]
7365
ec3aafa84d44 lookup: speed up partial lookup
Matt Mackall <mpm@selenic.com>
parents: 7363
diff changeset
  1877
                        return nl[0]
38841
df0873ab5c14 revlog: use specialized exception for ambiguous prefix lookup
Martin von Zweigbergk <martinvonz@google.com>
parents: 38783
diff changeset
  1878
                    raise AmbiguousPrefixLookupError(id, self.indexfile,
df0873ab5c14 revlog: use specialized exception for ambiguous prefix lookup
Martin von Zweigbergk <martinvonz@google.com>
parents: 38783
diff changeset
  1879
                                                     _('ambiguous identifier'))
32684
af854b1b36f8 revlog: add support for partial matching of wdir node id
Yuya Nishihara <yuya@tcha.org>
parents: 32659
diff changeset
  1880
                if maybewdir:
af854b1b36f8 revlog: add support for partial matching of wdir node id
Yuya Nishihara <yuya@tcha.org>
parents: 32659
diff changeset
  1881
                    raise error.WdirUnsupported
7365
ec3aafa84d44 lookup: speed up partial lookup
Matt Mackall <mpm@selenic.com>
parents: 7363
diff changeset
  1882
                return None
36238
f574cc00831a node: make bin() be a wrapper instead of just an alias
Augie Fackler <augie@google.com>
parents: 35974
diff changeset
  1883
            except TypeError:
3453
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
  1884
                pass
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
  1885
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
  1886
    def lookup(self, id):
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
  1887
        """locate a node based on:
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
  1888
            - revision number or str(revision number)
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
  1889
            - nodeid or subset of hex nodeid
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
  1890
        """
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
  1891
        n = self._match(id)
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
  1892
        if n is not None:
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
  1893
            return n
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
  1894
        n = self._partialmatch(id)
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
  1895
        if n:
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
  1896
            return n
515
03f27b1381f9 Whitespace cleanups
mpm@selenic.com
parents: 484
diff changeset
  1897
6228
c0c4c7b1e8d3 revlog: report node and file when lookup fails
Matt Mackall <mpm@selenic.com>
parents: 6212
diff changeset
  1898
        raise LookupError(id, self.indexfile, _('no match found'))
36
da28286bf6b7 Add smart node lookup by substring or by rev number
mpm@selenic.com
parents: 26
diff changeset
  1899
37767
44d1959acb3b revlog: make shortest() take a full binary nodeid (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 37494
diff changeset
  1900
    def shortest(self, node, minlength=1):
44d1959acb3b revlog: make shortest() take a full binary nodeid (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 37494
diff changeset
  1901
        """Find the shortest unambiguous prefix that matches node."""
37863
6921d3ecadc1 shortest: rename "test" variable to "prefix"
Martin von Zweigbergk <martinvonz@google.com>
parents: 37862
diff changeset
  1902
        def isvalid(prefix):
34250
448725a2ef73 templater: extract shortest() logic from template function
Martin von Zweigbergk <martinvonz@google.com>
parents: 34148
diff changeset
  1903
            try:
37865
da083d9fafab shortest: don't keep checking for longer prefix if node doesn't exist (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 37864
diff changeset
  1904
                node = self._partialmatch(prefix)
34250
448725a2ef73 templater: extract shortest() logic from template function
Martin von Zweigbergk <martinvonz@google.com>
parents: 34148
diff changeset
  1905
            except error.RevlogError:
448725a2ef73 templater: extract shortest() logic from template function
Martin von Zweigbergk <martinvonz@google.com>
parents: 34148
diff changeset
  1906
                return False
448725a2ef73 templater: extract shortest() logic from template function
Martin von Zweigbergk <martinvonz@google.com>
parents: 34148
diff changeset
  1907
            except error.WdirUnsupported:
448725a2ef73 templater: extract shortest() logic from template function
Martin von Zweigbergk <martinvonz@google.com>
parents: 34148
diff changeset
  1908
                # single 'ff...' match
448725a2ef73 templater: extract shortest() logic from template function
Martin von Zweigbergk <martinvonz@google.com>
parents: 34148
diff changeset
  1909
                return True
37865
da083d9fafab shortest: don't keep checking for longer prefix if node doesn't exist (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 37864
diff changeset
  1910
            if node is None:
da083d9fafab shortest: don't keep checking for longer prefix if node doesn't exist (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 37864
diff changeset
  1911
                raise LookupError(node, self.indexfile, _('no node'))
37970
76e933e0ccc9 shortest: remove unnecessary check for revnum in isvalid()
Martin von Zweigbergk <martinvonz@google.com>
parents: 37969
diff changeset
  1912
            return True
34250
448725a2ef73 templater: extract shortest() logic from template function
Martin von Zweigbergk <martinvonz@google.com>
parents: 34148
diff changeset
  1913
37968
0304f22497fa revlog: use node tree (native code) for shortest() calculation
Martin von Zweigbergk <martinvonz@google.com>
parents: 37865
diff changeset
  1914
        def maybewdir(prefix):
0304f22497fa revlog: use node tree (native code) for shortest() calculation
Martin von Zweigbergk <martinvonz@google.com>
parents: 37865
diff changeset
  1915
            return all(c == 'f' for c in prefix)
0304f22497fa revlog: use node tree (native code) for shortest() calculation
Martin von Zweigbergk <martinvonz@google.com>
parents: 37865
diff changeset
  1916
37767
44d1959acb3b revlog: make shortest() take a full binary nodeid (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 37494
diff changeset
  1917
        hexnode = hex(node)
37968
0304f22497fa revlog: use node tree (native code) for shortest() calculation
Martin von Zweigbergk <martinvonz@google.com>
parents: 37865
diff changeset
  1918
0304f22497fa revlog: use node tree (native code) for shortest() calculation
Martin von Zweigbergk <martinvonz@google.com>
parents: 37865
diff changeset
  1919
        def disambiguate(hexnode, minlength):
37971
3ac950cd5978 shortest: move revnum-disambiguation out of revlog
Martin von Zweigbergk <martinvonz@google.com>
parents: 37970
diff changeset
  1920
            """Disambiguate against wdirid."""
37968
0304f22497fa revlog: use node tree (native code) for shortest() calculation
Martin von Zweigbergk <martinvonz@google.com>
parents: 37865
diff changeset
  1921
            for length in range(minlength, 41):
0304f22497fa revlog: use node tree (native code) for shortest() calculation
Martin von Zweigbergk <martinvonz@google.com>
parents: 37865
diff changeset
  1922
                prefix = hexnode[:length]
37971
3ac950cd5978 shortest: move revnum-disambiguation out of revlog
Martin von Zweigbergk <martinvonz@google.com>
parents: 37970
diff changeset
  1923
                if not maybewdir(prefix):
37968
0304f22497fa revlog: use node tree (native code) for shortest() calculation
Martin von Zweigbergk <martinvonz@google.com>
parents: 37865
diff changeset
  1924
                    return prefix
0304f22497fa revlog: use node tree (native code) for shortest() calculation
Martin von Zweigbergk <martinvonz@google.com>
parents: 37865
diff changeset
  1925
0304f22497fa revlog: use node tree (native code) for shortest() calculation
Martin von Zweigbergk <martinvonz@google.com>
parents: 37865
diff changeset
  1926
        if not getattr(self, 'filteredrevs', None):
0304f22497fa revlog: use node tree (native code) for shortest() calculation
Martin von Zweigbergk <martinvonz@google.com>
parents: 37865
diff changeset
  1927
            try:
0304f22497fa revlog: use node tree (native code) for shortest() calculation
Martin von Zweigbergk <martinvonz@google.com>
parents: 37865
diff changeset
  1928
                length = max(self.index.shortest(node), minlength)
0304f22497fa revlog: use node tree (native code) for shortest() calculation
Martin von Zweigbergk <martinvonz@google.com>
parents: 37865
diff changeset
  1929
                return disambiguate(hexnode, length)
0304f22497fa revlog: use node tree (native code) for shortest() calculation
Martin von Zweigbergk <martinvonz@google.com>
parents: 37865
diff changeset
  1930
            except RevlogError:
37969
0db7fe7c34d3 shortest: make pure code also disambigute against revnums at end
Martin von Zweigbergk <martinvonz@google.com>
parents: 37968
diff changeset
  1931
                if node != wdirid:
37968
0304f22497fa revlog: use node tree (native code) for shortest() calculation
Martin von Zweigbergk <martinvonz@google.com>
parents: 37865
diff changeset
  1932
                    raise LookupError(node, self.indexfile, _('no node'))
0304f22497fa revlog: use node tree (native code) for shortest() calculation
Martin von Zweigbergk <martinvonz@google.com>
parents: 37865
diff changeset
  1933
            except AttributeError:
0304f22497fa revlog: use node tree (native code) for shortest() calculation
Martin von Zweigbergk <martinvonz@google.com>
parents: 37865
diff changeset
  1934
                # Fall through to pure code
0304f22497fa revlog: use node tree (native code) for shortest() calculation
Martin von Zweigbergk <martinvonz@google.com>
parents: 37865
diff changeset
  1935
                pass
0304f22497fa revlog: use node tree (native code) for shortest() calculation
Martin von Zweigbergk <martinvonz@google.com>
parents: 37865
diff changeset
  1936
37969
0db7fe7c34d3 shortest: make pure code also disambigute against revnums at end
Martin von Zweigbergk <martinvonz@google.com>
parents: 37968
diff changeset
  1937
        if node == wdirid:
0db7fe7c34d3 shortest: make pure code also disambigute against revnums at end
Martin von Zweigbergk <martinvonz@google.com>
parents: 37968
diff changeset
  1938
            for length in range(minlength, 41):
0db7fe7c34d3 shortest: make pure code also disambigute against revnums at end
Martin von Zweigbergk <martinvonz@google.com>
parents: 37968
diff changeset
  1939
                prefix = hexnode[:length]
0db7fe7c34d3 shortest: make pure code also disambigute against revnums at end
Martin von Zweigbergk <martinvonz@google.com>
parents: 37968
diff changeset
  1940
                if isvalid(prefix):
0db7fe7c34d3 shortest: make pure code also disambigute against revnums at end
Martin von Zweigbergk <martinvonz@google.com>
parents: 37968
diff changeset
  1941
                    return prefix
0db7fe7c34d3 shortest: make pure code also disambigute against revnums at end
Martin von Zweigbergk <martinvonz@google.com>
parents: 37968
diff changeset
  1942
0db7fe7c34d3 shortest: make pure code also disambigute against revnums at end
Martin von Zweigbergk <martinvonz@google.com>
parents: 37968
diff changeset
  1943
        for length in range(minlength, 41):
37863
6921d3ecadc1 shortest: rename "test" variable to "prefix"
Martin von Zweigbergk <martinvonz@google.com>
parents: 37862
diff changeset
  1944
            prefix = hexnode[:length]
6921d3ecadc1 shortest: rename "test" variable to "prefix"
Martin von Zweigbergk <martinvonz@google.com>
parents: 37862
diff changeset
  1945
            if isvalid(prefix):
37969
0db7fe7c34d3 shortest: make pure code also disambigute against revnums at end
Martin von Zweigbergk <martinvonz@google.com>
parents: 37968
diff changeset
  1946
                return disambiguate(hexnode, length)
34250
448725a2ef73 templater: extract shortest() logic from template function
Martin von Zweigbergk <martinvonz@google.com>
parents: 34148
diff changeset
  1947
2890
5df3e5cf16bc Move cmp bits from filelog to revlog
Matt Mackall <mpm@selenic.com>
parents: 2859
diff changeset
  1948
    def cmp(self, node, text):
11539
a463e3c50212 cmp: document the fact that we return True if content is different
Nicolas Dumazet <nicdumz.commits@gmail.com>
parents: 11323
diff changeset
  1949
        """compare text with a given file revision
a463e3c50212 cmp: document the fact that we return True if content is different
Nicolas Dumazet <nicdumz.commits@gmail.com>
parents: 11323
diff changeset
  1950
a463e3c50212 cmp: document the fact that we return True if content is different
Nicolas Dumazet <nicdumz.commits@gmail.com>
parents: 11323
diff changeset
  1951
        returns True if text is different than what is stored.
a463e3c50212 cmp: document the fact that we return True if content is different
Nicolas Dumazet <nicdumz.commits@gmail.com>
parents: 11323
diff changeset
  1952
        """
2890
5df3e5cf16bc Move cmp bits from filelog to revlog
Matt Mackall <mpm@selenic.com>
parents: 2859
diff changeset
  1953
        p1, p2 = self.parents(node)
5df3e5cf16bc Move cmp bits from filelog to revlog
Matt Mackall <mpm@selenic.com>
parents: 2859
diff changeset
  1954
        return hash(text, p1, p2) != node
5df3e5cf16bc Move cmp bits from filelog to revlog
Matt Mackall <mpm@selenic.com>
parents: 2859
diff changeset
  1955
32227
1395f843ece4 revlog: rename internal functions containing "chunk" to use "segment"
Gregory Szorc <gregory.szorc@gmail.com>
parents: 31856
diff changeset
  1956
    def _cachesegment(self, offset, data):
27070
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
  1957
        """Add a segment to the revlog cache.
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
  1958
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
  1959
        Accepts an absolute offset and the data that is at that location.
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
  1960
        """
8316
d593922cf480 revlog: clean up the chunk caching code
Matt Mackall <mpm@selenic.com>
parents: 8315
diff changeset
  1961
        o, d = self._chunkcache
d593922cf480 revlog: clean up the chunk caching code
Matt Mackall <mpm@selenic.com>
parents: 8315
diff changeset
  1962
        # try to add to existing cache
13253
61c9bc3da402 revlog: remove lazy index
Matt Mackall <mpm@selenic.com>
parents: 13239
diff changeset
  1963
        if o + len(d) == offset and len(d) + len(data) < _chunksize:
8316
d593922cf480 revlog: clean up the chunk caching code
Matt Mackall <mpm@selenic.com>
parents: 8315
diff changeset
  1964
            self._chunkcache = o, d + data
d593922cf480 revlog: clean up the chunk caching code
Matt Mackall <mpm@selenic.com>
parents: 8315
diff changeset
  1965
        else:
d593922cf480 revlog: clean up the chunk caching code
Matt Mackall <mpm@selenic.com>
parents: 8315
diff changeset
  1966
            self._chunkcache = offset, data
d593922cf480 revlog: clean up the chunk caching code
Matt Mackall <mpm@selenic.com>
parents: 8315
diff changeset
  1967
32227
1395f843ece4 revlog: rename internal functions containing "chunk" to use "segment"
Gregory Szorc <gregory.szorc@gmail.com>
parents: 31856
diff changeset
  1968
    def _readsegment(self, offset, length, df=None):
27070
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
  1969
        """Load a segment of raw data from the revlog.
26377
dfef0d3be65e revlog: support using an existing file handle when reading revlogs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26376
diff changeset
  1970
27070
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
  1971
        Accepts an absolute offset, length to read, and an optional existing
26377
dfef0d3be65e revlog: support using an existing file handle when reading revlogs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26376
diff changeset
  1972
        file handle to read from.
dfef0d3be65e revlog: support using an existing file handle when reading revlogs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26376
diff changeset
  1973
dfef0d3be65e revlog: support using an existing file handle when reading revlogs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26376
diff changeset
  1974
        If an existing file handle is passed, it will be seeked and the
dfef0d3be65e revlog: support using an existing file handle when reading revlogs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26376
diff changeset
  1975
        original seek position will NOT be restored.
27070
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
  1976
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
  1977
        Returns a str or buffer of raw byte data.
26377
dfef0d3be65e revlog: support using an existing file handle when reading revlogs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26376
diff changeset
  1978
        """
20179
5bb3826bdac4 revlog: read/cache chunks in fixed windows of 64 KB
Brodie Rao <brodie@sf.io>
parents: 20074
diff changeset
  1979
        # Cache data both forward and backward around the requested
5bb3826bdac4 revlog: read/cache chunks in fixed windows of 64 KB
Brodie Rao <brodie@sf.io>
parents: 20074
diff changeset
  1980
        # data, in a fixed size window. This helps speed up operations
5bb3826bdac4 revlog: read/cache chunks in fixed windows of 64 KB
Brodie Rao <brodie@sf.io>
parents: 20074
diff changeset
  1981
        # involving reading the revlog backwards.
20180
969148b49fc6 revlog: allow tuning of the chunk cache size (via format.chunkcachesize)
Brodie Rao <brodie@sf.io>
parents: 20179
diff changeset
  1982
        cachesize = self._chunkcachesize
969148b49fc6 revlog: allow tuning of the chunk cache size (via format.chunkcachesize)
Brodie Rao <brodie@sf.io>
parents: 20179
diff changeset
  1983
        realoffset = offset & ~(cachesize - 1)
969148b49fc6 revlog: allow tuning of the chunk cache size (via format.chunkcachesize)
Brodie Rao <brodie@sf.io>
parents: 20179
diff changeset
  1984
        reallength = (((offset + length + cachesize) & ~(cachesize - 1))
969148b49fc6 revlog: allow tuning of the chunk cache size (via format.chunkcachesize)
Brodie Rao <brodie@sf.io>
parents: 20179
diff changeset
  1985
                      - realoffset)
35973
4d66993bdcff revlog: add a _datareadfp context manager for data access needs
Boris Feld <boris.feld@octobus.net>
parents: 35972
diff changeset
  1986
        with self._datareadfp(df) as df:
4d66993bdcff revlog: add a _datareadfp context manager for data access needs
Boris Feld <boris.feld@octobus.net>
parents: 35972
diff changeset
  1987
            df.seek(realoffset)
4d66993bdcff revlog: add a _datareadfp context manager for data access needs
Boris Feld <boris.feld@octobus.net>
parents: 35972
diff changeset
  1988
            d = df.read(reallength)
32227
1395f843ece4 revlog: rename internal functions containing "chunk" to use "segment"
Gregory Szorc <gregory.szorc@gmail.com>
parents: 31856
diff changeset
  1989
        self._cachesegment(realoffset, d)
20179
5bb3826bdac4 revlog: read/cache chunks in fixed windows of 64 KB
Brodie Rao <brodie@sf.io>
parents: 20074
diff changeset
  1990
        if offset != realoffset or reallength != length:
5bb3826bdac4 revlog: read/cache chunks in fixed windows of 64 KB
Brodie Rao <brodie@sf.io>
parents: 20074
diff changeset
  1991
            return util.buffer(d, offset - realoffset, length)
8316
d593922cf480 revlog: clean up the chunk caching code
Matt Mackall <mpm@selenic.com>
parents: 8315
diff changeset
  1992
        return d
d593922cf480 revlog: clean up the chunk caching code
Matt Mackall <mpm@selenic.com>
parents: 8315
diff changeset
  1993
32227
1395f843ece4 revlog: rename internal functions containing "chunk" to use "segment"
Gregory Szorc <gregory.szorc@gmail.com>
parents: 31856
diff changeset
  1994
    def _getsegment(self, offset, length, df=None):
27070
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
  1995
        """Obtain a segment of raw data from the revlog.
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
  1996
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
  1997
        Accepts an absolute offset, length of bytes to obtain, and an
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
  1998
        optional file handle to the already-opened revlog. If the file
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
  1999
        handle is used, it's original seek position will not be preserved.
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
  2000
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
  2001
        Requests for data may be returned from a cache.
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
  2002
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
  2003
        Returns a str or a buffer instance of raw byte data.
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
  2004
        """
8316
d593922cf480 revlog: clean up the chunk caching code
Matt Mackall <mpm@selenic.com>
parents: 8315
diff changeset
  2005
        o, d = self._chunkcache
d593922cf480 revlog: clean up the chunk caching code
Matt Mackall <mpm@selenic.com>
parents: 8315
diff changeset
  2006
        l = len(d)
d593922cf480 revlog: clean up the chunk caching code
Matt Mackall <mpm@selenic.com>
parents: 8315
diff changeset
  2007
d593922cf480 revlog: clean up the chunk caching code
Matt Mackall <mpm@selenic.com>
parents: 8315
diff changeset
  2008
        # is it in the cache?
d593922cf480 revlog: clean up the chunk caching code
Matt Mackall <mpm@selenic.com>
parents: 8315
diff changeset
  2009
        cachestart = offset - o
d593922cf480 revlog: clean up the chunk caching code
Matt Mackall <mpm@selenic.com>
parents: 8315
diff changeset
  2010
        cacheend = cachestart + length
d593922cf480 revlog: clean up the chunk caching code
Matt Mackall <mpm@selenic.com>
parents: 8315
diff changeset
  2011
        if cachestart >= 0 and cacheend <= l:
d593922cf480 revlog: clean up the chunk caching code
Matt Mackall <mpm@selenic.com>
parents: 8315
diff changeset
  2012
            if cachestart == 0 and cacheend == l:
d593922cf480 revlog: clean up the chunk caching code
Matt Mackall <mpm@selenic.com>
parents: 8315
diff changeset
  2013
                return d # avoid a copy
16423
a150923b49ba revlog: avoid an expensive string copy
Bryan O'Sullivan <bryano@fb.com>
parents: 16418
diff changeset
  2014
            return util.buffer(d, cachestart, cacheend - cachestart)
8316
d593922cf480 revlog: clean up the chunk caching code
Matt Mackall <mpm@selenic.com>
parents: 8315
diff changeset
  2015
32227
1395f843ece4 revlog: rename internal functions containing "chunk" to use "segment"
Gregory Szorc <gregory.szorc@gmail.com>
parents: 31856
diff changeset
  2016
        return self._readsegment(offset, length, df=df)
8316
d593922cf480 revlog: clean up the chunk caching code
Matt Mackall <mpm@selenic.com>
parents: 8315
diff changeset
  2017
32229
75e93d95aae6 revlog: rename _chunkraw to _getsegmentforrevs()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32227
diff changeset
  2018
    def _getsegmentforrevs(self, startrev, endrev, df=None):
27070
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
  2019
        """Obtain a segment of raw data corresponding to a range of revisions.
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
  2020
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
  2021
        Accepts the start and end revisions and an optional already-open
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
  2022
        file handle to be used for reading. If the file handle is read, its
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
  2023
        seek position will not be preserved.
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
  2024
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
  2025
        Requests for data may be satisfied by a cache.
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
  2026
27649
6446e9b37c8b revlog: return offset from _chunkraw()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27637
diff changeset
  2027
        Returns a 2-tuple of (offset, data) for the requested range of
6446e9b37c8b revlog: return offset from _chunkraw()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27637
diff changeset
  2028
        revisions. Offset is the integer offset from the beginning of the
6446e9b37c8b revlog: return offset from _chunkraw()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27637
diff changeset
  2029
        revlog and data is a str or buffer of the raw byte data.
6446e9b37c8b revlog: return offset from _chunkraw()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27637
diff changeset
  2030
6446e9b37c8b revlog: return offset from _chunkraw()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27637
diff changeset
  2031
        Callers will need to call ``self.start(rev)`` and ``self.length(rev)``
6446e9b37c8b revlog: return offset from _chunkraw()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27637
diff changeset
  2032
        to determine where each revision's data begins and ends.
27070
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
  2033
        """
30288
ceddc3d94d74 revlog: inline start() and end() for perf reasons
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30287
diff changeset
  2034
        # Inlined self.start(startrev) & self.end(endrev) for perf reasons
ceddc3d94d74 revlog: inline start() and end() for perf reasons
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30287
diff changeset
  2035
        # (functions are expensive).
ceddc3d94d74 revlog: inline start() and end() for perf reasons
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30287
diff changeset
  2036
        index = self.index
ceddc3d94d74 revlog: inline start() and end() for perf reasons
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30287
diff changeset
  2037
        istart = index[startrev]
ceddc3d94d74 revlog: inline start() and end() for perf reasons
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30287
diff changeset
  2038
        start = int(istart[0] >> 16)
30289
1f92056c4066 revlog: optimize _chunkraw when startrev==endrev
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30288
diff changeset
  2039
        if startrev == endrev:
1f92056c4066 revlog: optimize _chunkraw when startrev==endrev
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30288
diff changeset
  2040
            end = start + istart[1]
1f92056c4066 revlog: optimize _chunkraw when startrev==endrev
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30288
diff changeset
  2041
        else:
1f92056c4066 revlog: optimize _chunkraw when startrev==endrev
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30288
diff changeset
  2042
            iend = index[endrev]
1f92056c4066 revlog: optimize _chunkraw when startrev==endrev
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30288
diff changeset
  2043
            end = int(iend[0] >> 16) + iend[1]
30288
ceddc3d94d74 revlog: inline start() and end() for perf reasons
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30287
diff changeset
  2044
8318
6b8513f8274a revlog: add cache priming for reconstructing delta chains
Matt Mackall <mpm@selenic.com>
parents: 8317
diff changeset
  2045
        if self._inline:
6b8513f8274a revlog: add cache priming for reconstructing delta chains
Matt Mackall <mpm@selenic.com>
parents: 8317
diff changeset
  2046
            start += (startrev + 1) * self._io.size
19714
0e07c0b5fb1c revlog.revision: fix cache preload for inline revlogs
Siddharth Agarwal <sid0@fb.com>
parents: 19713
diff changeset
  2047
            end += (endrev + 1) * self._io.size
0e07c0b5fb1c revlog.revision: fix cache preload for inline revlogs
Siddharth Agarwal <sid0@fb.com>
parents: 19713
diff changeset
  2048
        length = end - start
27649
6446e9b37c8b revlog: return offset from _chunkraw()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27637
diff changeset
  2049
32227
1395f843ece4 revlog: rename internal functions containing "chunk" to use "segment"
Gregory Szorc <gregory.szorc@gmail.com>
parents: 31856
diff changeset
  2050
        return start, self._getsegment(start, length, df=df)
8318
6b8513f8274a revlog: add cache priming for reconstructing delta chains
Matt Mackall <mpm@selenic.com>
parents: 8317
diff changeset
  2051
26377
dfef0d3be65e revlog: support using an existing file handle when reading revlogs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26376
diff changeset
  2052
    def _chunk(self, rev, df=None):
27070
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
  2053
        """Obtain a single decompressed chunk for a revision.
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
  2054
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
  2055
        Accepts an integer revision and an optional already-open file handle
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
  2056
        to be used for reading. If used, the seek position of the file will not
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
  2057
        be preserved.
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
  2058
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
  2059
        Returns a str holding uncompressed data for the requested revision.
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
  2060
        """
32229
75e93d95aae6 revlog: rename _chunkraw to _getsegmentforrevs()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32227
diff changeset
  2061
        return self.decompress(self._getsegmentforrevs(rev, rev, df=df)[1])
8650
ef393d6ec030 revlog: refactor chunk cache interface again
Matt Mackall <mpm@selenic.com>
parents: 8643
diff changeset
  2062
38644
43d0619cec90 revlog: enforce chunk slicing down to a certain size
Boris Feld <boris.feld@octobus.net>
parents: 38643
diff changeset
  2063
    def _chunks(self, revs, df=None, targetsize=None):
27070
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
  2064
        """Obtain decompressed chunks for the specified revisions.
19713
c2e27e57d250 revlog: add a fast method for getting a list of chunks
Siddharth Agarwal <sid0@fb.com>
parents: 19625
diff changeset
  2065
27070
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
  2066
        Accepts an iterable of numeric revisions that are assumed to be in
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
  2067
        ascending order. Also accepts an optional already-open file handle
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
  2068
        to be used for reading. If used, the seek position of the file will
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
  2069
        not be preserved.
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
  2070
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
  2071
        This function is similar to calling ``self._chunk()`` multiple times,
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
  2072
        but is faster.
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
  2073
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
  2074
        Returns a list with decompressed data for each requested revision.
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
  2075
        """
19716
e17976978ee4 revlog: move chunk cache preload from revision to _chunks
Siddharth Agarwal <sid0@fb.com>
parents: 19715
diff changeset
  2076
        if not revs:
e17976978ee4 revlog: move chunk cache preload from revision to _chunks
Siddharth Agarwal <sid0@fb.com>
parents: 19715
diff changeset
  2077
            return []
19713
c2e27e57d250 revlog: add a fast method for getting a list of chunks
Siddharth Agarwal <sid0@fb.com>
parents: 19625
diff changeset
  2078
        start = self.start
c2e27e57d250 revlog: add a fast method for getting a list of chunks
Siddharth Agarwal <sid0@fb.com>
parents: 19625
diff changeset
  2079
        length = self.length
c2e27e57d250 revlog: add a fast method for getting a list of chunks
Siddharth Agarwal <sid0@fb.com>
parents: 19625
diff changeset
  2080
        inline = self._inline
c2e27e57d250 revlog: add a fast method for getting a list of chunks
Siddharth Agarwal <sid0@fb.com>
parents: 19625
diff changeset
  2081
        iosize = self._io.size
19715
1aab406be57c revlog._chunks: inline getchunk
Siddharth Agarwal <sid0@fb.com>
parents: 19714
diff changeset
  2082
        buffer = util.buffer
19713
c2e27e57d250 revlog: add a fast method for getting a list of chunks
Siddharth Agarwal <sid0@fb.com>
parents: 19625
diff changeset
  2083
c2e27e57d250 revlog: add a fast method for getting a list of chunks
Siddharth Agarwal <sid0@fb.com>
parents: 19625
diff changeset
  2084
        l = []
c2e27e57d250 revlog: add a fast method for getting a list of chunks
Siddharth Agarwal <sid0@fb.com>
parents: 19625
diff changeset
  2085
        ladd = l.append
c2e27e57d250 revlog: add a fast method for getting a list of chunks
Siddharth Agarwal <sid0@fb.com>
parents: 19625
diff changeset
  2086
34824
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
  2087
        if not self._withsparseread:
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
  2088
            slicedchunks = (revs,)
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
  2089
        else:
38718
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents: 38717
diff changeset
  2090
            slicedchunks = _slicechunk(self, revs, targetsize=targetsize)
34824
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
  2091
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
  2092
        for revschunk in slicedchunks:
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
  2093
            firstrev = revschunk[0]
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
  2094
            # Skip trailing revisions with empty diff
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
  2095
            for lastrev in revschunk[::-1]:
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
  2096
                if length(lastrev) != 0:
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
  2097
                    break
34823
7891d243d821 revlog: ignore empty trailing chunks when reading segments
Paul Morelle <paul.morelle@octobus.net>
parents: 34296
diff changeset
  2098
34824
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
  2099
            try:
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
  2100
                offset, data = self._getsegmentforrevs(firstrev, lastrev, df=df)
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
  2101
            except OverflowError:
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
  2102
                # issue4215 - we can't cache a run of chunks greater than
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
  2103
                # 2G on Windows
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
  2104
                return [self._chunk(rev, df=df) for rev in revschunk]
19715
1aab406be57c revlog._chunks: inline getchunk
Siddharth Agarwal <sid0@fb.com>
parents: 19714
diff changeset
  2105
34824
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
  2106
            decomp = self.decompress
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
  2107
            for rev in revschunk:
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
  2108
                chunkstart = start(rev)
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
  2109
                if inline:
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
  2110
                    chunkstart += (rev + 1) * iosize
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
  2111
                chunklength = length(rev)
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse
Paul Morelle <paul.morelle@octobus.net>
parents: 34823
diff changeset
  2112
                ladd(decomp(buffer(data, chunkstart - offset, chunklength)))
19713
c2e27e57d250 revlog: add a fast method for getting a list of chunks
Siddharth Agarwal <sid0@fb.com>
parents: 19625
diff changeset
  2113
c2e27e57d250 revlog: add a fast method for getting a list of chunks
Siddharth Agarwal <sid0@fb.com>
parents: 19625
diff changeset
  2114
        return l
14075
bc101902a68d revlog: introduce _chunkbase to allow filelog to override
Sune Foldager <cryo@cyanite.org>
parents: 14064
diff changeset
  2115
8650
ef393d6ec030 revlog: refactor chunk cache interface again
Matt Mackall <mpm@selenic.com>
parents: 8643
diff changeset
  2116
    def _chunkclear(self):
27070
7860366b46c9 revlog: improve documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26907
diff changeset
  2117
        """Clear the raw chunk cache."""
8650
ef393d6ec030 revlog: refactor chunk cache interface again
Matt Mackall <mpm@selenic.com>
parents: 8643
diff changeset
  2118
        self._chunkcache = (0, '')
1598
14d1f1868bf6 cleanup of revlog.group when repository is local
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1559
diff changeset
  2119
11929
1839a7518b0d revlog: deltachain() returns chain of revs need to construct a revision
Pradeepkumar Gayam <in3xes@gmail.com>
parents: 11928
diff changeset
  2120
    def deltaparent(self, rev):
14195
0013d3eeb826 revlog: remove support for parentdelta
Sune Foldager <cryo@cyanite.org>
parents: 14164
diff changeset
  2121
        """return deltaparent of the given revision"""
14253
c28d5200374c revlog: support reading generaldelta revlogs
Sune Foldager <cryo@cyanite.org>
parents: 14252
diff changeset
  2122
        base = self.index[rev][3]
c28d5200374c revlog: support reading generaldelta revlogs
Sune Foldager <cryo@cyanite.org>
parents: 14252
diff changeset
  2123
        if base == rev:
14208
d62d597b8974 revlog: compute correct deltaparent in the deltaparent function
Sune Foldager <cryo@cyanite.org>
parents: 14196
diff changeset
  2124
            return nullrev
14253
c28d5200374c revlog: support reading generaldelta revlogs
Sune Foldager <cryo@cyanite.org>
parents: 14252
diff changeset
  2125
        elif self._generaldelta:
c28d5200374c revlog: support reading generaldelta revlogs
Sune Foldager <cryo@cyanite.org>
parents: 14252
diff changeset
  2126
            return base
14208
d62d597b8974 revlog: compute correct deltaparent in the deltaparent function
Sune Foldager <cryo@cyanite.org>
parents: 14196
diff changeset
  2127
        else:
d62d597b8974 revlog: compute correct deltaparent in the deltaparent function
Sune Foldager <cryo@cyanite.org>
parents: 14196
diff changeset
  2128
            return rev - 1
11929
1839a7518b0d revlog: deltachain() returns chain of revs need to construct a revision
Pradeepkumar Gayam <in3xes@gmail.com>
parents: 11928
diff changeset
  2129
39149
f8db458651c8 revlog: add a method to tells whether rev is stored as a snapshot
Paul Morelle <paul.morelle@octobus.net>
parents: 39147
diff changeset
  2130
    def issnapshot(self, rev):
f8db458651c8 revlog: add a method to tells whether rev is stored as a snapshot
Paul Morelle <paul.morelle@octobus.net>
parents: 39147
diff changeset
  2131
        """tells whether rev is a snapshot
f8db458651c8 revlog: add a method to tells whether rev is stored as a snapshot
Paul Morelle <paul.morelle@octobus.net>
parents: 39147
diff changeset
  2132
        """
f8db458651c8 revlog: add a method to tells whether rev is stored as a snapshot
Paul Morelle <paul.morelle@octobus.net>
parents: 39147
diff changeset
  2133
        if rev == nullrev:
f8db458651c8 revlog: add a method to tells whether rev is stored as a snapshot
Paul Morelle <paul.morelle@octobus.net>
parents: 39147
diff changeset
  2134
            return True
f8db458651c8 revlog: add a method to tells whether rev is stored as a snapshot
Paul Morelle <paul.morelle@octobus.net>
parents: 39147
diff changeset
  2135
        deltap = self.deltaparent(rev)
f8db458651c8 revlog: add a method to tells whether rev is stored as a snapshot
Paul Morelle <paul.morelle@octobus.net>
parents: 39147
diff changeset
  2136
        if deltap == nullrev:
f8db458651c8 revlog: add a method to tells whether rev is stored as a snapshot
Paul Morelle <paul.morelle@octobus.net>
parents: 39147
diff changeset
  2137
            return True
39150
f39efa885a6d revlog: also detect intermediate snapshots
Paul Morelle <paul.morelle@octobus.net>
parents: 39149
diff changeset
  2138
        p1, p2 = self.parentrevs(rev)
f39efa885a6d revlog: also detect intermediate snapshots
Paul Morelle <paul.morelle@octobus.net>
parents: 39149
diff changeset
  2139
        if deltap in (p1, p2):
f39efa885a6d revlog: also detect intermediate snapshots
Paul Morelle <paul.morelle@octobus.net>
parents: 39149
diff changeset
  2140
            return False
f39efa885a6d revlog: also detect intermediate snapshots
Paul Morelle <paul.morelle@octobus.net>
parents: 39149
diff changeset
  2141
        return self.issnapshot(deltap)
39149
f8db458651c8 revlog: add a method to tells whether rev is stored as a snapshot
Paul Morelle <paul.morelle@octobus.net>
parents: 39147
diff changeset
  2142
39152
3b1042cab4b4 revlog: add a method to retrieve snapshot depth
Boris Feld <boris.feld@octobus.net>
parents: 39150
diff changeset
  2143
    def snapshotdepth(self, rev):
3b1042cab4b4 revlog: add a method to retrieve snapshot depth
Boris Feld <boris.feld@octobus.net>
parents: 39150
diff changeset
  2144
        """number of snapshot in the chain before this one"""
3b1042cab4b4 revlog: add a method to retrieve snapshot depth
Boris Feld <boris.feld@octobus.net>
parents: 39150
diff changeset
  2145
        if not self.issnapshot(rev):
3b1042cab4b4 revlog: add a method to retrieve snapshot depth
Boris Feld <boris.feld@octobus.net>
parents: 39150
diff changeset
  2146
            raise ProgrammingError('revision %d not a snapshot')
3b1042cab4b4 revlog: add a method to retrieve snapshot depth
Boris Feld <boris.feld@octobus.net>
parents: 39150
diff changeset
  2147
        return len(self._deltachain(rev)[0]) - 1
3b1042cab4b4 revlog: add a method to retrieve snapshot depth
Boris Feld <boris.feld@octobus.net>
parents: 39150
diff changeset
  2148
1941
7518823709a2 revlog.py: factorization and fixes for rev < 0 (nullid)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1853
diff changeset
  2149
    def revdiff(self, rev1, rev2):
31753
5d11b5edcb0b revlog: use raw revisions in revdiff
Jun Wu <quark@fb.com>
parents: 31752
diff changeset
  2150
        """return or calculate a delta between two revisions
5d11b5edcb0b revlog: use raw revisions in revdiff
Jun Wu <quark@fb.com>
parents: 31752
diff changeset
  2151
5d11b5edcb0b revlog: use raw revisions in revdiff
Jun Wu <quark@fb.com>
parents: 31752
diff changeset
  2152
        The delta calculated is in binary form and is intended to be written to
5d11b5edcb0b revlog: use raw revisions in revdiff
Jun Wu <quark@fb.com>
parents: 31752
diff changeset
  2153
        revlog data directly. So this function needs raw revision data.
5d11b5edcb0b revlog: use raw revisions in revdiff
Jun Wu <quark@fb.com>
parents: 31752
diff changeset
  2154
        """
14208
d62d597b8974 revlog: compute correct deltaparent in the deltaparent function
Sune Foldager <cryo@cyanite.org>
parents: 14196
diff changeset
  2155
        if rev1 != nullrev and self.deltaparent(rev2) == rev1:
31369
b6f5af372c0c revlog: use bytes() instead of str() to get data from memoryview
Augie Fackler <augie@google.com>
parents: 31357
diff changeset
  2156
            return bytes(self._chunk(rev2))
5005
72082bfced9a revlog: minor revdiff reorganization
Matt Mackall <mpm@selenic.com>
parents: 5004
diff changeset
  2157
31753
5d11b5edcb0b revlog: use raw revisions in revdiff
Jun Wu <quark@fb.com>
parents: 31752
diff changeset
  2158
        return mdiff.textdiff(self.revision(rev1, raw=True),
5d11b5edcb0b revlog: use raw revisions in revdiff
Jun Wu <quark@fb.com>
parents: 31752
diff changeset
  2159
                              self.revision(rev2, raw=True))
119
c7a66f9752a4 Add code to retrieve or construct a revlog delta
mpm@selenic.com
parents: 117
diff changeset
  2160
30743
2df983125d37 revlog: add 'raw' argument to revision and _addrevision
Remi Chaintron <remi@fb.com>
parents: 30584
diff changeset
  2161
    def revision(self, nodeorrev, _df=None, raw=False):
16435
df347129305d revlog: fix partial revision() docstring (from d7d64b89a65c)
Patrick Mezard <patrick@mezard.eu>
parents: 16424
diff changeset
  2162
        """return an uncompressed revision of a given node or revision
df347129305d revlog: fix partial revision() docstring (from d7d64b89a65c)
Patrick Mezard <patrick@mezard.eu>
parents: 16424
diff changeset
  2163
        number.
26377
dfef0d3be65e revlog: support using an existing file handle when reading revlogs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26376
diff changeset
  2164
30743
2df983125d37 revlog: add 'raw' argument to revision and _addrevision
Remi Chaintron <remi@fb.com>
parents: 30584
diff changeset
  2165
        _df - an existing file handle to read from. (internal-only)
2df983125d37 revlog: add 'raw' argument to revision and _addrevision
Remi Chaintron <remi@fb.com>
parents: 30584
diff changeset
  2166
        raw - an optional argument specifying if the revision data is to be
2df983125d37 revlog: add 'raw' argument to revision and _addrevision
Remi Chaintron <remi@fb.com>
parents: 30584
diff changeset
  2167
        treated as raw data when applying flag transforms. 'raw' should be set
2df983125d37 revlog: add 'raw' argument to revision and _addrevision
Remi Chaintron <remi@fb.com>
parents: 30584
diff changeset
  2168
        to True when generating changegroups or in debug commands.
16435
df347129305d revlog: fix partial revision() docstring (from d7d64b89a65c)
Patrick Mezard <patrick@mezard.eu>
parents: 16424
diff changeset
  2169
        """
16375
d7d64b89a65c revlog: allow retrieving contents by revision number
Matt Mackall <mpm@selenic.com>
parents: 16374
diff changeset
  2170
        if isinstance(nodeorrev, int):
d7d64b89a65c revlog: allow retrieving contents by revision number
Matt Mackall <mpm@selenic.com>
parents: 16374
diff changeset
  2171
            rev = nodeorrev
d7d64b89a65c revlog: allow retrieving contents by revision number
Matt Mackall <mpm@selenic.com>
parents: 16374
diff changeset
  2172
            node = self.node(rev)
d7d64b89a65c revlog: allow retrieving contents by revision number
Matt Mackall <mpm@selenic.com>
parents: 16374
diff changeset
  2173
        else:
d7d64b89a65c revlog: allow retrieving contents by revision number
Matt Mackall <mpm@selenic.com>
parents: 16374
diff changeset
  2174
            node = nodeorrev
d7d64b89a65c revlog: allow retrieving contents by revision number
Matt Mackall <mpm@selenic.com>
parents: 16374
diff changeset
  2175
            rev = None
d7d64b89a65c revlog: allow retrieving contents by revision number
Matt Mackall <mpm@selenic.com>
parents: 16374
diff changeset
  2176
11996
3195cf01dfb9 revlog.revision(): don't use nullrev as the default value for the cache
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 11995
diff changeset
  2177
        cachedrev = None
31802
ac9a5e89113a revlog: avoid calculating "flags" twice in revision()
Jun Wu <quark@fb.com>
parents: 31801
diff changeset
  2178
        flags = None
31804
726f24123f02 revlog: avoid applying delta chain on cache hit
Jun Wu <quark@fb.com>
parents: 31803
diff changeset
  2179
        rawtext = None
4980
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
  2180
        if node == nullid:
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
  2181
            return ""
26242
d708873f1f33 revlog: drop local assignment of cache variable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26241
diff changeset
  2182
        if self._cache:
d708873f1f33 revlog: drop local assignment of cache variable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26241
diff changeset
  2183
            if self._cache[0] == node:
31751
2133437dad17 revlog: fix _cache usage in revision()
Jun Wu <quark@fb.com>
parents: 31750
diff changeset
  2184
                # _cache only stores rawtext
2133437dad17 revlog: fix _cache usage in revision()
Jun Wu <quark@fb.com>
parents: 31750
diff changeset
  2185
                if raw:
2133437dad17 revlog: fix _cache usage in revision()
Jun Wu <quark@fb.com>
parents: 31750
diff changeset
  2186
                    return self._cache[2]
31756
9ec03d5af48f revlog: add a fast path for revision(raw=False)
Jun Wu <quark@fb.com>
parents: 31755
diff changeset
  2187
                # duplicated, but good for perf
9ec03d5af48f revlog: add a fast path for revision(raw=False)
Jun Wu <quark@fb.com>
parents: 31755
diff changeset
  2188
                if rev is None:
9ec03d5af48f revlog: add a fast path for revision(raw=False)
Jun Wu <quark@fb.com>
parents: 31755
diff changeset
  2189
                    rev = self.rev(node)
31802
ac9a5e89113a revlog: avoid calculating "flags" twice in revision()
Jun Wu <quark@fb.com>
parents: 31801
diff changeset
  2190
                if flags is None:
ac9a5e89113a revlog: avoid calculating "flags" twice in revision()
Jun Wu <quark@fb.com>
parents: 31801
diff changeset
  2191
                    flags = self.flags(rev)
31756
9ec03d5af48f revlog: add a fast path for revision(raw=False)
Jun Wu <quark@fb.com>
parents: 31755
diff changeset
  2192
                # no extra flags set, no flag processor runs, text = rawtext
31802
ac9a5e89113a revlog: avoid calculating "flags" twice in revision()
Jun Wu <quark@fb.com>
parents: 31801
diff changeset
  2193
                if flags == REVIDX_DEFAULT_FLAGS:
31756
9ec03d5af48f revlog: add a fast path for revision(raw=False)
Jun Wu <quark@fb.com>
parents: 31755
diff changeset
  2194
                    return self._cache[2]
31804
726f24123f02 revlog: avoid applying delta chain on cache hit
Jun Wu <quark@fb.com>
parents: 31803
diff changeset
  2195
                # rawtext is reusable. need to run flag processor
726f24123f02 revlog: avoid applying delta chain on cache hit
Jun Wu <quark@fb.com>
parents: 31803
diff changeset
  2196
                rawtext = self._cache[2]
31756
9ec03d5af48f revlog: add a fast path for revision(raw=False)
Jun Wu <quark@fb.com>
parents: 31755
diff changeset
  2197
26242
d708873f1f33 revlog: drop local assignment of cache variable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26241
diff changeset
  2198
            cachedrev = self._cache[1]
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
  2199
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
  2200
        # look up what we need to read
31803
2be73f9720a8 revlog: indent block to make review easier
Jun Wu <quark@fb.com>
parents: 31802
diff changeset
  2201
        if rawtext is None:
2be73f9720a8 revlog: indent block to make review easier
Jun Wu <quark@fb.com>
parents: 31802
diff changeset
  2202
            if rev is None:
2be73f9720a8 revlog: indent block to make review easier
Jun Wu <quark@fb.com>
parents: 31802
diff changeset
  2203
                rev = self.rev(node)
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
  2204
31803
2be73f9720a8 revlog: indent block to make review easier
Jun Wu <quark@fb.com>
parents: 31802
diff changeset
  2205
            chain, stopped = self._deltachain(rev, stoprev=cachedrev)
2be73f9720a8 revlog: indent block to make review easier
Jun Wu <quark@fb.com>
parents: 31802
diff changeset
  2206
            if stopped:
2be73f9720a8 revlog: indent block to make review easier
Jun Wu <quark@fb.com>
parents: 31802
diff changeset
  2207
                rawtext = self._cache[2]
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
  2208
31803
2be73f9720a8 revlog: indent block to make review easier
Jun Wu <quark@fb.com>
parents: 31802
diff changeset
  2209
            # drop cache to save memory
2be73f9720a8 revlog: indent block to make review easier
Jun Wu <quark@fb.com>
parents: 31802
diff changeset
  2210
            self._cache = None
11754
6ccd130eab0e revlog: drop cache after use to save memory footprint
Matt Mackall <mpm@selenic.com>
parents: 11539
diff changeset
  2211
38644
43d0619cec90 revlog: enforce chunk slicing down to a certain size
Boris Feld <boris.feld@octobus.net>
parents: 38643
diff changeset
  2212
            targetsize = None
43d0619cec90 revlog: enforce chunk slicing down to a certain size
Boris Feld <boris.feld@octobus.net>
parents: 38643
diff changeset
  2213
            rawsize = self.index[rev][2]
43d0619cec90 revlog: enforce chunk slicing down to a certain size
Boris Feld <boris.feld@octobus.net>
parents: 38643
diff changeset
  2214
            if 0 <= rawsize:
43d0619cec90 revlog: enforce chunk slicing down to a certain size
Boris Feld <boris.feld@octobus.net>
parents: 38643
diff changeset
  2215
                targetsize = 4 * rawsize
43d0619cec90 revlog: enforce chunk slicing down to a certain size
Boris Feld <boris.feld@octobus.net>
parents: 38643
diff changeset
  2216
43d0619cec90 revlog: enforce chunk slicing down to a certain size
Boris Feld <boris.feld@octobus.net>
parents: 38643
diff changeset
  2217
            bins = self._chunks(chain, df=_df, targetsize=targetsize)
31803
2be73f9720a8 revlog: indent block to make review easier
Jun Wu <quark@fb.com>
parents: 31802
diff changeset
  2218
            if rawtext is None:
2be73f9720a8 revlog: indent block to make review easier
Jun Wu <quark@fb.com>
parents: 31802
diff changeset
  2219
                rawtext = bytes(bins[0])
2be73f9720a8 revlog: indent block to make review easier
Jun Wu <quark@fb.com>
parents: 31802
diff changeset
  2220
                bins = bins[1:]
8650
ef393d6ec030 revlog: refactor chunk cache interface again
Matt Mackall <mpm@selenic.com>
parents: 8643
diff changeset
  2221
31803
2be73f9720a8 revlog: indent block to make review easier
Jun Wu <quark@fb.com>
parents: 31802
diff changeset
  2222
            rawtext = mdiff.patches(rawtext, bins)
31804
726f24123f02 revlog: avoid applying delta chain on cache hit
Jun Wu <quark@fb.com>
parents: 31803
diff changeset
  2223
            self._cache = (node, rev, rawtext)
30745
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2224
31802
ac9a5e89113a revlog: avoid calculating "flags" twice in revision()
Jun Wu <quark@fb.com>
parents: 31801
diff changeset
  2225
        if flags is None:
31804
726f24123f02 revlog: avoid applying delta chain on cache hit
Jun Wu <quark@fb.com>
parents: 31803
diff changeset
  2226
            if rev is None:
726f24123f02 revlog: avoid applying delta chain on cache hit
Jun Wu <quark@fb.com>
parents: 31803
diff changeset
  2227
                rev = self.rev(node)
31802
ac9a5e89113a revlog: avoid calculating "flags" twice in revision()
Jun Wu <quark@fb.com>
parents: 31801
diff changeset
  2228
            flags = self.flags(rev)
ac9a5e89113a revlog: avoid calculating "flags" twice in revision()
Jun Wu <quark@fb.com>
parents: 31801
diff changeset
  2229
ac9a5e89113a revlog: avoid calculating "flags" twice in revision()
Jun Wu <quark@fb.com>
parents: 31801
diff changeset
  2230
        text, validatehash = self._processflags(rawtext, flags, 'read', raw=raw)
30745
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2231
        if validatehash:
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2232
            self.checkhash(text, node, rev=rev)
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2233
13239
12ed25f39d0b revlog: break hash checking into subfunction
Matt Mackall <mpm@selenic.com>
parents: 13031
diff changeset
  2234
        return text
12ed25f39d0b revlog: break hash checking into subfunction
Matt Mackall <mpm@selenic.com>
parents: 13031
diff changeset
  2235
22785
abc44fcc9c57 revlog: move references to revlog.hash to inside the revlog class
Augie Fackler <raf@durin42.com>
parents: 22784
diff changeset
  2236
    def hash(self, text, p1, p2):
abc44fcc9c57 revlog: move references to revlog.hash to inside the revlog class
Augie Fackler <raf@durin42.com>
parents: 22784
diff changeset
  2237
        """Compute a node hash.
abc44fcc9c57 revlog: move references to revlog.hash to inside the revlog class
Augie Fackler <raf@durin42.com>
parents: 22784
diff changeset
  2238
abc44fcc9c57 revlog: move references to revlog.hash to inside the revlog class
Augie Fackler <raf@durin42.com>
parents: 22784
diff changeset
  2239
        Available as a function so that subclasses can replace the hash
abc44fcc9c57 revlog: move references to revlog.hash to inside the revlog class
Augie Fackler <raf@durin42.com>
parents: 22784
diff changeset
  2240
        as needed.
abc44fcc9c57 revlog: move references to revlog.hash to inside the revlog class
Augie Fackler <raf@durin42.com>
parents: 22784
diff changeset
  2241
        """
abc44fcc9c57 revlog: move references to revlog.hash to inside the revlog class
Augie Fackler <raf@durin42.com>
parents: 22784
diff changeset
  2242
        return hash(text, p1, p2)
abc44fcc9c57 revlog: move references to revlog.hash to inside the revlog class
Augie Fackler <raf@durin42.com>
parents: 22784
diff changeset
  2243
30745
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2244
    def _processflags(self, text, flags, operation, raw=False):
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2245
        """Inspect revision data flags and applies transforms defined by
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2246
        registered flag processors.
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2247
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2248
        ``text`` - the revision data to process
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2249
        ``flags`` - the revision flags
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2250
        ``operation`` - the operation being performed (read or write)
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2251
        ``raw`` - an optional argument describing if the raw transform should be
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2252
        applied.
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2253
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2254
        This method processes the flags in the order (or reverse order if
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2255
        ``operation`` is 'write') defined by REVIDX_FLAGS_ORDER, applying the
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2256
        flag processors registered for present flags. The order of flags defined
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2257
        in REVIDX_FLAGS_ORDER needs to be stable to allow non-commutativity.
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2258
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2259
        Returns a 2-tuple of ``(text, validatehash)`` where ``text`` is the
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2260
        processed text and ``validatehash`` is a bool indicating whether the
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2261
        returned text should be checked for hash integrity.
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2262
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2263
        Note: If the ``raw`` argument is set, it has precedence over the
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2264
        operation and will only update the value of ``validatehash``.
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2265
        """
32286
539cbe0f8fa3 flagprocessor: add a fast path when flags is 0
Jun Wu <quark@fb.com>
parents: 32244
diff changeset
  2266
        # fast path: no flag processors will run
539cbe0f8fa3 flagprocessor: add a fast path when flags is 0
Jun Wu <quark@fb.com>
parents: 32244
diff changeset
  2267
        if flags == 0:
539cbe0f8fa3 flagprocessor: add a fast path when flags is 0
Jun Wu <quark@fb.com>
parents: 32244
diff changeset
  2268
            return text, True
30745
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2269
        if not operation in ('read', 'write'):
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2270
            raise ProgrammingError(_("invalid '%s' operation ") % (operation))
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2271
        # Check all flags are known.
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2272
        if flags & ~REVIDX_KNOWN_FLAGS:
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2273
            raise RevlogError(_("incompatible revision flag '%#x'") %
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2274
                              (flags & ~REVIDX_KNOWN_FLAGS))
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2275
        validatehash = True
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2276
        # Depending on the operation (read or write), the order might be
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2277
        # reversed due to non-commutative transforms.
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2278
        orderedflags = REVIDX_FLAGS_ORDER
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2279
        if operation == 'write':
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2280
            orderedflags = reversed(orderedflags)
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2281
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2282
        for flag in orderedflags:
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2283
            # If a flagprocessor has been registered for a known flag, apply the
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2284
            # related operation transform and update result tuple.
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2285
            if flag & flags:
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2286
                vhash = True
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2287
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2288
                if flag not in _flagprocessors:
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2289
                    message = _("missing processor for flag '%#x'") % (flag)
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2290
                    raise RevlogError(message)
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2291
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2292
                processor = _flagprocessors[flag]
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2293
                if processor is not None:
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2294
                    readtransform, writetransform, rawtransform = processor
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2295
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2296
                    if raw:
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2297
                        vhash = rawtransform(self, text)
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2298
                    elif operation == 'read':
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2299
                        text, vhash = readtransform(self, text)
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2300
                    else: # write operation
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2301
                        text, vhash = writetransform(self, text)
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2302
                validatehash = validatehash and vhash
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2303
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2304
        return text, validatehash
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2305
30584
be5b2098a817 revlog: merge hash checking subfunctions
Remi Chaintron <remi@fb.com>
parents: 30543
diff changeset
  2306
    def checkhash(self, text, node, p1=None, p2=None, rev=None):
be5b2098a817 revlog: merge hash checking subfunctions
Remi Chaintron <remi@fb.com>
parents: 30543
diff changeset
  2307
        """Check node hash integrity.
19624
55749cb14d24 revlog: extract 'checkhash' method
Wojciech Lopata <lopek@fb.com>
parents: 19471
diff changeset
  2308
30584
be5b2098a817 revlog: merge hash checking subfunctions
Remi Chaintron <remi@fb.com>
parents: 30543
diff changeset
  2309
        Available as a function so that subclasses can extend hash mismatch
be5b2098a817 revlog: merge hash checking subfunctions
Remi Chaintron <remi@fb.com>
parents: 30543
diff changeset
  2310
        behaviors as needed.
be5b2098a817 revlog: merge hash checking subfunctions
Remi Chaintron <remi@fb.com>
parents: 30543
diff changeset
  2311
        """
37443
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
  2312
        try:
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
  2313
            if p1 is None and p2 is None:
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
  2314
                p1, p2 = self.parents(node)
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
  2315
            if node != self.hash(text, p1, p2):
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
  2316
                revornode = rev
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
  2317
                if revornode is None:
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
  2318
                    revornode = templatefilters.short(hex(node))
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
  2319
                raise RevlogError(_("integrity check failed on %s:%s")
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
  2320
                    % (self.indexfile, pycompat.bytestr(revornode)))
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
  2321
        except RevlogError:
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
  2322
            if self._censorable and _censoredtext(text):
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
  2323
                raise error.CensoredNodeError(self.indexfile, node, text)
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
  2324
            raise
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
  2325
35974
9ba1d0c724e2 revlog: rename 'self.checkinlinesize' into '_enforceinlinesize'
Boris Feld <boris.feld@octobus.net>
parents: 35973
diff changeset
  2326
    def _enforceinlinesize(self, tr, fp=None):
26376
344a1621674b revlog: add docstring for checkinlinesize()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26243
diff changeset
  2327
        """Check if the revlog is too big for inline and convert if so.
344a1621674b revlog: add docstring for checkinlinesize()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26243
diff changeset
  2328
344a1621674b revlog: add docstring for checkinlinesize()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26243
diff changeset
  2329
        This should be called after revisions are added to the revlog. If the
344a1621674b revlog: add docstring for checkinlinesize()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26243
diff changeset
  2330
        revlog has grown too large to be an inline revlog, it will convert it
344a1621674b revlog: add docstring for checkinlinesize()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26243
diff changeset
  2331
        to use multiple index and data files.
344a1621674b revlog: add docstring for checkinlinesize()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26243
diff changeset
  2332
        """
38844
119d14f41cb2 revlog: remove some knowledge of sentinel nullid in index
Martin von Zweigbergk <martinvonz@google.com>
parents: 38841
diff changeset
  2333
        tiprev = len(self) - 1
119d14f41cb2 revlog: remove some knowledge of sentinel nullid in index
Martin von Zweigbergk <martinvonz@google.com>
parents: 38841
diff changeset
  2334
        if (not self._inline or
119d14f41cb2 revlog: remove some knowledge of sentinel nullid in index
Martin von Zweigbergk <martinvonz@google.com>
parents: 38841
diff changeset
  2335
            (self.start(tiprev) + self.length(tiprev)) < _maxinline):
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
  2336
            return
8315
c8493310ad9b revlog: use index to find index size
Matt Mackall <mpm@selenic.com>
parents: 8314
diff changeset
  2337
2084
Chris Mason <mason@suse.com>
parents: 2082
diff changeset
  2338
        trinfo = tr.find(self.indexfile)
8527
f9a80054dd3c use 'x is None' instead of 'x == None'
Martin Geisler <mg@lazybytes.net>
parents: 8464
diff changeset
  2339
        if trinfo is None:
3680
69cf255a55a1 Indentation cleanups for 2956948b81f3.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 3679
diff changeset
  2340
            raise RevlogError(_("%s not found in the transaction")
69cf255a55a1 Indentation cleanups for 2956948b81f3.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 3679
diff changeset
  2341
                              % self.indexfile)
2084
Chris Mason <mason@suse.com>
parents: 2082
diff changeset
  2342
Chris Mason <mason@suse.com>
parents: 2082
diff changeset
  2343
        trindex = trinfo[2]
24454
59904edf0a5e revlog: make converting from inline to non-line work after a strip
Mike Edgar <adgar@google.com>
parents: 24444
diff changeset
  2344
        if trindex is not None:
59904edf0a5e revlog: make converting from inline to non-line work after a strip
Mike Edgar <adgar@google.com>
parents: 24444
diff changeset
  2345
            dataoff = self.start(trindex)
59904edf0a5e revlog: make converting from inline to non-line work after a strip
Mike Edgar <adgar@google.com>
parents: 24444
diff changeset
  2346
        else:
59904edf0a5e revlog: make converting from inline to non-line work after a strip
Mike Edgar <adgar@google.com>
parents: 24444
diff changeset
  2347
            # revlog was stripped at start of transaction, use all leftover data
59904edf0a5e revlog: make converting from inline to non-line work after a strip
Mike Edgar <adgar@google.com>
parents: 24444
diff changeset
  2348
            trindex = len(self) - 1
38844
119d14f41cb2 revlog: remove some knowledge of sentinel nullid in index
Martin von Zweigbergk <martinvonz@google.com>
parents: 38841
diff changeset
  2349
            dataoff = self.end(tiprev)
2084
Chris Mason <mason@suse.com>
parents: 2082
diff changeset
  2350
Chris Mason <mason@suse.com>
parents: 2082
diff changeset
  2351
        tr.add(self.datafile, dataoff)
8315
c8493310ad9b revlog: use index to find index size
Matt Mackall <mpm@selenic.com>
parents: 8314
diff changeset
  2352
8317
5cdf4067857a revlog: use chunk cache to avoid rereading when splitting inline files
Matt Mackall <mpm@selenic.com>
parents: 8316
diff changeset
  2353
        if fp:
5cdf4067857a revlog: use chunk cache to avoid rereading when splitting inline files
Matt Mackall <mpm@selenic.com>
parents: 8316
diff changeset
  2354
            fp.flush()
5cdf4067857a revlog: use chunk cache to avoid rereading when splitting inline files
Matt Mackall <mpm@selenic.com>
parents: 8316
diff changeset
  2355
            fp.close()
8315
c8493310ad9b revlog: use index to find index size
Matt Mackall <mpm@selenic.com>
parents: 8314
diff changeset
  2356
35970
69cf2e422490 revlog: use context manager for data file lifetime in checkinlinesize
Boris Feld <boris.feld@octobus.net>
parents: 35969
diff changeset
  2357
        with self._datafp('w') as df:
6750
fb42030d79d6 add __len__ and __iter__ methods to repo and revlog
Matt Mackall <mpm@selenic.com>
parents: 6703
diff changeset
  2358
            for r in self:
32229
75e93d95aae6 revlog: rename _chunkraw to _getsegmentforrevs()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32227
diff changeset
  2359
                df.write(self._getsegmentforrevs(r, r)[1])
6261
7c8101b5ceb1 revlog: make sure the files are closed after an exception happens
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 6228
diff changeset
  2360
35971
0f2c51afafb2 revlog: use context manager for index file lifetime in checkinlinesize
Boris Feld <boris.feld@octobus.net>
parents: 35970
diff changeset
  2361
        with self._indexfp('w') as fp:
0f2c51afafb2 revlog: use context manager for index file lifetime in checkinlinesize
Boris Feld <boris.feld@octobus.net>
parents: 35970
diff changeset
  2362
            self.version &= ~FLAG_INLINE_DATA
0f2c51afafb2 revlog: use context manager for index file lifetime in checkinlinesize
Boris Feld <boris.feld@octobus.net>
parents: 35970
diff changeset
  2363
            self._inline = False
0f2c51afafb2 revlog: use context manager for index file lifetime in checkinlinesize
Boris Feld <boris.feld@octobus.net>
parents: 35970
diff changeset
  2364
            io = self._io
0f2c51afafb2 revlog: use context manager for index file lifetime in checkinlinesize
Boris Feld <boris.feld@octobus.net>
parents: 35970
diff changeset
  2365
            for i in self:
0f2c51afafb2 revlog: use context manager for index file lifetime in checkinlinesize
Boris Feld <boris.feld@octobus.net>
parents: 35970
diff changeset
  2366
                e = io.packentry(self.index[i], self.node, self.version, i)
0f2c51afafb2 revlog: use context manager for index file lifetime in checkinlinesize
Boris Feld <boris.feld@octobus.net>
parents: 35970
diff changeset
  2367
                fp.write(e)
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
  2368
35971
0f2c51afafb2 revlog: use context manager for index file lifetime in checkinlinesize
Boris Feld <boris.feld@octobus.net>
parents: 35970
diff changeset
  2369
            # the temp file replace the real index when we exit the context
0f2c51afafb2 revlog: use context manager for index file lifetime in checkinlinesize
Boris Feld <boris.feld@octobus.net>
parents: 35970
diff changeset
  2370
            # manager
2084
Chris Mason <mason@suse.com>
parents: 2082
diff changeset
  2371
8650
ef393d6ec030 revlog: refactor chunk cache interface again
Matt Mackall <mpm@selenic.com>
parents: 8643
diff changeset
  2372
        tr.replace(self.indexfile, trindex * self._io.size)
ef393d6ec030 revlog: refactor chunk cache interface again
Matt Mackall <mpm@selenic.com>
parents: 8643
diff changeset
  2373
        self._chunkclear()
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
  2374
19625
6a411a06cb1f revlog: pass node as an argument of addrevision
Wojciech Lopata <lopek@fb.com>
parents: 19624
diff changeset
  2375
    def addrevision(self, text, transaction, link, p1, p2, cachedelta=None,
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
  2376
                    node=None, flags=REVIDX_DEFAULT_FLAGS, deltacomputer=None):
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
  2377
        """add a revision to the log
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
  2378
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
  2379
        text - the revision data to add
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
  2380
        transaction - the transaction object used for rollback
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
  2381
        link - the linkrev data to add
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
  2382
        p1, p2 - the parent nodeids of the revision
12012
bade7a9c5c07 revlog: fix docstring
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 12011
diff changeset
  2383
        cachedelta - an optional precomputed delta
19625
6a411a06cb1f revlog: pass node as an argument of addrevision
Wojciech Lopata <lopek@fb.com>
parents: 19624
diff changeset
  2384
        node - nodeid of revision; typically node is not specified, and it is
6a411a06cb1f revlog: pass node as an argument of addrevision
Wojciech Lopata <lopek@fb.com>
parents: 19624
diff changeset
  2385
            computed by default as hash(text, p1, p2), however subclasses might
6a411a06cb1f revlog: pass node as an argument of addrevision
Wojciech Lopata <lopek@fb.com>
parents: 19624
diff changeset
  2386
            use different hashing method (and override checkhash() in such case)
30744
e12c0fa1f65b revlog: pass revlog flags to addrevision
Remi Chaintron <remi@fb.com>
parents: 30743
diff changeset
  2387
        flags - the known flags to set on the revision
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
  2388
        deltacomputer - an optional _deltacomputer instance shared between
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
  2389
            multiple calls
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
  2390
        """
19326
7014526d67a8 revlog: add exception when linkrev == nullrev
Durham Goode <durham@fb.com>
parents: 19200
diff changeset
  2391
        if link == nullrev:
7014526d67a8 revlog: add exception when linkrev == nullrev
Durham Goode <durham@fb.com>
parents: 19200
diff changeset
  2392
            raise RevlogError(_("attempted to add linkrev -1 to %s")
7014526d67a8 revlog: add exception when linkrev == nullrev
Durham Goode <durham@fb.com>
parents: 19200
diff changeset
  2393
                              % self.indexfile)
25459
0bda5bfaf0b1 revlog: move size limit check to addrevision
Matt Mackall <mpm@selenic.com>
parents: 25410
diff changeset
  2394
30745
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2395
        if flags:
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2396
            node = node or self.hash(text, p1, p2)
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2397
31750
f319981c24c9 revlog: rename some "text"s to "rawtext"
Jun Wu <quark@fb.com>
parents: 31749
diff changeset
  2398
        rawtext, validatehash = self._processflags(text, flags, 'write')
30745
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2399
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2400
        # If the flag processor modifies the revision data, ignore any provided
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2401
        # cachedelta.
31750
f319981c24c9 revlog: rename some "text"s to "rawtext"
Jun Wu <quark@fb.com>
parents: 31749
diff changeset
  2402
        if rawtext != text:
30745
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2403
            cachedelta = None
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2404
31750
f319981c24c9 revlog: rename some "text"s to "rawtext"
Jun Wu <quark@fb.com>
parents: 31749
diff changeset
  2405
        if len(rawtext) > _maxentrysize:
25459
0bda5bfaf0b1 revlog: move size limit check to addrevision
Matt Mackall <mpm@selenic.com>
parents: 25410
diff changeset
  2406
            raise RevlogError(
0bda5bfaf0b1 revlog: move size limit check to addrevision
Matt Mackall <mpm@selenic.com>
parents: 25410
diff changeset
  2407
                _("%s: size of %d bytes exceeds maximum revlog storage of 2GiB")
31750
f319981c24c9 revlog: rename some "text"s to "rawtext"
Jun Wu <quark@fb.com>
parents: 31749
diff changeset
  2408
                % (self.indexfile, len(rawtext)))
25459
0bda5bfaf0b1 revlog: move size limit check to addrevision
Matt Mackall <mpm@selenic.com>
parents: 25410
diff changeset
  2409
31750
f319981c24c9 revlog: rename some "text"s to "rawtext"
Jun Wu <quark@fb.com>
parents: 31749
diff changeset
  2410
        node = node or self.hash(rawtext, p1, p2)
14196
e7483ec3c374 revlog: remove support for punched/shallow
Sune Foldager <cryo@cyanite.org>
parents: 14195
diff changeset
  2411
        if node in self.nodemap:
12023
44c22dc193a4 revlog.addrevision(): move computation of nodeid in addrevision()
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 12012
diff changeset
  2412
            return node
44c22dc193a4 revlog.addrevision(): move computation of nodeid in addrevision()
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 12012
diff changeset
  2413
30745
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2414
        if validatehash:
31750
f319981c24c9 revlog: rename some "text"s to "rawtext"
Jun Wu <quark@fb.com>
parents: 31749
diff changeset
  2415
            self.checkhash(rawtext, node, p1=p1, p2=p2)
30745
c1b7b2285522 revlog: flag processor
Remi Chaintron <remi@fb.com>
parents: 30744
diff changeset
  2416
32244
3de4c61b5087 revlog: move part of "addrevision" to "addrawrevision"
Jun Wu <quark@fb.com>
parents: 32229
diff changeset
  2417
        return self.addrawrevision(rawtext, transaction, link, p1, p2, node,
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
  2418
                                   flags, cachedelta=cachedelta,
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
  2419
                                   deltacomputer=deltacomputer)
32244
3de4c61b5087 revlog: move part of "addrevision" to "addrawrevision"
Jun Wu <quark@fb.com>
parents: 32229
diff changeset
  2420
3de4c61b5087 revlog: move part of "addrevision" to "addrawrevision"
Jun Wu <quark@fb.com>
parents: 32229
diff changeset
  2421
    def addrawrevision(self, rawtext, transaction, link, p1, p2, node, flags,
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
  2422
                       cachedelta=None, deltacomputer=None):
32244
3de4c61b5087 revlog: move part of "addrevision" to "addrawrevision"
Jun Wu <quark@fb.com>
parents: 32229
diff changeset
  2423
        """add a raw revision with known flags, node and parents
3de4c61b5087 revlog: move part of "addrevision" to "addrawrevision"
Jun Wu <quark@fb.com>
parents: 32229
diff changeset
  2424
        useful when reusing a revision not stored in this revlog (ex: received
3de4c61b5087 revlog: move part of "addrevision" to "addrawrevision"
Jun Wu <quark@fb.com>
parents: 32229
diff changeset
  2425
        over wire, or read from an external bundle).
3de4c61b5087 revlog: move part of "addrevision" to "addrawrevision"
Jun Wu <quark@fb.com>
parents: 32229
diff changeset
  2426
        """
4981
e7131935fbb3 revlog: simplify addrevision
Matt Mackall <mpm@selenic.com>
parents: 4980
diff changeset
  2427
        dfh = None
4982
9672e3c42b0c revlog: change _inline from a function to a variable
Matt Mackall <mpm@selenic.com>
parents: 4981
diff changeset
  2428
        if not self._inline:
35967
61326dd7cb8d revlog: move datafile opening in a method
Boris Feld <boris.feld@octobus.net>
parents: 35845
diff changeset
  2429
            dfh = self._datafp("a+")
35968
91b3b11565e5 revlog: move index file opening in a method
Boris Feld <boris.feld@octobus.net>
parents: 35967
diff changeset
  2430
        ifh = self._indexfp("a+")
6261
7c8101b5ceb1 revlog: make sure the files are closed after an exception happens
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 6228
diff changeset
  2431
        try:
31750
f319981c24c9 revlog: rename some "text"s to "rawtext"
Jun Wu <quark@fb.com>
parents: 31749
diff changeset
  2432
            return self._addrevision(node, rawtext, transaction, link, p1, p2,
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
  2433
                                     flags, cachedelta, ifh, dfh,
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
  2434
                                     deltacomputer=deltacomputer)
6261
7c8101b5ceb1 revlog: make sure the files are closed after an exception happens
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 6228
diff changeset
  2435
        finally:
7c8101b5ceb1 revlog: make sure the files are closed after an exception happens
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 6228
diff changeset
  2436
            if dfh:
7c8101b5ceb1 revlog: make sure the files are closed after an exception happens
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 6228
diff changeset
  2437
                dfh.close()
7c8101b5ceb1 revlog: make sure the files are closed after an exception happens
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 6228
diff changeset
  2438
            ifh.close()
3390
a74addddd092 make revlog.addgroup pass its file handles to addrevision
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 3360
diff changeset
  2439
30795
78ac56aebab6 revlog: use compression engine API for compression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30793
diff changeset
  2440
    def compress(self, data):
78ac56aebab6 revlog: use compression engine API for compression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30793
diff changeset
  2441
        """Generate a possibly-compressed representation of data."""
78ac56aebab6 revlog: use compression engine API for compression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30793
diff changeset
  2442
        if not data:
78ac56aebab6 revlog: use compression engine API for compression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30793
diff changeset
  2443
            return '', data
78ac56aebab6 revlog: use compression engine API for compression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30793
diff changeset
  2444
78ac56aebab6 revlog: use compression engine API for compression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30793
diff changeset
  2445
        compressed = self._compressor.compress(data)
78ac56aebab6 revlog: use compression engine API for compression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30793
diff changeset
  2446
78ac56aebab6 revlog: use compression engine API for compression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30793
diff changeset
  2447
        if compressed:
78ac56aebab6 revlog: use compression engine API for compression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30793
diff changeset
  2448
            # The revlog compressor added the header in the returned data.
78ac56aebab6 revlog: use compression engine API for compression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30793
diff changeset
  2449
            return '', compressed
78ac56aebab6 revlog: use compression engine API for compression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30793
diff changeset
  2450
31643
6ceb3c4c3ab6 py3: fix slicing of byte string in revlog.compress()
Yuya Nishihara <yuya@tcha.org>
parents: 31574
diff changeset
  2451
        if data[0:1] == '\0':
30795
78ac56aebab6 revlog: use compression engine API for compression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30793
diff changeset
  2452
            return '', data
78ac56aebab6 revlog: use compression engine API for compression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30793
diff changeset
  2453
        return 'u', data
17128
1028a1c9077a revlog: make compress a method
Bryan O'Sullivan <bryano@fb.com>
parents: 17009
diff changeset
  2454
30793
b6f455a6e4d6 revlog: move decompress() from module to revlog class (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30792
diff changeset
  2455
    def decompress(self, data):
b6f455a6e4d6 revlog: move decompress() from module to revlog class (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30792
diff changeset
  2456
        """Decompress a revlog chunk.
b6f455a6e4d6 revlog: move decompress() from module to revlog class (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30792
diff changeset
  2457
b6f455a6e4d6 revlog: move decompress() from module to revlog class (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30792
diff changeset
  2458
        The chunk is expected to begin with a header identifying the
b6f455a6e4d6 revlog: move decompress() from module to revlog class (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30792
diff changeset
  2459
        format type so it can be routed to an appropriate decompressor.
b6f455a6e4d6 revlog: move decompress() from module to revlog class (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30792
diff changeset
  2460
        """
b6f455a6e4d6 revlog: move decompress() from module to revlog class (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30792
diff changeset
  2461
        if not data:
b6f455a6e4d6 revlog: move decompress() from module to revlog class (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30792
diff changeset
  2462
            return data
30817
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
  2463
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
  2464
        # Revlogs are read much more frequently than they are written and many
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
  2465
        # chunks only take microseconds to decompress, so performance is
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
  2466
        # important here.
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
  2467
        #
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
  2468
        # We can make a few assumptions about revlogs:
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
  2469
        #
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
  2470
        # 1) the majority of chunks will be compressed (as opposed to inline
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
  2471
        #    raw data).
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
  2472
        # 2) decompressing *any* data will likely by at least 10x slower than
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
  2473
        #    returning raw inline data.
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
  2474
        # 3) we want to prioritize common and officially supported compression
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
  2475
        #    engines
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
  2476
        #
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
  2477
        # It follows that we want to optimize for "decompress compressed data
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
  2478
        # when encoded with common and officially supported compression engines"
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
  2479
        # case over "raw data" and "data encoded by less common or non-official
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
  2480
        # compression engines." That is why we have the inline lookup first
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
  2481
        # followed by the compengines lookup.
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
  2482
        #
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
  2483
        # According to `hg perfrevlogchunks`, this is ~0.5% faster for zlib
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
  2484
        # compressed chunks. And this matters for changelog and manifest reads.
31356
ef6888172437 revlog: extract first byte of revlog with a slice so it's portable
Augie Fackler <augie@google.com>
parents: 30829
diff changeset
  2485
        t = data[0:1]
30817
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
  2486
30793
b6f455a6e4d6 revlog: move decompress() from module to revlog class (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30792
diff changeset
  2487
        if t == 'x':
b6f455a6e4d6 revlog: move decompress() from module to revlog class (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30792
diff changeset
  2488
            try:
30817
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
  2489
                return _zlibdecompress(data)
30793
b6f455a6e4d6 revlog: move decompress() from module to revlog class (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30792
diff changeset
  2490
            except zlib.error as e:
36702
f659a407e5ee py3: use util.forcebytestr instead of str to convert error messages
Pulkit Goyal <7895pulkit@gmail.com>
parents: 36238
diff changeset
  2491
                raise RevlogError(_('revlog decompress error: %s') %
37084
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module
Yuya Nishihara <yuya@tcha.org>
parents: 36760
diff changeset
  2492
                                  stringutil.forcebytestr(e))
30817
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
  2493
        # '\0' is more common than 'u' so it goes first.
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
  2494
        elif t == '\0':
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
  2495
            return data
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
  2496
        elif t == 'u':
30793
b6f455a6e4d6 revlog: move decompress() from module to revlog class (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30792
diff changeset
  2497
            return util.buffer(data, 1)
30817
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
  2498
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
  2499
        try:
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
  2500
            compressor = self._decompressors[t]
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
  2501
        except KeyError:
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
  2502
            try:
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
  2503
                engine = util.compengines.forrevlogheader(t)
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
  2504
                compressor = engine.revlogcompressor()
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
  2505
                self._decompressors[t] = compressor
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
  2506
            except KeyError:
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
  2507
                raise RevlogError(_('unknown compression type %r') % t)
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
  2508
2b279126b8f5 revlog: use compression engine APIs for decompression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30795
diff changeset
  2509
        return compressor.decompress(data)
30793
b6f455a6e4d6 revlog: move decompress() from module to revlog class (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30792
diff changeset
  2510
38117
69ec6f98cfa6 revlog: isgooddeltainfo takes the whole revinfo object
Paul Morelle <paul.morelle@octobus.net>
parents: 38116
diff changeset
  2511
    def _isgooddeltainfo(self, deltainfo, revinfo):
26115
748347e0e8d4 revlog: move delta check to it's own function
Durham Goode <durham@fb.com>
parents: 25892
diff changeset
  2512
        """Returns True if the given delta is good. Good means that it is within
748347e0e8d4 revlog: move delta check to it's own function
Durham Goode <durham@fb.com>
parents: 25892
diff changeset
  2513
        the disk span, disk size, and chain length bounds that we know to be
748347e0e8d4 revlog: move delta check to it's own function
Durham Goode <durham@fb.com>
parents: 25892
diff changeset
  2514
        performant."""
38116
bf59f95583c1 revlog: make variable name 'd' more explicit in _isgooddeltainfo
Paul Morelle <paul.morelle@octobus.net>
parents: 38114
diff changeset
  2515
        if deltainfo is None:
26115
748347e0e8d4 revlog: move delta check to it's own function
Durham Goode <durham@fb.com>
parents: 25892
diff changeset
  2516
            return False
748347e0e8d4 revlog: move delta check to it's own function
Durham Goode <durham@fb.com>
parents: 25892
diff changeset
  2517
38116
bf59f95583c1 revlog: make variable name 'd' more explicit in _isgooddeltainfo
Paul Morelle <paul.morelle@octobus.net>
parents: 38114
diff changeset
  2518
        # - 'deltainfo.distance' is the distance from the base revision --
bf59f95583c1 revlog: make variable name 'd' more explicit in _isgooddeltainfo
Paul Morelle <paul.morelle@octobus.net>
parents: 38114
diff changeset
  2519
        #   bounding it limits the amount of I/O we need to do.
bf59f95583c1 revlog: make variable name 'd' more explicit in _isgooddeltainfo
Paul Morelle <paul.morelle@octobus.net>
parents: 38114
diff changeset
  2520
        # - 'deltainfo.compresseddeltalen' is the sum of the total size of
bf59f95583c1 revlog: make variable name 'd' more explicit in _isgooddeltainfo
Paul Morelle <paul.morelle@octobus.net>
parents: 38114
diff changeset
  2521
        #   deltas we need to apply -- bounding it limits the amount of CPU
bf59f95583c1 revlog: make variable name 'd' more explicit in _isgooddeltainfo
Paul Morelle <paul.morelle@octobus.net>
parents: 38114
diff changeset
  2522
        #   we consume.
33207
895ecec31c70 revlog: add an experimental option to mitigated delta issues (issue5480)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 33171
diff changeset
  2523
38718
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents: 38717
diff changeset
  2524
        if self._sparserevlog:
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents: 38717
diff changeset
  2525
            # As sparse-read will be used, we can consider that the distance,
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents: 38717
diff changeset
  2526
            # instead of being the span of the whole chunk,
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents: 38717
diff changeset
  2527
            # is the span of the largest read chunk
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents: 38717
diff changeset
  2528
            base = deltainfo.base
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents: 38717
diff changeset
  2529
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents: 38717
diff changeset
  2530
            if base != nullrev:
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents: 38717
diff changeset
  2531
                deltachain = self._deltachain(base)[0]
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents: 38717
diff changeset
  2532
            else:
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents: 38717
diff changeset
  2533
                deltachain = []
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents: 38717
diff changeset
  2534
39157
a98e926b2f5b revlog: only consider the span of the delta section
Paul Morelle <paul.morelle@octobus.net>
parents: 39156
diff changeset
  2535
            # search for the first non-snapshot revision
a98e926b2f5b revlog: only consider the span of the delta section
Paul Morelle <paul.morelle@octobus.net>
parents: 39156
diff changeset
  2536
            for idx, r in enumerate(deltachain):
a98e926b2f5b revlog: only consider the span of the delta section
Paul Morelle <paul.morelle@octobus.net>
parents: 39156
diff changeset
  2537
                if not self.issnapshot(r):
a98e926b2f5b revlog: only consider the span of the delta section
Paul Morelle <paul.morelle@octobus.net>
parents: 39156
diff changeset
  2538
                    break
a98e926b2f5b revlog: only consider the span of the delta section
Paul Morelle <paul.morelle@octobus.net>
parents: 39156
diff changeset
  2539
            deltachain = deltachain[idx:]
38718
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents: 38717
diff changeset
  2540
            chunks = _slicechunk(self, deltachain, deltainfo)
39139
3730b779ed5b sparse-revlog: fix delta validity computation
Boris Feld <boris.feld@octobus.net>
parents: 38763
diff changeset
  2541
            all_span = [_segmentspan(self, revs, deltainfo) for revs in chunks]
3730b779ed5b sparse-revlog: fix delta validity computation
Boris Feld <boris.feld@octobus.net>
parents: 38763
diff changeset
  2542
            distance = max(all_span)
38718
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents: 38717
diff changeset
  2543
        else:
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents: 38717
diff changeset
  2544
            distance = deltainfo.distance
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents: 38717
diff changeset
  2545
38117
69ec6f98cfa6 revlog: isgooddeltainfo takes the whole revinfo object
Paul Morelle <paul.morelle@octobus.net>
parents: 38116
diff changeset
  2546
        textlen = revinfo.textlen
33207
895ecec31c70 revlog: add an experimental option to mitigated delta issues (issue5480)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 33171
diff changeset
  2547
        defaultmax = textlen * 4
895ecec31c70 revlog: add an experimental option to mitigated delta issues (issue5480)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 33171
diff changeset
  2548
        maxdist = self._maxdeltachainspan
895ecec31c70 revlog: add an experimental option to mitigated delta issues (issue5480)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 33171
diff changeset
  2549
        if not maxdist:
38716
c67093e81a3e revlog: extract `deltainfo.distance` for future conditional redefinition
Paul Morelle <paul.morelle@octobus.net>
parents: 38710
diff changeset
  2550
            maxdist = distance # ensure the conditional pass
33207
895ecec31c70 revlog: add an experimental option to mitigated delta issues (issue5480)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 33171
diff changeset
  2551
        maxdist = max(maxdist, defaultmax)
38718
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents: 38717
diff changeset
  2552
        if self._sparserevlog and maxdist < self._srmingapsize:
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents: 38717
diff changeset
  2553
            # In multiple place, we are ignoring irrelevant data range below a
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents: 38717
diff changeset
  2554
            # certain size. Be also apply this tradeoff here and relax span
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents: 38717
diff changeset
  2555
            # constraint for small enought content.
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480)
Paul Morelle <paul.morelle@octobus.net>
parents: 38717
diff changeset
  2556
            maxdist = self._srmingapsize
39084
152ae0f84f9a revlog: split and document good delta conditional
Boris Feld <boris.feld@octobus.net>
parents: 39083
diff changeset
  2557
152ae0f84f9a revlog: split and document good delta conditional
Boris Feld <boris.feld@octobus.net>
parents: 39083
diff changeset
  2558
        # Bad delta from read span:
152ae0f84f9a revlog: split and document good delta conditional
Boris Feld <boris.feld@octobus.net>
parents: 39083
diff changeset
  2559
        #
152ae0f84f9a revlog: split and document good delta conditional
Boris Feld <boris.feld@octobus.net>
parents: 39083
diff changeset
  2560
        #   If the span of data read is larger than the maximum allowed.
152ae0f84f9a revlog: split and document good delta conditional
Boris Feld <boris.feld@octobus.net>
parents: 39083
diff changeset
  2561
        if maxdist < distance:
152ae0f84f9a revlog: split and document good delta conditional
Boris Feld <boris.feld@octobus.net>
parents: 39083
diff changeset
  2562
            return False
152ae0f84f9a revlog: split and document good delta conditional
Boris Feld <boris.feld@octobus.net>
parents: 39083
diff changeset
  2563
152ae0f84f9a revlog: split and document good delta conditional
Boris Feld <boris.feld@octobus.net>
parents: 39083
diff changeset
  2564
        # Bad delta from new delta size:
152ae0f84f9a revlog: split and document good delta conditional
Boris Feld <boris.feld@octobus.net>
parents: 39083
diff changeset
  2565
        #
152ae0f84f9a revlog: split and document good delta conditional
Boris Feld <boris.feld@octobus.net>
parents: 39083
diff changeset
  2566
        #   If the delta size is larger than the target text, storing the
152ae0f84f9a revlog: split and document good delta conditional
Boris Feld <boris.feld@octobus.net>
parents: 39083
diff changeset
  2567
        #   delta will be inefficient.
152ae0f84f9a revlog: split and document good delta conditional
Boris Feld <boris.feld@octobus.net>
parents: 39083
diff changeset
  2568
        if textlen < deltainfo.deltalen:
152ae0f84f9a revlog: split and document good delta conditional
Boris Feld <boris.feld@octobus.net>
parents: 39083
diff changeset
  2569
            return False
152ae0f84f9a revlog: split and document good delta conditional
Boris Feld <boris.feld@octobus.net>
parents: 39083
diff changeset
  2570
152ae0f84f9a revlog: split and document good delta conditional
Boris Feld <boris.feld@octobus.net>
parents: 39083
diff changeset
  2571
        # Bad delta from cumulated payload size:
152ae0f84f9a revlog: split and document good delta conditional
Boris Feld <boris.feld@octobus.net>
parents: 39083
diff changeset
  2572
        #
152ae0f84f9a revlog: split and document good delta conditional
Boris Feld <boris.feld@octobus.net>
parents: 39083
diff changeset
  2573
        #   If the sum of delta get larger than K * target text length.
39086
64ddad2f26bb revlog: use a symbolic constant for the deltas/text ration
Boris Feld <boris.feld@octobus.net>
parents: 39085
diff changeset
  2574
        if textlen * LIMIT_DELTA2TEXT < deltainfo.compresseddeltalen:
39084
152ae0f84f9a revlog: split and document good delta conditional
Boris Feld <boris.feld@octobus.net>
parents: 39083
diff changeset
  2575
            return False
152ae0f84f9a revlog: split and document good delta conditional
Boris Feld <boris.feld@octobus.net>
parents: 39083
diff changeset
  2576
152ae0f84f9a revlog: split and document good delta conditional
Boris Feld <boris.feld@octobus.net>
parents: 39083
diff changeset
  2577
        # Bad delta from chain length:
152ae0f84f9a revlog: split and document good delta conditional
Boris Feld <boris.feld@octobus.net>
parents: 39083
diff changeset
  2578
        #
152ae0f84f9a revlog: split and document good delta conditional
Boris Feld <boris.feld@octobus.net>
parents: 39083
diff changeset
  2579
        #   If the number of delta in the chain gets too high.
152ae0f84f9a revlog: split and document good delta conditional
Boris Feld <boris.feld@octobus.net>
parents: 39083
diff changeset
  2580
        if self._maxchainlen and  self._maxchainlen < deltainfo.chainlen:
26115
748347e0e8d4 revlog: move delta check to it's own function
Durham Goode <durham@fb.com>
parents: 25892
diff changeset
  2581
            return False
748347e0e8d4 revlog: move delta check to it's own function
Durham Goode <durham@fb.com>
parents: 25892
diff changeset
  2582
39155
a43ef77cab1d revlog: bound number of snapshots in a chain
Paul Morelle <paul.morelle@octobus.net>
parents: 39154
diff changeset
  2583
        # bad delta from intermediate snapshot size limit
a43ef77cab1d revlog: bound number of snapshots in a chain
Paul Morelle <paul.morelle@octobus.net>
parents: 39154
diff changeset
  2584
        #
a43ef77cab1d revlog: bound number of snapshots in a chain
Paul Morelle <paul.morelle@octobus.net>
parents: 39154
diff changeset
  2585
        #   If an intermediate snapshot size is higher than the limit.  The
a43ef77cab1d revlog: bound number of snapshots in a chain
Paul Morelle <paul.morelle@octobus.net>
parents: 39154
diff changeset
  2586
        #   limit exist to prevent endless chain of intermediate delta to be
a43ef77cab1d revlog: bound number of snapshots in a chain
Paul Morelle <paul.morelle@octobus.net>
parents: 39154
diff changeset
  2587
        #   created.
a43ef77cab1d revlog: bound number of snapshots in a chain
Paul Morelle <paul.morelle@octobus.net>
parents: 39154
diff changeset
  2588
        if (deltainfo.snapshotdepth is not None and
a43ef77cab1d revlog: bound number of snapshots in a chain
Paul Morelle <paul.morelle@octobus.net>
parents: 39154
diff changeset
  2589
                (textlen >> deltainfo.snapshotdepth) < deltainfo.deltalen):
a43ef77cab1d revlog: bound number of snapshots in a chain
Paul Morelle <paul.morelle@octobus.net>
parents: 39154
diff changeset
  2590
            return False
a43ef77cab1d revlog: bound number of snapshots in a chain
Paul Morelle <paul.morelle@octobus.net>
parents: 39154
diff changeset
  2591
39156
b3b4bee161cf revlog: ensure intermediate snapshot have decreasing size
Boris Feld <boris.feld@octobus.net>
parents: 39155
diff changeset
  2592
        # bad delta if new intermediate snapshot is larger than the previous
b3b4bee161cf revlog: ensure intermediate snapshot have decreasing size
Boris Feld <boris.feld@octobus.net>
parents: 39155
diff changeset
  2593
        # snapshot
b3b4bee161cf revlog: ensure intermediate snapshot have decreasing size
Boris Feld <boris.feld@octobus.net>
parents: 39155
diff changeset
  2594
        if (deltainfo.snapshotdepth
b3b4bee161cf revlog: ensure intermediate snapshot have decreasing size
Boris Feld <boris.feld@octobus.net>
parents: 39155
diff changeset
  2595
                and self.length(deltainfo.base) < deltainfo.deltalen):
b3b4bee161cf revlog: ensure intermediate snapshot have decreasing size
Boris Feld <boris.feld@octobus.net>
parents: 39155
diff changeset
  2596
            return False
b3b4bee161cf revlog: ensure intermediate snapshot have decreasing size
Boris Feld <boris.feld@octobus.net>
parents: 39155
diff changeset
  2597
26115
748347e0e8d4 revlog: move delta check to it's own function
Durham Goode <durham@fb.com>
parents: 25892
diff changeset
  2598
        return True
748347e0e8d4 revlog: move delta check to it's own function
Durham Goode <durham@fb.com>
parents: 25892
diff changeset
  2599
31755
ec48d57de110 revlog: make _addrevision only accept rawtext
Jun Wu <quark@fb.com>
parents: 31754
diff changeset
  2600
    def _addrevision(self, node, rawtext, transaction, link, p1, p2, flags,
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
  2601
                     cachedelta, ifh, dfh, alwayscache=False,
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
  2602
                     deltacomputer=None):
14292
c97d8485b5fa revlog: add docstring to _addrevision
Sune Foldager <cryo@cyanite.org>
parents: 14270
diff changeset
  2603
        """internal function to add revisions to the log
12623
8f97b50a8d10 revlog._addrevision(): allow text argument to be None, build it lazily
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 12336
diff changeset
  2604
14292
c97d8485b5fa revlog: add docstring to _addrevision
Sune Foldager <cryo@cyanite.org>
parents: 14270
diff changeset
  2605
        see addrevision for argument descriptions.
31755
ec48d57de110 revlog: make _addrevision only accept rawtext
Jun Wu <quark@fb.com>
parents: 31754
diff changeset
  2606
ec48d57de110 revlog: make _addrevision only accept rawtext
Jun Wu <quark@fb.com>
parents: 31754
diff changeset
  2607
        note: "addrevision" takes non-raw text, "_addrevision" takes raw text.
ec48d57de110 revlog: make _addrevision only accept rawtext
Jun Wu <quark@fb.com>
parents: 31754
diff changeset
  2608
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
  2609
        if "deltacomputer" is not provided or None, a defaultdeltacomputer will
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
  2610
        be used.
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
  2611
14292
c97d8485b5fa revlog: add docstring to _addrevision
Sune Foldager <cryo@cyanite.org>
parents: 14270
diff changeset
  2612
        invariants:
31755
ec48d57de110 revlog: make _addrevision only accept rawtext
Jun Wu <quark@fb.com>
parents: 31754
diff changeset
  2613
        - rawtext is optional (can be None); if not set, cachedelta must be set.
17424
e7cfe3587ea4 fix trivial spelling errors
Mads Kiilerich <mads@kiilerich.com>
parents: 17150
diff changeset
  2614
          if both are set, they must correspond to each other.
14292
c97d8485b5fa revlog: add docstring to _addrevision
Sune Foldager <cryo@cyanite.org>
parents: 14270
diff changeset
  2615
        """
33938
9180f8f593f3 revlog: abort on attempt to write null revision
Martin von Zweigbergk <martinvonz@google.com>
parents: 33392
diff changeset
  2616
        if node == nullid:
9180f8f593f3 revlog: abort on attempt to write null revision
Martin von Zweigbergk <martinvonz@google.com>
parents: 33392
diff changeset
  2617
            raise RevlogError(_("%s: attempt to add null revision") %
9180f8f593f3 revlog: abort on attempt to write null revision
Martin von Zweigbergk <martinvonz@google.com>
parents: 33392
diff changeset
  2618
                              (self.indexfile))
37449
a0d71618074f revlog: detect pseudo file nodeids to raise WdirUnsupported exception
Yuya Nishihara <yuya@tcha.org>
parents: 37443
diff changeset
  2619
        if node == wdirid or node in wdirfilenodeids:
34028
bfb38c5cebf4 revlog: move check for wdir from changelog to revlog
Martin von Zweigbergk <martinvonz@google.com>
parents: 34026
diff changeset
  2620
            raise RevlogError(_("%s: attempt to add wdir revision") %
bfb38c5cebf4 revlog: move check for wdir from changelog to revlog
Martin von Zweigbergk <martinvonz@google.com>
parents: 34026
diff changeset
  2621
                              (self.indexfile))
bfb38c5cebf4 revlog: move check for wdir from changelog to revlog
Martin von Zweigbergk <martinvonz@google.com>
parents: 34026
diff changeset
  2622
35635
0b52c0ecbc23 revlog: choose between ifh and dfh once for all
Paul Morelle <paul.morelle@octobus.net>
parents: 35634
diff changeset
  2623
        if self._inline:
0b52c0ecbc23 revlog: choose between ifh and dfh once for all
Paul Morelle <paul.morelle@octobus.net>
parents: 35634
diff changeset
  2624
            fh = ifh
0b52c0ecbc23 revlog: choose between ifh and dfh once for all
Paul Morelle <paul.morelle@octobus.net>
parents: 35634
diff changeset
  2625
        else:
0b52c0ecbc23 revlog: choose between ifh and dfh once for all
Paul Morelle <paul.morelle@octobus.net>
parents: 35634
diff changeset
  2626
            fh = dfh
0b52c0ecbc23 revlog: choose between ifh and dfh once for all
Paul Morelle <paul.morelle@octobus.net>
parents: 35634
diff changeset
  2627
31755
ec48d57de110 revlog: make _addrevision only accept rawtext
Jun Wu <quark@fb.com>
parents: 31754
diff changeset
  2628
        btext = [rawtext]
12623
8f97b50a8d10 revlog._addrevision(): allow text argument to be None, build it lazily
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 12336
diff changeset
  2629
6750
fb42030d79d6 add __len__ and __iter__ methods to repo and revlog
Matt Mackall <mpm@selenic.com>
parents: 6703
diff changeset
  2630
        curr = len(self)
4981
e7131935fbb3 revlog: simplify addrevision
Matt Mackall <mpm@selenic.com>
parents: 4980
diff changeset
  2631
        prev = curr - 1
e7131935fbb3 revlog: simplify addrevision
Matt Mackall <mpm@selenic.com>
parents: 4980
diff changeset
  2632
        offset = self.end(prev)
12889
5482c6b826f4 revlog: precalculate p1 and p2 revisions
Matt Mackall <mpm@selenic.com>
parents: 12888
diff changeset
  2633
        p1r, p2r = self.rev(p1), self.rev(p2)
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
  2634
26116
562cfc99e611 revlog: move textlen calculation to be above delta chooser
Durham Goode <durham@fb.com>
parents: 26115
diff changeset
  2635
        # full versions are inserted when the needed deltas
562cfc99e611 revlog: move textlen calculation to be above delta chooser
Durham Goode <durham@fb.com>
parents: 26115
diff changeset
  2636
        # become comparable to the uncompressed text
31755
ec48d57de110 revlog: make _addrevision only accept rawtext
Jun Wu <quark@fb.com>
parents: 31754
diff changeset
  2637
        if rawtext is None:
36748
369aadf7a326 revlog: resolve lfs rawtext to vanilla rawtext before applying delta
Jun Wu <quark@fb.com>
parents: 36744
diff changeset
  2638
            # need rawtext size, before changed by flag processors, which is
369aadf7a326 revlog: resolve lfs rawtext to vanilla rawtext before applying delta
Jun Wu <quark@fb.com>
parents: 36744
diff changeset
  2639
            # the non-raw size. use revlog explicitly to avoid filelog's extra
369aadf7a326 revlog: resolve lfs rawtext to vanilla rawtext before applying delta
Jun Wu <quark@fb.com>
parents: 36744
diff changeset
  2640
            # logic that might remove metadata size.
369aadf7a326 revlog: resolve lfs rawtext to vanilla rawtext before applying delta
Jun Wu <quark@fb.com>
parents: 36744
diff changeset
  2641
            textlen = mdiff.patchedsize(revlog.size(self, cachedelta[0]),
26116
562cfc99e611 revlog: move textlen calculation to be above delta chooser
Durham Goode <durham@fb.com>
parents: 26115
diff changeset
  2642
                                        cachedelta[1])
562cfc99e611 revlog: move textlen calculation to be above delta chooser
Durham Goode <durham@fb.com>
parents: 26115
diff changeset
  2643
        else:
31755
ec48d57de110 revlog: make _addrevision only accept rawtext
Jun Wu <quark@fb.com>
parents: 31754
diff changeset
  2644
            textlen = len(rawtext)
26116
562cfc99e611 revlog: move textlen calculation to be above delta chooser
Durham Goode <durham@fb.com>
parents: 26115
diff changeset
  2645
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
  2646
        if deltacomputer is None:
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
  2647
            deltacomputer = _deltacomputer(self)
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
  2648
35737
d99b07bc69fb revlog: refactor out _finddeltainfo from _addrevision
Paul Morelle <paul.morelle@octobus.net>
parents: 35641
diff changeset
  2649
        revinfo = _revisioninfo(node, p1, p2, btext, textlen, cachedelta, flags)
36744
33275ab5e837 revlog: do not use delta for lfs revisions
Jun Wu <quark@fb.com>
parents: 36743
diff changeset
  2650
33275ab5e837 revlog: do not use delta for lfs revisions
Jun Wu <quark@fb.com>
parents: 36743
diff changeset
  2651
        # no delta for flag processor revision (see "candelta" for why)
33275ab5e837 revlog: do not use delta for lfs revisions
Jun Wu <quark@fb.com>
parents: 36743
diff changeset
  2652
        # not calling candelta since only one revision needs test, also to
33275ab5e837 revlog: do not use delta for lfs revisions
Jun Wu <quark@fb.com>
parents: 36743
diff changeset
  2653
        # avoid overhead fetching flags again.
33275ab5e837 revlog: do not use delta for lfs revisions
Jun Wu <quark@fb.com>
parents: 36743
diff changeset
  2654
        if flags & REVIDX_RAWTEXT_CHANGING_FLAGS:
33275ab5e837 revlog: do not use delta for lfs revisions
Jun Wu <quark@fb.com>
parents: 36743
diff changeset
  2655
            deltainfo = None
33275ab5e837 revlog: do not use delta for lfs revisions
Jun Wu <quark@fb.com>
parents: 36743
diff changeset
  2656
        else:
33275ab5e837 revlog: do not use delta for lfs revisions
Jun Wu <quark@fb.com>
parents: 36743
diff changeset
  2657
            deltainfo = deltacomputer.finddeltainfo(revinfo, fh)
35634
b43578ec483a revlog: refactor out the selection of candidate revisions
Paul Morelle <paul.morelle@octobus.net>
parents: 35521
diff changeset
  2658
35638
edc9330acac1 revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents: 35637
diff changeset
  2659
        if deltainfo is not None:
edc9330acac1 revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents: 35637
diff changeset
  2660
            base = deltainfo.base
edc9330acac1 revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents: 35637
diff changeset
  2661
            chainbase = deltainfo.chainbase
edc9330acac1 revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents: 35637
diff changeset
  2662
            data = deltainfo.data
edc9330acac1 revlog: introduce 'deltainfo' to distinguish from 'delta'
Paul Morelle <paul.morelle@octobus.net>
parents: 35637
diff changeset
  2663
            l = deltainfo.deltalen
27250
bff71fe05768 revlog: make calls to _isgooddelta() consistent
Martin von Zweigbergk <martinvonz@google.com>
parents: 27249
diff changeset
  2664
        else:
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
  2665
            rawtext = deltacomputer.buildtext(revinfo, fh)
31755
ec48d57de110 revlog: make _addrevision only accept rawtext
Jun Wu <quark@fb.com>
parents: 31754
diff changeset
  2666
            data = self.compress(rawtext)
1533
3d11f81c9145 Reduce string duplication in compression code
mason@suse.com
parents: 1509
diff changeset
  2667
            l = len(data[1]) + len(data[0])
14296
62e25c63fb3a revlog: fix bug in chainbase cache
Sune Foldager <cryo@cyanite.org>
parents: 14292
diff changeset
  2668
            base = chainbase = curr
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
  2669
12623
8f97b50a8d10 revlog._addrevision(): allow text argument to be None, build it lazily
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 12336
diff changeset
  2670
        e = (offset_type(offset, flags), l, textlen,
12889
5482c6b826f4 revlog: precalculate p1 and p2 revisions
Matt Mackall <mpm@selenic.com>
parents: 12888
diff changeset
  2671
             base, link, p1r, p2r, node)
38850
6104b203bec8 index: replace insert(-1, e) method by append(e) method
Martin von Zweigbergk <martinvonz@google.com>
parents: 38849
diff changeset
  2672
        self.index.append(e)
4981
e7131935fbb3 revlog: simplify addrevision
Matt Mackall <mpm@selenic.com>
parents: 4980
diff changeset
  2673
        self.nodemap[node] = curr
4977
6cb30bc4ca32 revlog: parse revlogv0 indexes into v1 internally
Matt Mackall <mpm@selenic.com>
parents: 4976
diff changeset
  2674
5338
f87685355c9c revlog: fix revlogio.packentry corner case
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5325
diff changeset
  2675
        entry = self._io.packentry(e, self.node, self.version, curr)
20217
33394f2e331e revlog: move file writing to a separate function
Durham Goode <durham@fb.com>
parents: 20180
diff changeset
  2676
        self._writeentry(transaction, ifh, dfh, entry, data, link, offset)
33394f2e331e revlog: move file writing to a separate function
Durham Goode <durham@fb.com>
parents: 20180
diff changeset
  2677
31755
ec48d57de110 revlog: make _addrevision only accept rawtext
Jun Wu <quark@fb.com>
parents: 31754
diff changeset
  2678
        if alwayscache and rawtext is None:
39192
dbc5ead9f40f revlog: fix typo in 'buildtext' name
Boris Feld <boris.feld@octobus.net>
parents: 39191
diff changeset
  2679
            rawtext = deltacomputer.buildtext(revinfo, fh)
26243
836291420d53 revlog: optionally cache the full text when adding revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26242
diff changeset
  2680
35845
be923ce44d6a revlog: correct type in check to verify rawtext is immutable
Augie Fackler <augie@google.com>
parents: 35738
diff changeset
  2681
        if type(rawtext) == bytes: # only accept immutable objects
31755
ec48d57de110 revlog: make _addrevision only accept rawtext
Jun Wu <quark@fb.com>
parents: 31754
diff changeset
  2682
            self._cache = (node, curr, rawtext)
29830
92ac2baaea86 revlog: use an LRU cache for delta chain bases
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29829
diff changeset
  2683
        self._chainbasecache[curr] = chainbase
20217
33394f2e331e revlog: move file writing to a separate function
Durham Goode <durham@fb.com>
parents: 20180
diff changeset
  2684
        return node
33394f2e331e revlog: move file writing to a separate function
Durham Goode <durham@fb.com>
parents: 20180
diff changeset
  2685
33394f2e331e revlog: move file writing to a separate function
Durham Goode <durham@fb.com>
parents: 20180
diff changeset
  2686
    def _writeentry(self, transaction, ifh, dfh, entry, data, link, offset):
27430
e240e914d226 revlog: seek to end of file before writing (issue4943)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26705
diff changeset
  2687
        # Files opened in a+ mode have inconsistent behavior on various
e240e914d226 revlog: seek to end of file before writing (issue4943)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26705
diff changeset
  2688
        # platforms. Windows requires that a file positioning call be made
e240e914d226 revlog: seek to end of file before writing (issue4943)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26705
diff changeset
  2689
        # when the file handle transitions between reads and writes. See
e240e914d226 revlog: seek to end of file before writing (issue4943)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26705
diff changeset
  2690
        # 3686fa2b8eee and the mixedfilemodewrapper in windows.py. On other
e240e914d226 revlog: seek to end of file before writing (issue4943)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26705
diff changeset
  2691
        # platforms, Python or the platform itself can be buggy. Some versions
e240e914d226 revlog: seek to end of file before writing (issue4943)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26705
diff changeset
  2692
        # of Solaris have been observed to not append at the end of the file
e240e914d226 revlog: seek to end of file before writing (issue4943)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26705
diff changeset
  2693
        # if the file was seeked to before the end. See issue4943 for more.
e240e914d226 revlog: seek to end of file before writing (issue4943)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26705
diff changeset
  2694
        #
e240e914d226 revlog: seek to end of file before writing (issue4943)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26705
diff changeset
  2695
        # We work around this issue by inserting a seek() before writing.
e240e914d226 revlog: seek to end of file before writing (issue4943)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26705
diff changeset
  2696
        # Note: This is likely not necessary on Python 3.
e240e914d226 revlog: seek to end of file before writing (issue4943)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26705
diff changeset
  2697
        ifh.seek(0, os.SEEK_END)
27441
e47841c8343d revlog: fix bad indentation (replace tab by space)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27430
diff changeset
  2698
        if dfh:
27430
e240e914d226 revlog: seek to end of file before writing (issue4943)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26705
diff changeset
  2699
            dfh.seek(0, os.SEEK_END)
e240e914d226 revlog: seek to end of file before writing (issue4943)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26705
diff changeset
  2700
20217
33394f2e331e revlog: move file writing to a separate function
Durham Goode <durham@fb.com>
parents: 20180
diff changeset
  2701
        curr = len(self) - 1
4982
9672e3c42b0c revlog: change _inline from a function to a variable
Matt Mackall <mpm@selenic.com>
parents: 4981
diff changeset
  2702
        if not self._inline:
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
  2703
            transaction.add(self.datafile, offset)
4981
e7131935fbb3 revlog: simplify addrevision
Matt Mackall <mpm@selenic.com>
parents: 4980
diff changeset
  2704
            transaction.add(self.indexfile, curr * len(entry))
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
  2705
            if data[0]:
3390
a74addddd092 make revlog.addgroup pass its file handles to addrevision
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 3360
diff changeset
  2706
                dfh.write(data[0])
a74addddd092 make revlog.addgroup pass its file handles to addrevision
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 3360
diff changeset
  2707
            dfh.write(data[1])
4981
e7131935fbb3 revlog: simplify addrevision
Matt Mackall <mpm@selenic.com>
parents: 4980
diff changeset
  2708
            ifh.write(entry)
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
  2709
        else:
4996
a0d37976cd5b revlog: avoid some unnecessary seek/tell syscalls
Matt Mackall <mpm@selenic.com>
parents: 4994
diff changeset
  2710
            offset += curr * self._io.size
5324
8409a2e3a78d revlog: fix inlined revision transaction extra data (issue 749)
Patrick Mezard <pmezard@gmail.com>
parents: 5007
diff changeset
  2711
            transaction.add(self.indexfile, offset, curr)
4981
e7131935fbb3 revlog: simplify addrevision
Matt Mackall <mpm@selenic.com>
parents: 4980
diff changeset
  2712
            ifh.write(entry)
3390
a74addddd092 make revlog.addgroup pass its file handles to addrevision
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 3360
diff changeset
  2713
            ifh.write(data[0])
a74addddd092 make revlog.addgroup pass its file handles to addrevision
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 3360
diff changeset
  2714
            ifh.write(data[1])
35974
9ba1d0c724e2 revlog: rename 'self.checkinlinesize' into '_enforceinlinesize'
Boris Feld <boris.feld@octobus.net>
parents: 35973
diff changeset
  2715
            self._enforceinlinesize(transaction, ifh)
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
  2716
34291
1db9abf407c5 revlog: add revmap back to revlog.addgroup
Durham Goode <durham@fb.com>
parents: 34250
diff changeset
  2717
    def addgroup(self, deltas, linkmapper, transaction, addrevisioncb=None):
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
  2718
        """
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
  2719
        add a delta group
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  2720
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
  2721
        given a set of deltas, add them to the revision log. the
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
  2722
        first delta is against its parent, which should be in our
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
  2723
        log, the rest are against the previous delta.
25822
00e3f909907f revlog: add support for a callback whenever revisions are added
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
  2724
00e3f909907f revlog: add support for a callback whenever revisions are added
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
  2725
        If ``addrevisioncb`` is defined, it will be called with arguments of
00e3f909907f revlog: add support for a callback whenever revisions are added
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
  2726
        this revlog and the node that was added.
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
  2727
        """
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
  2728
32868
ef015ba5ba2e revlog: rename list of nodes from "content" to "nodes"
Martin von Zweigbergk <martinvonz@google.com>
parents: 32867
diff changeset
  2729
        nodes = []
515
03f27b1381f9 Whitespace cleanups
mpm@selenic.com
parents: 484
diff changeset
  2730
12624
557988c691d1 revlog.addgroup(): always use _addrevision() to add new revlog entries
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 12623
diff changeset
  2731
        r = len(self)
557988c691d1 revlog.addgroup(): always use _addrevision() to add new revlog entries
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 12623
diff changeset
  2732
        end = 0
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  2733
        if r:
12624
557988c691d1 revlog.addgroup(): always use _addrevision() to add new revlog entries
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 12623
diff changeset
  2734
            end = self.end(r - 1)
35968
91b3b11565e5 revlog: move index file opening in a method
Boris Feld <boris.feld@octobus.net>
parents: 35967
diff changeset
  2735
        ifh = self._indexfp("a+")
4996
a0d37976cd5b revlog: avoid some unnecessary seek/tell syscalls
Matt Mackall <mpm@selenic.com>
parents: 4994
diff changeset
  2736
        isize = r * self._io.size
4982
9672e3c42b0c revlog: change _inline from a function to a variable
Matt Mackall <mpm@selenic.com>
parents: 4981
diff changeset
  2737
        if self._inline:
4996
a0d37976cd5b revlog: avoid some unnecessary seek/tell syscalls
Matt Mackall <mpm@selenic.com>
parents: 4994
diff changeset
  2738
            transaction.add(self.indexfile, end + isize, r)
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
  2739
            dfh = None
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
  2740
        else:
4996
a0d37976cd5b revlog: avoid some unnecessary seek/tell syscalls
Matt Mackall <mpm@selenic.com>
parents: 4994
diff changeset
  2741
            transaction.add(self.indexfile, isize, r)
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
  2742
            transaction.add(self.datafile, end)
35967
61326dd7cb8d revlog: move datafile opening in a method
Boris Feld <boris.feld@octobus.net>
parents: 35845
diff changeset
  2743
            dfh = self._datafp("a+")
24255
4bfe9f2d9761 revlog: addgroup checks if incoming deltas add censored revs, sets flag bit
Mike Edgar <adgar@google.com>
parents: 24123
diff changeset
  2744
        def flush():
4bfe9f2d9761 revlog: addgroup checks if incoming deltas add censored revs, sets flag bit
Mike Edgar <adgar@google.com>
parents: 24123
diff changeset
  2745
            if dfh:
4bfe9f2d9761 revlog: addgroup checks if incoming deltas add censored revs, sets flag bit
Mike Edgar <adgar@google.com>
parents: 24123
diff changeset
  2746
                dfh.flush()
4bfe9f2d9761 revlog: addgroup checks if incoming deltas add censored revs, sets flag bit
Mike Edgar <adgar@google.com>
parents: 24123
diff changeset
  2747
            ifh.flush()
6261
7c8101b5ceb1 revlog: make sure the files are closed after an exception happens
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 6228
diff changeset
  2748
        try:
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
  2749
            deltacomputer = _deltacomputer(self)
6261
7c8101b5ceb1 revlog: make sure the files are closed after an exception happens
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 6228
diff changeset
  2750
            # loop through our set of deltas
34148
c8b6ed51386b changegroup: remove changegroup dependency from revlog.addgroup
Durham Goode <durham@fb.com>
parents: 34147
diff changeset
  2751
            for data in deltas:
34291
1db9abf407c5 revlog: add revmap back to revlog.addgroup
Durham Goode <durham@fb.com>
parents: 34250
diff changeset
  2752
                node, p1, p2, linknode, deltabase, delta, flags = data
1db9abf407c5 revlog: add revmap back to revlog.addgroup
Durham Goode <durham@fb.com>
parents: 34250
diff changeset
  2753
                link = linkmapper(linknode)
34148
c8b6ed51386b changegroup: remove changegroup dependency from revlog.addgroup
Durham Goode <durham@fb.com>
parents: 34147
diff changeset
  2754
                flags = flags or REVIDX_DEFAULT_FLAGS
12336
9d234f7d8a77 bundle: move chunk parsing into unbundle class
Matt Mackall <mpm@selenic.com>
parents: 12335
diff changeset
  2755
32868
ef015ba5ba2e revlog: rename list of nodes from "content" to "nodes"
Martin von Zweigbergk <martinvonz@google.com>
parents: 32867
diff changeset
  2756
                nodes.append(node)
15890
e234eda20984 revlog: make addgroup returns a list of node contained in the added source
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 15835
diff changeset
  2757
14196
e7483ec3c374 revlog: remove support for punched/shallow
Sune Foldager <cryo@cyanite.org>
parents: 14195
diff changeset
  2758
                if node in self.nodemap:
6261
7c8101b5ceb1 revlog: make sure the files are closed after an exception happens
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 6228
diff changeset
  2759
                    # this can happen if two branches make the same change
7c8101b5ceb1 revlog: make sure the files are closed after an exception happens
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 6228
diff changeset
  2760
                    continue
192
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
  2761
6261
7c8101b5ceb1 revlog: make sure the files are closed after an exception happens
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 6228
diff changeset
  2762
                for p in (p1, p2):
16686
67964cda8701 cleanup: "not x in y" -> "x not in y"
Brodie Rao <brodie@sf.io>
parents: 16665
diff changeset
  2763
                    if p not in self.nodemap:
14196
e7483ec3c374 revlog: remove support for punched/shallow
Sune Foldager <cryo@cyanite.org>
parents: 14195
diff changeset
  2764
                        raise LookupError(p, self.indexfile,
e7483ec3c374 revlog: remove support for punched/shallow
Sune Foldager <cryo@cyanite.org>
parents: 14195
diff changeset
  2765
                                          _('unknown parent'))
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  2766
14141
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14075
diff changeset
  2767
                if deltabase not in self.nodemap:
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14075
diff changeset
  2768
                    raise LookupError(deltabase, self.indexfile,
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14075
diff changeset
  2769
                                      _('unknown delta base'))
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  2770
14141
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14075
diff changeset
  2771
                baserev = self.rev(deltabase)
24120
a450e0a2ba0a revlog: in addgroup, reject ill-formed deltas based on censored nodes
Mike Edgar <adgar@google.com>
parents: 24118
diff changeset
  2772
a450e0a2ba0a revlog: in addgroup, reject ill-formed deltas based on censored nodes
Mike Edgar <adgar@google.com>
parents: 24118
diff changeset
  2773
                if baserev != nullrev and self.iscensored(baserev):
a450e0a2ba0a revlog: in addgroup, reject ill-formed deltas based on censored nodes
Mike Edgar <adgar@google.com>
parents: 24118
diff changeset
  2774
                    # if base is censored, delta must be full replacement in a
a450e0a2ba0a revlog: in addgroup, reject ill-formed deltas based on censored nodes
Mike Edgar <adgar@google.com>
parents: 24118
diff changeset
  2775
                    # single patch operation
a450e0a2ba0a revlog: in addgroup, reject ill-formed deltas based on censored nodes
Mike Edgar <adgar@google.com>
parents: 24118
diff changeset
  2776
                    hlen = struct.calcsize(">lll")
a450e0a2ba0a revlog: in addgroup, reject ill-formed deltas based on censored nodes
Mike Edgar <adgar@google.com>
parents: 24118
diff changeset
  2777
                    oldlen = self.rawsize(baserev)
a450e0a2ba0a revlog: in addgroup, reject ill-formed deltas based on censored nodes
Mike Edgar <adgar@google.com>
parents: 24118
diff changeset
  2778
                    newlen = len(delta) - hlen
a450e0a2ba0a revlog: in addgroup, reject ill-formed deltas based on censored nodes
Mike Edgar <adgar@google.com>
parents: 24118
diff changeset
  2779
                    if delta[:hlen] != mdiff.replacediffheader(oldlen, newlen):
a450e0a2ba0a revlog: in addgroup, reject ill-formed deltas based on censored nodes
Mike Edgar <adgar@google.com>
parents: 24118
diff changeset
  2780
                        raise error.CensoredBaseError(self.indexfile,
a450e0a2ba0a revlog: in addgroup, reject ill-formed deltas based on censored nodes
Mike Edgar <adgar@google.com>
parents: 24118
diff changeset
  2781
                                                      self.node(baserev))
a450e0a2ba0a revlog: in addgroup, reject ill-formed deltas based on censored nodes
Mike Edgar <adgar@google.com>
parents: 24118
diff changeset
  2782
27433
12f727a5b434 changegroup: add flags field to cg3 delta header
Mike Edgar <adgar@google.com>
parents: 27431
diff changeset
  2783
                if not flags and self._peek_iscensored(baserev, delta, flush):
24255
4bfe9f2d9761 revlog: addgroup checks if incoming deltas add censored revs, sets flag bit
Mike Edgar <adgar@google.com>
parents: 24123
diff changeset
  2784
                    flags |= REVIDX_ISCENSORED
4bfe9f2d9761 revlog: addgroup checks if incoming deltas add censored revs, sets flag bit
Mike Edgar <adgar@google.com>
parents: 24123
diff changeset
  2785
26243
836291420d53 revlog: optionally cache the full text when adding revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26242
diff changeset
  2786
                # We assume consumers of addrevisioncb will want to retrieve
836291420d53 revlog: optionally cache the full text when adding revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26242
diff changeset
  2787
                # the added revision, which will require a call to
836291420d53 revlog: optionally cache the full text when adding revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26242
diff changeset
  2788
                # revision(). revision() will fast path if there is a cache
836291420d53 revlog: optionally cache the full text when adding revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26242
diff changeset
  2789
                # hit. So, we tell _addrevision() to always cache in this case.
30743
2df983125d37 revlog: add 'raw' argument to revision and _addrevision
Remi Chaintron <remi@fb.com>
parents: 30584
diff changeset
  2790
                # We're only using addgroup() in the context of changegroup
2df983125d37 revlog: add 'raw' argument to revision and _addrevision
Remi Chaintron <remi@fb.com>
parents: 30584
diff changeset
  2791
                # generation so the revision data can always be handled as raw
2df983125d37 revlog: add 'raw' argument to revision and _addrevision
Remi Chaintron <remi@fb.com>
parents: 30584
diff changeset
  2792
                # by the flagprocessor.
34147
b96cfc309ac5 revlog: refactor chain variable
Durham Goode <durham@fb.com>
parents: 34028
diff changeset
  2793
                self._addrevision(node, None, transaction, link,
b96cfc309ac5 revlog: refactor chain variable
Durham Goode <durham@fb.com>
parents: 34028
diff changeset
  2794
                                  p1, p2, flags, (baserev, delta),
b96cfc309ac5 revlog: refactor chain variable
Durham Goode <durham@fb.com>
parents: 34028
diff changeset
  2795
                                  ifh, dfh,
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
  2796
                                  alwayscache=bool(addrevisioncb),
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
  2797
                                  deltacomputer=deltacomputer)
25822
00e3f909907f revlog: add support for a callback whenever revisions are added
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
  2798
00e3f909907f revlog: add support for a callback whenever revisions are added
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
  2799
                if addrevisioncb:
34147
b96cfc309ac5 revlog: refactor chain variable
Durham Goode <durham@fb.com>
parents: 34028
diff changeset
  2800
                    addrevisioncb(self, node)
25822
00e3f909907f revlog: add support for a callback whenever revisions are added
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
  2801
12624
557988c691d1 revlog.addgroup(): always use _addrevision() to add new revlog entries
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 12623
diff changeset
  2802
                if not dfh and not self._inline:
557988c691d1 revlog.addgroup(): always use _addrevision() to add new revlog entries
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 12623
diff changeset
  2803
                    # addrevision switched from inline to conventional
557988c691d1 revlog.addgroup(): always use _addrevision() to add new revlog entries
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 12623
diff changeset
  2804
                    # reopen the index
13400
14f3795a5ed7 explicitly close files
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 13284
diff changeset
  2805
                    ifh.close()
35967
61326dd7cb8d revlog: move datafile opening in a method
Boris Feld <boris.feld@octobus.net>
parents: 35845
diff changeset
  2806
                    dfh = self._datafp("a+")
35968
91b3b11565e5 revlog: move index file opening in a method
Boris Feld <boris.feld@octobus.net>
parents: 35967
diff changeset
  2807
                    ifh = self._indexfp("a+")
6261
7c8101b5ceb1 revlog: make sure the files are closed after an exception happens
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 6228
diff changeset
  2808
        finally:
7c8101b5ceb1 revlog: make sure the files are closed after an exception happens
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 6228
diff changeset
  2809
            if dfh:
7c8101b5ceb1 revlog: make sure the files are closed after an exception happens
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 6228
diff changeset
  2810
                dfh.close()
7c8101b5ceb1 revlog: make sure the files are closed after an exception happens
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 6228
diff changeset
  2811
            ifh.close()
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  2812
32868
ef015ba5ba2e revlog: rename list of nodes from "content" to "nodes"
Martin von Zweigbergk <martinvonz@google.com>
parents: 32867
diff changeset
  2813
        return nodes
1493
1a216cb4ee64 verify: add check for mismatch of index and data length
Matt Mackall <mpm@selenic.com>
parents: 1469
diff changeset
  2814
24118
76f6ae06ddf5 revlog: add "iscensored()" to revlog public API
Mike Edgar <adgar@google.com>
parents: 24030
diff changeset
  2815
    def iscensored(self, rev):
76f6ae06ddf5 revlog: add "iscensored()" to revlog public API
Mike Edgar <adgar@google.com>
parents: 24030
diff changeset
  2816
        """Check if a file revision is censored."""
37443
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
  2817
        if not self._censorable:
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
  2818
            return False
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
  2819
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
  2820
        return self.flags(rev) & REVIDX_ISCENSORED
24118
76f6ae06ddf5 revlog: add "iscensored()" to revlog public API
Mike Edgar <adgar@google.com>
parents: 24030
diff changeset
  2821
24255
4bfe9f2d9761 revlog: addgroup checks if incoming deltas add censored revs, sets flag bit
Mike Edgar <adgar@google.com>
parents: 24123
diff changeset
  2822
    def _peek_iscensored(self, baserev, delta, flush):
4bfe9f2d9761 revlog: addgroup checks if incoming deltas add censored revs, sets flag bit
Mike Edgar <adgar@google.com>
parents: 24123
diff changeset
  2823
        """Quickly check if a delta produces a censored revision."""
37443
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
  2824
        if not self._censorable:
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
  2825
            return False
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
  2826
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
  2827
        # Fragile heuristic: unless new file meta keys are added alphabetically
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
  2828
        # preceding "censored", all censored revisions are prefixed by
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
  2829
        # "\1\ncensored:". A delta producing such a censored revision must be a
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
  2830
        # full-replacement delta, so we inspect the first and only patch in the
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
  2831
        # delta for this prefix.
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
  2832
        hlen = struct.calcsize(">lll")
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
  2833
        if len(delta) <= hlen:
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
  2834
            return False
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
  2835
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
  2836
        oldlen = self.rawsize(baserev)
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
  2837
        newlen = len(delta) - hlen
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
  2838
        if delta[:hlen] != mdiff.replacediffheader(oldlen, newlen):
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
  2839
            return False
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
  2840
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
  2841
        add = "\1\ncensored:"
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
  2842
        addlen = len(add)
65250a66b55c revlog: move censor logic into main revlog class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37442
diff changeset
  2843
        return newlen >= addlen and delta[hlen:hlen + addlen] == add
24255
4bfe9f2d9761 revlog: addgroup checks if incoming deltas add censored revs, sets flag bit
Mike Edgar <adgar@google.com>
parents: 24123
diff changeset
  2844
20074
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
  2845
    def getstrippoint(self, minlink):
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
  2846
        """find the minimum rev that must be stripped to strip the linkrev
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
  2847
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
  2848
        Returns a tuple containing the minimum rev and a set of all revs that
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
  2849
        have linkrevs that will be broken by this strip.
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
  2850
        """
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
  2851
        brokenrevs = set()
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
  2852
        strippoint = len(self)
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
  2853
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
  2854
        heads = {}
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
  2855
        futurelargelinkrevs = set()
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
  2856
        for head in self.headrevs():
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
  2857
            headlinkrev = self.linkrev(head)
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
  2858
            heads[head] = headlinkrev
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
  2859
            if headlinkrev >= minlink:
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
  2860
                futurelargelinkrevs.add(headlinkrev)
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
  2861
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
  2862
        # This algorithm involves walking down the rev graph, starting at the
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
  2863
        # heads. Since the revs are topologically sorted according to linkrev,
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
  2864
        # once all head linkrevs are below the minlink, we know there are
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
  2865
        # no more revs that could have a linkrev greater than minlink.
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
  2866
        # So we can stop walking.
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
  2867
        while futurelargelinkrevs:
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
  2868
            strippoint -= 1
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
  2869
            linkrev = heads.pop(strippoint)
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
  2870
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
  2871
            if linkrev < minlink:
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
  2872
                brokenrevs.add(strippoint)
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
  2873
            else:
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
  2874
                futurelargelinkrevs.remove(linkrev)
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
  2875
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
  2876
            for p in self.parentrevs(strippoint):
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
  2877
                if p != nullrev:
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
  2878
                    plinkrev = self.linkrev(p)
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
  2879
                    heads[p] = plinkrev
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
  2880
                    if plinkrev >= minlink:
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
  2881
                        futurelargelinkrevs.add(plinkrev)
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
  2882
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
  2883
        return strippoint, brokenrevs
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
  2884
8073
e8a28556a0a8 strip: make repair.strip transactional to avoid repository corruption
Henrik Stuart <henrik.stuart@edlund.dk>
parents: 8017
diff changeset
  2885
    def strip(self, minlink, transaction):
5910
b9a830fa10f6 simplify revlog.strip interface and callers; add docstring
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5909
diff changeset
  2886
        """truncate the revlog on the first revision with a linkrev >= minlink
b9a830fa10f6 simplify revlog.strip interface and callers; add docstring
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5909
diff changeset
  2887
b9a830fa10f6 simplify revlog.strip interface and callers; add docstring
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5909
diff changeset
  2888
        This function is called when we're stripping revision minlink and
b9a830fa10f6 simplify revlog.strip interface and callers; add docstring
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5909
diff changeset
  2889
        its descendants from the repository.
b9a830fa10f6 simplify revlog.strip interface and callers; add docstring
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5909
diff changeset
  2890
b9a830fa10f6 simplify revlog.strip interface and callers; add docstring
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5909
diff changeset
  2891
        We have to remove all revisions with linkrev >= minlink, because
b9a830fa10f6 simplify revlog.strip interface and callers; add docstring
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5909
diff changeset
  2892
        the equivalent changelog revisions will be renumbered after the
b9a830fa10f6 simplify revlog.strip interface and callers; add docstring
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5909
diff changeset
  2893
        strip.
b9a830fa10f6 simplify revlog.strip interface and callers; add docstring
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5909
diff changeset
  2894
b9a830fa10f6 simplify revlog.strip interface and callers; add docstring
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5909
diff changeset
  2895
        So we truncate the revlog on the first of these revisions, and
b9a830fa10f6 simplify revlog.strip interface and callers; add docstring
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5909
diff changeset
  2896
        trust that the caller has saved the revisions that shouldn't be
15827
1dacf7672556 revlog: clarify strip docstring "readd" -> "re-add"
Steven Brown <StevenGBrown@gmail.com>
parents: 15407
diff changeset
  2897
        removed and that it'll re-add them after this truncation.
5910
b9a830fa10f6 simplify revlog.strip interface and callers; add docstring
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5909
diff changeset
  2898
        """
6750
fb42030d79d6 add __len__ and __iter__ methods to repo and revlog
Matt Mackall <mpm@selenic.com>
parents: 6703
diff changeset
  2899
        if len(self) == 0:
1535
7ae0ce7a3dc4 Add revlog.strip to truncate away revisions.
mason@suse.com
parents: 1533
diff changeset
  2900
            return
7ae0ce7a3dc4 Add revlog.strip to truncate away revisions.
mason@suse.com
parents: 1533
diff changeset
  2901
20074
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
  2902
        rev, _ = self.getstrippoint(minlink)
5fc2ae1c631b strip: add faster revlog strip computation
Durham Goode <durham@fb.com>
parents: 20073
diff changeset
  2903
        if rev == len(self):
5909
f45f7390c1c5 strip: calculate list of extra nodes to save and pass it to changegroupsubset
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5659
diff changeset
  2904
            return
1535
7ae0ce7a3dc4 Add revlog.strip to truncate away revisions.
mason@suse.com
parents: 1533
diff changeset
  2905
7ae0ce7a3dc4 Add revlog.strip to truncate away revisions.
mason@suse.com
parents: 1533
diff changeset
  2906
        # first truncate the files on disk
7ae0ce7a3dc4 Add revlog.strip to truncate away revisions.
mason@suse.com
parents: 1533
diff changeset
  2907
        end = self.start(rev)
4982
9672e3c42b0c revlog: change _inline from a function to a variable
Matt Mackall <mpm@selenic.com>
parents: 4981
diff changeset
  2908
        if not self._inline:
8073
e8a28556a0a8 strip: make repair.strip transactional to avoid repository corruption
Henrik Stuart <henrik.stuart@edlund.dk>
parents: 8017
diff changeset
  2909
            transaction.add(self.datafile, end)
4977
6cb30bc4ca32 revlog: parse revlogv0 indexes into v1 internally
Matt Mackall <mpm@selenic.com>
parents: 4976
diff changeset
  2910
            end = rev * self._io.size
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
  2911
        else:
4977
6cb30bc4ca32 revlog: parse revlogv0 indexes into v1 internally
Matt Mackall <mpm@selenic.com>
parents: 4976
diff changeset
  2912
            end += rev * self._io.size
2072
74d3f5336b66 Implement revlogng.
mason@suse.com
parents: 2002
diff changeset
  2913
8073
e8a28556a0a8 strip: make repair.strip transactional to avoid repository corruption
Henrik Stuart <henrik.stuart@edlund.dk>
parents: 8017
diff changeset
  2914
        transaction.add(self.indexfile, end)
1535
7ae0ce7a3dc4 Add revlog.strip to truncate away revisions.
mason@suse.com
parents: 1533
diff changeset
  2915
7ae0ce7a3dc4 Add revlog.strip to truncate away revisions.
mason@suse.com
parents: 1533
diff changeset
  2916
        # then reset internal state in memory to forget those revisions
4984
b4066fcbd6ba revlog: mark cache private
Matt Mackall <mpm@selenic.com>
parents: 4983
diff changeset
  2917
        self._cache = None
23306
f7a42f8e82bd revlog: cache chain info after calculating it for a rev (issue4452)
Siddharth Agarwal <sid0@fb.com>
parents: 23288
diff changeset
  2918
        self._chaininfocache = {}
8650
ef393d6ec030 revlog: refactor chunk cache interface again
Matt Mackall <mpm@selenic.com>
parents: 8643
diff changeset
  2919
        self._chunkclear()
38783
e7aa113b14f7 global: use pycompat.xrange()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38777
diff changeset
  2920
        for x in pycompat.xrange(rev, len(self)):
2072
74d3f5336b66 Implement revlogng.
mason@suse.com
parents: 2002
diff changeset
  2921
            del self.nodemap[self.node(x)]
1535
7ae0ce7a3dc4 Add revlog.strip to truncate away revisions.
mason@suse.com
parents: 1533
diff changeset
  2922
4979
06abdaf78788 revlog: add a magic null revision to our index
Matt Mackall <mpm@selenic.com>
parents: 4978
diff changeset
  2923
        del self.index[rev:-1]
37494
1ce7a55b09d1 revlog: reset _nodepos after strip
Joerg Sonnenberger <joerg@bec.de>
parents: 37449
diff changeset
  2924
        self._nodepos = None
1535
7ae0ce7a3dc4 Add revlog.strip to truncate away revisions.
mason@suse.com
parents: 1533
diff changeset
  2925
1493
1a216cb4ee64 verify: add check for mismatch of index and data length
Matt Mackall <mpm@selenic.com>
parents: 1469
diff changeset
  2926
    def checksize(self):
1a216cb4ee64 verify: add check for mismatch of index and data length
Matt Mackall <mpm@selenic.com>
parents: 1469
diff changeset
  2927
        expected = 0
6750
fb42030d79d6 add __len__ and __iter__ methods to repo and revlog
Matt Mackall <mpm@selenic.com>
parents: 6703
diff changeset
  2928
        if len(self):
fb42030d79d6 add __len__ and __iter__ methods to repo and revlog
Matt Mackall <mpm@selenic.com>
parents: 6703
diff changeset
  2929
            expected = max(0, self.end(len(self) - 1))
1667
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
  2930
1494
249ca10d37f4 Handle empty logs in repo.checksize
Matt Mackall <mpm@selenic.com>
parents: 1493
diff changeset
  2931
        try:
35972
82afb1a5ed94 revlog: use context manager for data file lifetime in checksize
Boris Feld <boris.feld@octobus.net>
parents: 35971
diff changeset
  2932
            with self._datafp() as f:
82afb1a5ed94 revlog: use context manager for data file lifetime in checksize
Boris Feld <boris.feld@octobus.net>
parents: 35971
diff changeset
  2933
                f.seek(0, 2)
82afb1a5ed94 revlog: use context manager for data file lifetime in checksize
Boris Feld <boris.feld@octobus.net>
parents: 35971
diff changeset
  2934
                actual = f.tell()
1667
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
  2935
            dd = actual - expected
25660
328739ea70c3 global: mass rewrite to use modern exception syntax
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25459
diff changeset
  2936
        except IOError as inst:
1667
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
  2937
            if inst.errno != errno.ENOENT:
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
  2938
                raise
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
  2939
            dd = 0
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
  2940
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
  2941
        try:
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
  2942
            f = self.opener(self.indexfile)
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
  2943
            f.seek(0, 2)
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
  2944
            actual = f.tell()
13400
14f3795a5ed7 explicitly close files
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 13284
diff changeset
  2945
            f.close()
4977
6cb30bc4ca32 revlog: parse revlogv0 indexes into v1 internally
Matt Mackall <mpm@selenic.com>
parents: 4976
diff changeset
  2946
            s = self._io.size
9029
0001e49f1c11 compat: use // for integer division
Alejandro Santos <alejolp@alejolp.com>
parents: 8658
diff changeset
  2947
            i = max(0, actual // s)
1667
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
  2948
            di = actual - (i * s)
4982
9672e3c42b0c revlog: change _inline from a function to a variable
Matt Mackall <mpm@selenic.com>
parents: 4981
diff changeset
  2949
            if self._inline:
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
  2950
                databytes = 0
6750
fb42030d79d6 add __len__ and __iter__ methods to repo and revlog
Matt Mackall <mpm@selenic.com>
parents: 6703
diff changeset
  2951
                for r in self:
5312
fb070713ff36 revlog: more robust for damaged indexes
Matt Mackall <mpm@selenic.com>
parents: 5007
diff changeset
  2952
                    databytes += max(0, self.length(r))
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
  2953
                dd = 0
6750
fb42030d79d6 add __len__ and __iter__ methods to repo and revlog
Matt Mackall <mpm@selenic.com>
parents: 6703
diff changeset
  2954
                di = actual - len(self) * s - databytes
25660
328739ea70c3 global: mass rewrite to use modern exception syntax
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25459
diff changeset
  2955
        except IOError as inst:
1667
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
  2956
            if inst.errno != errno.ENOENT:
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
  2957
                raise
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
  2958
            di = 0
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
  2959
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
  2960
        return (dd, di)
6891
22cb82433842 revlog: add files method
Adrian Buehlmann <adrian@cadifra.com>
parents: 6872
diff changeset
  2961
22cb82433842 revlog: add files method
Adrian Buehlmann <adrian@cadifra.com>
parents: 6872
diff changeset
  2962
    def files(self):
10282
08a0f04b56bd many, many trivial check-code fixups
Matt Mackall <mpm@selenic.com>
parents: 10264
diff changeset
  2963
        res = [self.indexfile]
6891
22cb82433842 revlog: add files method
Adrian Buehlmann <adrian@cadifra.com>
parents: 6872
diff changeset
  2964
        if not self._inline:
22cb82433842 revlog: add files method
Adrian Buehlmann <adrian@cadifra.com>
parents: 6872
diff changeset
  2965
            res.append(self.datafile)
22cb82433842 revlog: add files method
Adrian Buehlmann <adrian@cadifra.com>
parents: 6872
diff changeset
  2966
        return res
30778
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  2967
39231
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  2968
    def emitrevisiondeltas(self, requests):
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  2969
        frev = self.rev
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  2970
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  2971
        prevrev = None
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  2972
        for request in requests:
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  2973
            node = request.node
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  2974
            rev = frev(node)
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  2975
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  2976
            if prevrev is None:
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  2977
                prevrev = self.index[rev][5]
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  2978
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  2979
            # Requesting a full revision.
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  2980
            if request.basenode == nullid:
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  2981
                baserev = nullrev
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  2982
            # Requesting an explicit revision.
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  2983
            elif request.basenode is not None:
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  2984
                baserev = frev(request.basenode)
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  2985
            # Allowing us to choose.
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  2986
            else:
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  2987
                p1rev, p2rev = self.parentrevs(rev)
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  2988
                deltaparentrev = self.deltaparent(rev)
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  2989
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  2990
                # Avoid sending full revisions when delta parent is null. Pick
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  2991
                # prev in that case. It's tempting to pick p1 in this case, as
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  2992
                # p1 will be smaller in the common case. However, computing a
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  2993
                # delta against p1 may require resolving the raw text of p1,
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  2994
                # which could be expensive. The revlog caches should have prev
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  2995
                # cached, meaning less CPU for delta generation. There is
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  2996
                # likely room to add a flag and/or config option to control this
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  2997
                # behavior.
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  2998
                if deltaparentrev == nullrev and self.storedeltachains:
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  2999
                    baserev = prevrev
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  3000
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  3001
                # Revlog is configured to use full snapshot for a reason.
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  3002
                # Stick to full snapshot.
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  3003
                elif deltaparentrev == nullrev:
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  3004
                    baserev = nullrev
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  3005
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  3006
                # Pick previous when we can't be sure the base is available
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  3007
                # on consumer.
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  3008
                elif deltaparentrev not in (p1rev, p2rev, prevrev):
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  3009
                    baserev = prevrev
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  3010
                else:
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  3011
                    baserev = deltaparentrev
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  3012
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  3013
                if baserev != nullrev and not self.candelta(baserev, rev):
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  3014
                    baserev = nullrev
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  3015
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  3016
            revision = None
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  3017
            delta = None
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  3018
            baserevisionsize = None
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  3019
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  3020
            if self.iscensored(baserev) or self.iscensored(rev):
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  3021
                try:
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  3022
                    revision = self.revision(node, raw=True)
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  3023
                except error.CensoredNodeError as e:
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  3024
                    revision = e.tombstone
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  3025
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  3026
                if baserev != nullrev:
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  3027
                    baserevisionsize = self.rawsize(baserev)
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  3028
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  3029
            elif baserev == nullrev:
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  3030
                revision = self.revision(node, raw=True)
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  3031
            else:
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  3032
                delta = self.revdiff(baserev, rev)
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  3033
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  3034
            extraflags = REVIDX_ELLIPSIS if request.ellipsis else 0
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  3035
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  3036
            yield revlogrevisiondelta(
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  3037
                node=node,
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  3038
                p1node=request.p1node,
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  3039
                p2node=request.p2node,
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  3040
                linknode=request.linknode,
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  3041
                basenode=self.node(baserev),
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  3042
                flags=self.flags(rev) | extraflags,
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  3043
                baserevisionsize=baserevisionsize,
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  3044
                revision=revision,
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  3045
                delta=delta)
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  3046
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  3047
            prevrev = rev
b41d023a412a repository: establish API for emitting revision deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39192
diff changeset
  3048
30778
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3049
    DELTAREUSEALWAYS = 'always'
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3050
    DELTAREUSESAMEREVS = 'samerevs'
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3051
    DELTAREUSENEVER = 'never'
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3052
35345
6226668a7169 upgrade: add a 'redeltafullall' mode
Boris Feld <boris.feld@octobus.net>
parents: 34898
diff changeset
  3053
    DELTAREUSEFULLADD = 'fulladd'
6226668a7169 upgrade: add a 'redeltafullall' mode
Boris Feld <boris.feld@octobus.net>
parents: 34898
diff changeset
  3054
6226668a7169 upgrade: add a 'redeltafullall' mode
Boris Feld <boris.feld@octobus.net>
parents: 34898
diff changeset
  3055
    DELTAREUSEALL = {'always', 'samerevs', 'never', 'fulladd'}
30778
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3056
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3057
    def clone(self, tr, destrevlog, addrevisioncb=None,
38736
93777d16a25d aggressivemergedeltas: rename variable internally
Boris Feld <boris.feld@octobus.net>
parents: 38718
diff changeset
  3058
              deltareuse=DELTAREUSESAMEREVS, deltabothparents=None):
30778
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3059
        """Copy this revlog to another, possibly with format changes.
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3060
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3061
        The destination revlog will contain the same revisions and nodes.
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3062
        However, it may not be bit-for-bit identical due to e.g. delta encoding
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3063
        differences.
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3064
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3065
        The ``deltareuse`` argument control how deltas from the existing revlog
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3066
        are preserved in the destination revlog. The argument can have the
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3067
        following values:
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3068
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3069
        DELTAREUSEALWAYS
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3070
           Deltas will always be reused (if possible), even if the destination
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3071
           revlog would not select the same revisions for the delta. This is the
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3072
           fastest mode of operation.
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3073
        DELTAREUSESAMEREVS
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3074
           Deltas will be reused if the destination revlog would pick the same
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3075
           revisions for the delta. This mode strikes a balance between speed
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3076
           and optimization.
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3077
        DELTAREUSENEVER
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3078
           Deltas will never be reused. This is the slowest mode of execution.
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3079
           This mode can be used to recompute deltas (e.g. if the diff/delta
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3080
           algorithm changes).
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3081
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3082
        Delta computation can be slow, so the choice of delta reuse policy can
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3083
        significantly affect run time.
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3084
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3085
        The default policy (``DELTAREUSESAMEREVS``) strikes a balance between
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3086
        two extremes. Deltas will be reused if they are appropriate. But if the
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3087
        delta could choose a better revision, it will do so. This means if you
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3088
        are converting a non-generaldelta revlog to a generaldelta revlog,
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3089
        deltas will be recomputed if the delta's parent isn't a parent of the
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3090
        revision.
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3091
38736
93777d16a25d aggressivemergedeltas: rename variable internally
Boris Feld <boris.feld@octobus.net>
parents: 38718
diff changeset
  3092
        In addition to the delta policy, the ``deltabothparents`` argument
30778
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3093
        controls whether to compute deltas against both parents for merges.
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3094
        By default, the current default is used.
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3095
        """
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3096
        if deltareuse not in self.DELTAREUSEALL:
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3097
            raise ValueError(_('value for deltareuse invalid: %s') % deltareuse)
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3098
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3099
        if len(destrevlog):
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3100
            raise ValueError(_('destination revlog is not empty'))
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3101
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3102
        if getattr(self, 'filteredrevs', None):
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3103
            raise ValueError(_('source revlog has filtered revisions'))
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3104
        if getattr(destrevlog, 'filteredrevs', None):
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3105
            raise ValueError(_('destination revlog has filtered revisions'))
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3106
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3107
        # lazydeltabase controls whether to reuse a cached delta, if possible.
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3108
        oldlazydeltabase = destrevlog._lazydeltabase
38736
93777d16a25d aggressivemergedeltas: rename variable internally
Boris Feld <boris.feld@octobus.net>
parents: 38718
diff changeset
  3109
        oldamd = destrevlog._deltabothparents
30778
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3110
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3111
        try:
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3112
            if deltareuse == self.DELTAREUSEALWAYS:
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3113
                destrevlog._lazydeltabase = True
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3114
            elif deltareuse == self.DELTAREUSESAMEREVS:
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3115
                destrevlog._lazydeltabase = False
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3116
38736
93777d16a25d aggressivemergedeltas: rename variable internally
Boris Feld <boris.feld@octobus.net>
parents: 38718
diff changeset
  3117
            destrevlog._deltabothparents = deltabothparents or oldamd
30778
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3118
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3119
            populatecachedelta = deltareuse in (self.DELTAREUSEALWAYS,
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3120
                                                self.DELTAREUSESAMEREVS)
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3121
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
  3122
            deltacomputer = _deltacomputer(destrevlog)
30778
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3123
            index = self.index
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3124
            for rev in self:
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3125
                entry = index[rev]
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3126
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3127
                # Some classes override linkrev to take filtered revs into
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3128
                # account. Use raw entry from index.
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3129
                flags = entry[0] & 0xffff
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3130
                linkrev = entry[4]
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3131
                p1 = index[entry[5]][7]
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3132
                p2 = index[entry[6]][7]
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3133
                node = entry[7]
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3134
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3135
                # (Possibly) reuse the delta from the revlog if allowed and
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3136
                # the revlog chunk is a delta.
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3137
                cachedelta = None
31754
5b93c6fdb391 revlog: use raw revisions in clone
Jun Wu <quark@fb.com>
parents: 31753
diff changeset
  3138
                rawtext = None
30778
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3139
                if populatecachedelta:
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3140
                    dp = self.deltaparent(rev)
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3141
                    if dp != nullrev:
36722
6bacb2f663cb py3: use bytes() instead of str()
Pulkit Goyal <7895pulkit@gmail.com>
parents: 36721
diff changeset
  3142
                        cachedelta = (dp, bytes(self._chunk(rev)))
30778
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3143
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3144
                if not cachedelta:
31754
5b93c6fdb391 revlog: use raw revisions in clone
Jun Wu <quark@fb.com>
parents: 31753
diff changeset
  3145
                    rawtext = self.revision(rev, raw=True)
30778
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3146
35345
6226668a7169 upgrade: add a 'redeltafullall' mode
Boris Feld <boris.feld@octobus.net>
parents: 34898
diff changeset
  3147
6226668a7169 upgrade: add a 'redeltafullall' mode
Boris Feld <boris.feld@octobus.net>
parents: 34898
diff changeset
  3148
                if deltareuse == self.DELTAREUSEFULLADD:
6226668a7169 upgrade: add a 'redeltafullall' mode
Boris Feld <boris.feld@octobus.net>
parents: 34898
diff changeset
  3149
                    destrevlog.addrevision(rawtext, tr, linkrev, p1, p2,
6226668a7169 upgrade: add a 'redeltafullall' mode
Boris Feld <boris.feld@octobus.net>
parents: 34898
diff changeset
  3150
                                           cachedelta=cachedelta,
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
  3151
                                           node=node, flags=flags,
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
  3152
                                           deltacomputer=deltacomputer)
35345
6226668a7169 upgrade: add a 'redeltafullall' mode
Boris Feld <boris.feld@octobus.net>
parents: 34898
diff changeset
  3153
                else:
6226668a7169 upgrade: add a 'redeltafullall' mode
Boris Feld <boris.feld@octobus.net>
parents: 34898
diff changeset
  3154
                    ifh = destrevlog.opener(destrevlog.indexfile, 'a+',
6226668a7169 upgrade: add a 'redeltafullall' mode
Boris Feld <boris.feld@octobus.net>
parents: 34898
diff changeset
  3155
                                            checkambig=False)
6226668a7169 upgrade: add a 'redeltafullall' mode
Boris Feld <boris.feld@octobus.net>
parents: 34898
diff changeset
  3156
                    dfh = None
6226668a7169 upgrade: add a 'redeltafullall' mode
Boris Feld <boris.feld@octobus.net>
parents: 34898
diff changeset
  3157
                    if not destrevlog._inline:
6226668a7169 upgrade: add a 'redeltafullall' mode
Boris Feld <boris.feld@octobus.net>
parents: 34898
diff changeset
  3158
                        dfh = destrevlog.opener(destrevlog.datafile, 'a+')
6226668a7169 upgrade: add a 'redeltafullall' mode
Boris Feld <boris.feld@octobus.net>
parents: 34898
diff changeset
  3159
                    try:
6226668a7169 upgrade: add a 'redeltafullall' mode
Boris Feld <boris.feld@octobus.net>
parents: 34898
diff changeset
  3160
                        destrevlog._addrevision(node, rawtext, tr, linkrev, p1,
35738
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
  3161
                                                p2, flags, cachedelta, ifh, dfh,
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object
Paul Morelle <paul.morelle@octobus.net>
parents: 35737
diff changeset
  3162
                                                deltacomputer=deltacomputer)
35345
6226668a7169 upgrade: add a 'redeltafullall' mode
Boris Feld <boris.feld@octobus.net>
parents: 34898
diff changeset
  3163
                    finally:
6226668a7169 upgrade: add a 'redeltafullall' mode
Boris Feld <boris.feld@octobus.net>
parents: 34898
diff changeset
  3164
                        if dfh:
6226668a7169 upgrade: add a 'redeltafullall' mode
Boris Feld <boris.feld@octobus.net>
parents: 34898
diff changeset
  3165
                            dfh.close()
6226668a7169 upgrade: add a 'redeltafullall' mode
Boris Feld <boris.feld@octobus.net>
parents: 34898
diff changeset
  3166
                        ifh.close()
30778
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3167
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3168
                if addrevisioncb:
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3169
                    addrevisioncb(self, rev, node)
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3170
        finally:
1c7368d1a25f revlog: add clone method
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30746
diff changeset
  3171
            destrevlog._lazydeltabase = oldlazydeltabase
38736
93777d16a25d aggressivemergedeltas: rename variable internally
Boris Feld <boris.feld@octobus.net>
parents: 38718
diff changeset
  3172
            destrevlog._deltabothparents = oldamd