mercurial/revlogutils/constants.py
author Georges Racinet <georges.racinet@octobus.net>
Fri, 20 Oct 2023 08:54:49 +0200
changeset 51228 61a6ef876efd
parent 49768 bcae90c53def
child 51863 f4733654f144
permissions -rw-r--r--
rust-index: simplification in find_gca_candidates() `parent_seen` can be made a mutable ref, making this part more obvious, not needing to be commented so much. The micro-optimization of avoiding the union if `parent_seen` and `current_seen` agree is pushed down in the `union()` method of the fast, `u64` based bit set implementation (in case it matters).
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
47249
130c9f7ed914 revlog: add a "data compression mode" entry in the index tuple
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47234
diff changeset
     1
# revlogdeltas.py - constant used for revlog logic.
39329
729082bb9938 revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
     2
#
46819
d4ba4d51f85f contributor: change mentions of mpm to olivia
Raphaël Gomès <rgomes@octobus.net>
parents: 46704
diff changeset
     3
# Copyright 2005-2007 Olivia Mackall <olivia@selenic.com>
39329
729082bb9938 revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
     4
# Copyright 2018 Octobus <contact@octobus.net>
729082bb9938 revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
     5
#
729082bb9938 revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
     6
# This software may be used and distributed according to the terms of the
729082bb9938 revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
     7
# GNU General Public License version 2 or any later version.
729082bb9938 revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
     8
"""Helper class to compute deltas stored inside revlogs"""
729082bb9938 revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
     9
729082bb9938 revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
    10
46856
34e1fa4b548a revlog: move the details of revlog "v0" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46855
diff changeset
    11
import struct
34e1fa4b548a revlog: move the details of revlog "v0" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46855
diff changeset
    12
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43032
diff changeset
    13
from ..interfaces import repository
47395
a669404f0f4a revlog: add a function to build index entry tuple
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47393
diff changeset
    14
from .. import revlogutils
39329
729082bb9938 revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
    15
47072
4c041c71ec01 revlog: introduce an explicit tracking of what the revlog is about
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46859
diff changeset
    16
### Internal utily constants
4c041c71ec01 revlog: introduce an explicit tracking of what the revlog is about
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46859
diff changeset
    17
4c041c71ec01 revlog: introduce an explicit tracking of what the revlog is about
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46859
diff changeset
    18
KIND_CHANGELOG = 1001  # over 256 to not be comparable with a bytes
4c041c71ec01 revlog: introduce an explicit tracking of what the revlog is about
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46859
diff changeset
    19
KIND_MANIFESTLOG = 1002
4c041c71ec01 revlog: introduce an explicit tracking of what the revlog is about
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46859
diff changeset
    20
KIND_FILELOG = 1003
4c041c71ec01 revlog: introduce an explicit tracking of what the revlog is about
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46859
diff changeset
    21
KIND_OTHER = 1004
4c041c71ec01 revlog: introduce an explicit tracking of what the revlog is about
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46859
diff changeset
    22
4c041c71ec01 revlog: introduce an explicit tracking of what the revlog is about
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46859
diff changeset
    23
ALL_KINDS = {
4c041c71ec01 revlog: introduce an explicit tracking of what the revlog is about
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46859
diff changeset
    24
    KIND_CHANGELOG,
4c041c71ec01 revlog: introduce an explicit tracking of what the revlog is about
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46859
diff changeset
    25
    KIND_MANIFESTLOG,
4c041c71ec01 revlog: introduce an explicit tracking of what the revlog is about
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46859
diff changeset
    26
    KIND_FILELOG,
4c041c71ec01 revlog: introduce an explicit tracking of what the revlog is about
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46859
diff changeset
    27
    KIND_OTHER,
4c041c71ec01 revlog: introduce an explicit tracking of what the revlog is about
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46859
diff changeset
    28
}
4c041c71ec01 revlog: introduce an explicit tracking of what the revlog is about
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46859
diff changeset
    29
47393
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    30
### Index entry key
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    31
#
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    32
#
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    33
#    Internal details
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    34
#    ----------------
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    35
#
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    36
#    A large part of the revlog logic deals with revisions' "index entries", tuple
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    37
#    objects that contains the same "items" whatever the revlog version.
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    38
#    Different versions will have different ways of storing these items (sometimes
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    39
#    not having them at all), but the tuple will always be the same. New fields
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    40
#    are usually added at the end to avoid breaking existing code that relies
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    41
#    on the existing order. The field are defined as follows:
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    42
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    43
#    [0] offset:
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    44
#            The byte index of the start of revision data chunk.
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    45
#            That value is shifted up by 16 bits. use "offset = field >> 16" to
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    46
#            retrieve it.
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    47
#
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    48
#        flags:
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    49
#            A flag field that carries special information or changes the behavior
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    50
#            of the revision. (see `REVIDX_*` constants for details)
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    51
#            The flag field only occupies the first 16 bits of this field,
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    52
#            use "flags = field & 0xFFFF" to retrieve the value.
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    53
ENTRY_DATA_OFFSET = 0
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    54
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    55
#    [1] compressed length:
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    56
#            The size, in bytes, of the chunk on disk
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    57
ENTRY_DATA_COMPRESSED_LENGTH = 1
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    58
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    59
#    [2] uncompressed length:
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    60
#            The size, in bytes, of the full revision once reconstructed.
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    61
ENTRY_DATA_UNCOMPRESSED_LENGTH = 2
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    62
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    63
#    [3] base rev:
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    64
#            Either the base of the revision delta chain (without general
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    65
#            delta), or the base of the delta (stored in the data chunk)
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    66
#            with general delta.
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    67
ENTRY_DELTA_BASE = 3
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    68
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    69
#    [4] link rev:
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    70
#            Changelog revision number of the changeset introducing this
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    71
#            revision.
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    72
ENTRY_LINK_REV = 4
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    73
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    74
#    [5] parent 1 rev:
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    75
#            Revision number of the first parent
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    76
ENTRY_PARENT_1 = 5
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    77
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    78
#    [6] parent 2 rev:
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    79
#            Revision number of the second parent
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    80
ENTRY_PARENT_2 = 6
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    81
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    82
#    [7] node id:
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    83
#            The node id of the current revision
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    84
ENTRY_NODE_ID = 7
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    85
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    86
#    [8] sidedata offset:
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    87
#            The byte index of the start of the revision's side-data chunk.
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    88
ENTRY_SIDEDATA_OFFSET = 8
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    89
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    90
#    [9] sidedata chunk length:
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    91
#            The size, in bytes, of the revision's side-data chunk.
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    92
ENTRY_SIDEDATA_COMPRESSED_LENGTH = 9
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    93
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    94
#    [10] data compression mode:
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    95
#            two bits that detail the way the data chunk is compressed on disk.
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    96
#            (see "COMP_MODE_*" constants for details). For revlog version 0 and
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    97
#            1 this will always be COMP_MODE_INLINE.
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    98
ENTRY_DATA_COMPRESSION_MODE = 10
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
    99
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
   100
#    [11] side-data compression mode:
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
   101
#            two bits that detail the way the sidedata chunk is compressed on disk.
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
   102
#            (see "COMP_MODE_*" constants for details)
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
   103
ENTRY_SIDEDATA_COMPRESSION_MODE = 11
7a0ec25d5836 revlog: move entry documentation alongside new related constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
   104
48499
52034c42c09d rank: add a "rank" value to the revlog-entry tuple
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48498
diff changeset
   105
#    [12] Revision rank:
52034c42c09d rank: add a "rank" value to the revlog-entry tuple
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48498
diff changeset
   106
#            The number of revision under this one.
52034c42c09d rank: add a "rank" value to the revlog-entry tuple
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48498
diff changeset
   107
#
52034c42c09d rank: add a "rank" value to the revlog-entry tuple
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48498
diff changeset
   108
#            Formally this is defined as : rank(X) = len(ancestors(X) + X)
52034c42c09d rank: add a "rank" value to the revlog-entry tuple
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48498
diff changeset
   109
#
52034c42c09d rank: add a "rank" value to the revlog-entry tuple
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48498
diff changeset
   110
#            If rank == -1; then we do not have this information available.
52034c42c09d rank: add a "rank" value to the revlog-entry tuple
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48498
diff changeset
   111
#            Only `null` has a rank of 0.
52034c42c09d rank: add a "rank" value to the revlog-entry tuple
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48498
diff changeset
   112
ENTRY_RANK = 12
52034c42c09d rank: add a "rank" value to the revlog-entry tuple
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48498
diff changeset
   113
52034c42c09d rank: add a "rank" value to the revlog-entry tuple
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48498
diff changeset
   114
RANK_UNKNOWN = -1
52034c42c09d rank: add a "rank" value to the revlog-entry tuple
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48498
diff changeset
   115
46855
aba724bf550e revlog: add some comment in the header sections
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46819
diff changeset
   116
### main revlog header
aba724bf550e revlog: add some comment in the header sections
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46819
diff changeset
   117
47371
298d4400ea68 index: use an explicit constant for INDEX_HEADER format and use it for docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47270
diff changeset
   118
# We cannot rely on  Struct.format is inconsistent for python <=3.6 versus above
298d4400ea68 index: use an explicit constant for INDEX_HEADER format and use it for docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47270
diff changeset
   119
INDEX_HEADER_FMT = b">I"
298d4400ea68 index: use an explicit constant for INDEX_HEADER format and use it for docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47270
diff changeset
   120
INDEX_HEADER = struct.Struct(INDEX_HEADER_FMT)
46859
c6e23fb4bfb4 revlog: move the "index header" struct inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46858
diff changeset
   121
46855
aba724bf550e revlog: add some comment in the header sections
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46819
diff changeset
   122
## revlog version
39329
729082bb9938 revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
   123
REVLOGV0 = 0
729082bb9938 revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
   124
REVLOGV1 = 1
729082bb9938 revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
   125
# Dummy value until file format is finalized.
729082bb9938 revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
   126
REVLOGV2 = 0xDEAD
47266
921648d31553 changelogv2: use a dedicated version number
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47255
diff changeset
   127
# Dummy value until file format is finalized.
921648d31553 changelogv2: use a dedicated version number
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47255
diff changeset
   128
CHANGELOGV2 = 0xD34D
46855
aba724bf550e revlog: add some comment in the header sections
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46819
diff changeset
   129
aba724bf550e revlog: add some comment in the header sections
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46819
diff changeset
   130
##  global revlog header flags
41202
e7a2cc84dbc0 revlog: always enable generaldelta on version 2 revlogs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40047
diff changeset
   131
# Shared across v1 and v2.
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43032
diff changeset
   132
FLAG_INLINE_DATA = 1 << 16
41202
e7a2cc84dbc0 revlog: always enable generaldelta on version 2 revlogs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40047
diff changeset
   133
# Only used by v1, implied by v2.
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43032
diff changeset
   134
FLAG_GENERALDELTA = 1 << 17
39329
729082bb9938 revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
   135
REVLOG_DEFAULT_FLAGS = FLAG_INLINE_DATA
729082bb9938 revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
   136
REVLOG_DEFAULT_FORMAT = REVLOGV1
729082bb9938 revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
   137
REVLOG_DEFAULT_VERSION = REVLOG_DEFAULT_FORMAT | REVLOG_DEFAULT_FLAGS
47230
0e9105bf54cb revlog: unify checks for supported flag
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47077
diff changeset
   138
REVLOGV0_FLAGS = 0
39329
729082bb9938 revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
   139
REVLOGV1_FLAGS = FLAG_INLINE_DATA | FLAG_GENERALDELTA
41202
e7a2cc84dbc0 revlog: always enable generaldelta on version 2 revlogs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40047
diff changeset
   140
REVLOGV2_FLAGS = FLAG_INLINE_DATA
47266
921648d31553 changelogv2: use a dedicated version number
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47255
diff changeset
   141
CHANGELOGV2_FLAGS = 0
39329
729082bb9938 revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
   142
46855
aba724bf550e revlog: add some comment in the header sections
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46819
diff changeset
   143
### individual entry
aba724bf550e revlog: add some comment in the header sections
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46819
diff changeset
   144
46856
34e1fa4b548a revlog: move the details of revlog "v0" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46855
diff changeset
   145
## index v0:
34e1fa4b548a revlog: move the details of revlog "v0" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46855
diff changeset
   146
#  4 bytes: offset
34e1fa4b548a revlog: move the details of revlog "v0" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46855
diff changeset
   147
#  4 bytes: compressed length
34e1fa4b548a revlog: move the details of revlog "v0" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46855
diff changeset
   148
#  4 bytes: base rev
34e1fa4b548a revlog: move the details of revlog "v0" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46855
diff changeset
   149
#  4 bytes: link rev
34e1fa4b548a revlog: move the details of revlog "v0" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46855
diff changeset
   150
# 20 bytes: parent 1 nodeid
34e1fa4b548a revlog: move the details of revlog "v0" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46855
diff changeset
   151
# 20 bytes: parent 2 nodeid
34e1fa4b548a revlog: move the details of revlog "v0" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46855
diff changeset
   152
# 20 bytes: nodeid
34e1fa4b548a revlog: move the details of revlog "v0" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46855
diff changeset
   153
INDEX_ENTRY_V0 = struct.Struct(b">4l20s20s20s")
34e1fa4b548a revlog: move the details of revlog "v0" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46855
diff changeset
   154
46857
cc65cea90edb revlog: move the details of revlog "v1" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46856
diff changeset
   155
## index v1
cc65cea90edb revlog: move the details of revlog "v1" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46856
diff changeset
   156
#  6 bytes: offset
cc65cea90edb revlog: move the details of revlog "v1" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46856
diff changeset
   157
#  2 bytes: flags
cc65cea90edb revlog: move the details of revlog "v1" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46856
diff changeset
   158
#  4 bytes: compressed length
cc65cea90edb revlog: move the details of revlog "v1" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46856
diff changeset
   159
#  4 bytes: uncompressed length
cc65cea90edb revlog: move the details of revlog "v1" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46856
diff changeset
   160
#  4 bytes: base rev
cc65cea90edb revlog: move the details of revlog "v1" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46856
diff changeset
   161
#  4 bytes: link rev
cc65cea90edb revlog: move the details of revlog "v1" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46856
diff changeset
   162
#  4 bytes: parent 1 rev
cc65cea90edb revlog: move the details of revlog "v1" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46856
diff changeset
   163
#  4 bytes: parent 2 rev
cc65cea90edb revlog: move the details of revlog "v1" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46856
diff changeset
   164
# 32 bytes: nodeid
cc65cea90edb revlog: move the details of revlog "v1" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46856
diff changeset
   165
INDEX_ENTRY_V1 = struct.Struct(b">Qiiiiii20s12x")
cc65cea90edb revlog: move the details of revlog "v1" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46856
diff changeset
   166
assert INDEX_ENTRY_V1.size == 32 * 2
cc65cea90edb revlog: move the details of revlog "v1" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46856
diff changeset
   167
46858
85e3a630cad9 revlog: move the details of revlog "v2" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46857
diff changeset
   168
#  6 bytes: offset
85e3a630cad9 revlog: move the details of revlog "v2" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46857
diff changeset
   169
#  2 bytes: flags
85e3a630cad9 revlog: move the details of revlog "v2" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46857
diff changeset
   170
#  4 bytes: compressed length
85e3a630cad9 revlog: move the details of revlog "v2" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46857
diff changeset
   171
#  4 bytes: uncompressed length
85e3a630cad9 revlog: move the details of revlog "v2" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46857
diff changeset
   172
#  4 bytes: base rev
85e3a630cad9 revlog: move the details of revlog "v2" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46857
diff changeset
   173
#  4 bytes: link rev
85e3a630cad9 revlog: move the details of revlog "v2" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46857
diff changeset
   174
#  4 bytes: parent 1 rev
85e3a630cad9 revlog: move the details of revlog "v2" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46857
diff changeset
   175
#  4 bytes: parent 2 rev
85e3a630cad9 revlog: move the details of revlog "v2" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46857
diff changeset
   176
# 32 bytes: nodeid
85e3a630cad9 revlog: move the details of revlog "v2" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46857
diff changeset
   177
#  8 bytes: sidedata offset
85e3a630cad9 revlog: move the details of revlog "v2" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46857
diff changeset
   178
#  4 bytes: sidedata compressed length
47251
6bfa6c2c5f15 revlogv2: preserve the compression mode on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47249
diff changeset
   179
#  1 bytes: compression mode (2 lower bit are data_compression_mode)
6bfa6c2c5f15 revlogv2: preserve the compression mode on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47249
diff changeset
   180
#  19 bytes: Padding to align to 96 bytes (see RevlogV2Plan wiki page)
6bfa6c2c5f15 revlogv2: preserve the compression mode on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47249
diff changeset
   181
INDEX_ENTRY_V2 = struct.Struct(b">Qiiiiii20s12xQiB19x")
6bfa6c2c5f15 revlogv2: preserve the compression mode on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47249
diff changeset
   182
assert INDEX_ENTRY_V2.size == 32 * 3, INDEX_ENTRY_V2.size
46858
85e3a630cad9 revlog: move the details of revlog "v2" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46857
diff changeset
   183
47270
25ce16bf724b changelogv2: use a dedicated on disk format for changelogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
   184
#  6 bytes: offset
25ce16bf724b changelogv2: use a dedicated on disk format for changelogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
   185
#  2 bytes: flags
25ce16bf724b changelogv2: use a dedicated on disk format for changelogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
   186
#  4 bytes: compressed length
25ce16bf724b changelogv2: use a dedicated on disk format for changelogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
   187
#  4 bytes: uncompressed length
25ce16bf724b changelogv2: use a dedicated on disk format for changelogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
   188
#  4 bytes: parent 1 rev
25ce16bf724b changelogv2: use a dedicated on disk format for changelogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
   189
#  4 bytes: parent 2 rev
25ce16bf724b changelogv2: use a dedicated on disk format for changelogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
   190
# 32 bytes: nodeid
25ce16bf724b changelogv2: use a dedicated on disk format for changelogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
   191
#  8 bytes: sidedata offset
25ce16bf724b changelogv2: use a dedicated on disk format for changelogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
   192
#  4 bytes: sidedata compressed length
25ce16bf724b changelogv2: use a dedicated on disk format for changelogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
   193
#  1 bytes: compression mode (2 lower bit are data_compression_mode)
48500
c5d6c874766a rank: actually persist revision's rank in changelog-v2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48499
diff changeset
   194
#  4 bytes: changeset rank (i.e. `len(::REV)`)
c5d6c874766a rank: actually persist revision's rank in changelog-v2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48499
diff changeset
   195
#  23 bytes: Padding to align to 96 bytes (see RevlogV2Plan wiki page)
c5d6c874766a rank: actually persist revision's rank in changelog-v2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48499
diff changeset
   196
INDEX_ENTRY_CL_V2 = struct.Struct(b">Qiiii20s12xQiBi23x")
48498
d5137c00ab17 changelog-v2: fix an assertion error to display the right data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48497
diff changeset
   197
assert INDEX_ENTRY_CL_V2.size == 32 * 3, INDEX_ENTRY_CL_V2.size
48497
2c6084f67a86 changelog-v2: use helper constant in the code to pack/unpack entries
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
   198
INDEX_ENTRY_V2_IDX_OFFSET = 0
2c6084f67a86 changelog-v2: use helper constant in the code to pack/unpack entries
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
   199
INDEX_ENTRY_V2_IDX_COMPRESSED_LENGTH = 1
2c6084f67a86 changelog-v2: use helper constant in the code to pack/unpack entries
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
   200
INDEX_ENTRY_V2_IDX_UNCOMPRESSED_LENGTH = 2
2c6084f67a86 changelog-v2: use helper constant in the code to pack/unpack entries
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
   201
INDEX_ENTRY_V2_IDX_PARENT_1 = 3
2c6084f67a86 changelog-v2: use helper constant in the code to pack/unpack entries
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
   202
INDEX_ENTRY_V2_IDX_PARENT_2 = 4
2c6084f67a86 changelog-v2: use helper constant in the code to pack/unpack entries
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
   203
INDEX_ENTRY_V2_IDX_NODEID = 5
2c6084f67a86 changelog-v2: use helper constant in the code to pack/unpack entries
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
   204
INDEX_ENTRY_V2_IDX_SIDEDATA_OFFSET = 6
2c6084f67a86 changelog-v2: use helper constant in the code to pack/unpack entries
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
   205
INDEX_ENTRY_V2_IDX_SIDEDATA_COMPRESSED_LENGTH = 7
2c6084f67a86 changelog-v2: use helper constant in the code to pack/unpack entries
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
   206
INDEX_ENTRY_V2_IDX_COMPRESSION_MODE = 8
48500
c5d6c874766a rank: actually persist revision's rank in changelog-v2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48499
diff changeset
   207
INDEX_ENTRY_V2_IDX_RANK = 9
47270
25ce16bf724b changelogv2: use a dedicated on disk format for changelogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
   208
39329
729082bb9938 revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
   209
# revlog index flags
40047
8e398628a3f2 repository: define and use revision flag constants
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39506
diff changeset
   210
8e398628a3f2 repository: define and use revision flag constants
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39506
diff changeset
   211
# For historical reasons, revlog's internal flags were exposed via the
8e398628a3f2 repository: define and use revision flag constants
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39506
diff changeset
   212
# wire protocol and are even exposed in parts of the storage APIs.
8e398628a3f2 repository: define and use revision flag constants
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39506
diff changeset
   213
8e398628a3f2 repository: define and use revision flag constants
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39506
diff changeset
   214
# revision has censor metadata, must be verified
8e398628a3f2 repository: define and use revision flag constants
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39506
diff changeset
   215
REVIDX_ISCENSORED = repository.REVISION_FLAG_CENSORED
8e398628a3f2 repository: define and use revision flag constants
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39506
diff changeset
   216
# revision hash does not match data (narrowhg)
8e398628a3f2 repository: define and use revision flag constants
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39506
diff changeset
   217
REVIDX_ELLIPSIS = repository.REVISION_FLAG_ELLIPSIS
8e398628a3f2 repository: define and use revision flag constants
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39506
diff changeset
   218
# revision data is stored externally
8e398628a3f2 repository: define and use revision flag constants
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39506
diff changeset
   219
REVIDX_EXTSTORED = repository.REVISION_FLAG_EXTSTORED
45671
2d6aea053153 copies: add a HASCOPIESINFO flag to highlight rev with useful data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43076
diff changeset
   220
# revision changes files in a way that could affect copy tracing.
2d6aea053153 copies: add a HASCOPIESINFO flag to highlight rev with useful data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43076
diff changeset
   221
REVIDX_HASCOPIESINFO = repository.REVISION_FLAG_HASCOPIESINFO
39329
729082bb9938 revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
   222
REVIDX_DEFAULT_FLAGS = 0
729082bb9938 revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
   223
# stable order in which flags need to be processed and their processors applied
729082bb9938 revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
   224
REVIDX_FLAGS_ORDER = [
729082bb9938 revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
   225
    REVIDX_ISCENSORED,
729082bb9938 revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
   226
    REVIDX_ELLIPSIS,
729082bb9938 revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
   227
    REVIDX_EXTSTORED,
45671
2d6aea053153 copies: add a HASCOPIESINFO flag to highlight rev with useful data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43076
diff changeset
   228
    REVIDX_HASCOPIESINFO,
39329
729082bb9938 revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
   229
]
42730
92ac6b1697a7 flagutil: move REVIDX_KNOWN_FLAGS source of truth in flagutil (API)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41202
diff changeset
   230
39329
729082bb9938 revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
   231
# bitmark for flags that could cause rawdata content change
47077
119790e1c67c cg4: introduce protocol flag to signify the presence of sidedata
Raphaël Gomès <rgomes@octobus.net>
parents: 47072
diff changeset
   232
REVIDX_RAWTEXT_CHANGING_FLAGS = REVIDX_ISCENSORED | REVIDX_EXTSTORED
39506
b66ea3fc3a86 sparse-revlog: set max delta chain length to on thousand
Boris Feld <boris.feld@octobus.net>
parents: 39330
diff changeset
   233
47249
130c9f7ed914 revlog: add a "data compression mode" entry in the index tuple
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47234
diff changeset
   234
## chunk compression mode constants:
130c9f7ed914 revlog: add a "data compression mode" entry in the index tuple
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47234
diff changeset
   235
# These constants are used in revlog version >=2 to denote the compression used
130c9f7ed914 revlog: add a "data compression mode" entry in the index tuple
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47234
diff changeset
   236
# for a chunk.
130c9f7ed914 revlog: add a "data compression mode" entry in the index tuple
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47234
diff changeset
   237
47253
b876f0bf7366 revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47251
diff changeset
   238
# Chunk use no compression, the data stored on disk can be directly use as
b876f0bf7366 revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47251
diff changeset
   239
# chunk value. Without any header information prefixed.
b876f0bf7366 revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47251
diff changeset
   240
COMP_MODE_PLAIN = 0
b876f0bf7366 revlog: introduce a plain compression mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47251
diff changeset
   241
47255
ff9fd7107d11 revlog: implement a "default compression" mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47253
diff changeset
   242
# Chunk use the "default compression" for the revlog (usually defined in the
ff9fd7107d11 revlog: implement a "default compression" mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47253
diff changeset
   243
# revlog docket). A header is still used.
ff9fd7107d11 revlog: implement a "default compression" mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47253
diff changeset
   244
#
ff9fd7107d11 revlog: implement a "default compression" mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47253
diff changeset
   245
# XXX: keeping a header is probably not useful and we should probably drop it.
ff9fd7107d11 revlog: implement a "default compression" mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47253
diff changeset
   246
#
ff9fd7107d11 revlog: implement a "default compression" mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47253
diff changeset
   247
# XXX: The value of allow mixed type of compression in the revlog is unclear
ff9fd7107d11 revlog: implement a "default compression" mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47253
diff changeset
   248
#      and we should consider making PLAIN/DEFAULT the only available mode for
ff9fd7107d11 revlog: implement a "default compression" mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47253
diff changeset
   249
#      revlog v2, disallowing INLINE mode.
ff9fd7107d11 revlog: implement a "default compression" mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47253
diff changeset
   250
COMP_MODE_DEFAULT = 1
ff9fd7107d11 revlog: implement a "default compression" mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47253
diff changeset
   251
47249
130c9f7ed914 revlog: add a "data compression mode" entry in the index tuple
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47234
diff changeset
   252
# Chunk use a compression mode stored "inline" at the start of the chunk
130c9f7ed914 revlog: add a "data compression mode" entry in the index tuple
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47234
diff changeset
   253
# itself.  This is the mode always used for revlog version "0" and "1"
47395
a669404f0f4a revlog: add a function to build index entry tuple
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47393
diff changeset
   254
COMP_MODE_INLINE = revlogutils.COMP_MODE_INLINE
47249
130c9f7ed914 revlog: add a "data compression mode" entry in the index tuple
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47234
diff changeset
   255
47230
0e9105bf54cb revlog: unify checks for supported flag
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47077
diff changeset
   256
SUPPORTED_FLAGS = {
0e9105bf54cb revlog: unify checks for supported flag
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47077
diff changeset
   257
    REVLOGV0: REVLOGV0_FLAGS,
0e9105bf54cb revlog: unify checks for supported flag
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47077
diff changeset
   258
    REVLOGV1: REVLOGV1_FLAGS,
0e9105bf54cb revlog: unify checks for supported flag
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47077
diff changeset
   259
    REVLOGV2: REVLOGV2_FLAGS,
47266
921648d31553 changelogv2: use a dedicated version number
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47255
diff changeset
   260
    CHANGELOGV2: CHANGELOGV2_FLAGS,
47230
0e9105bf54cb revlog: unify checks for supported flag
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47077
diff changeset
   261
}
0e9105bf54cb revlog: unify checks for supported flag
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47077
diff changeset
   262
47231
4d1c893b9095 revlog: unify flag processing when loading index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47230
diff changeset
   263
_no = lambda flags: False
4d1c893b9095 revlog: unify flag processing when loading index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47230
diff changeset
   264
_yes = lambda flags: True
4d1c893b9095 revlog: unify flag processing when loading index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47230
diff changeset
   265
4d1c893b9095 revlog: unify flag processing when loading index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47230
diff changeset
   266
4d1c893b9095 revlog: unify flag processing when loading index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47230
diff changeset
   267
def _from_flag(flag):
4d1c893b9095 revlog: unify flag processing when loading index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47230
diff changeset
   268
    return lambda flags: bool(flags & flag)
4d1c893b9095 revlog: unify flag processing when loading index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47230
diff changeset
   269
4d1c893b9095 revlog: unify flag processing when loading index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47230
diff changeset
   270
4d1c893b9095 revlog: unify flag processing when loading index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47230
diff changeset
   271
FEATURES_BY_VERSION = {
4d1c893b9095 revlog: unify flag processing when loading index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47230
diff changeset
   272
    REVLOGV0: {
4d1c893b9095 revlog: unify flag processing when loading index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47230
diff changeset
   273
        b'inline': _no,
4d1c893b9095 revlog: unify flag processing when loading index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47230
diff changeset
   274
        b'generaldelta': _no,
4d1c893b9095 revlog: unify flag processing when loading index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47230
diff changeset
   275
        b'sidedata': False,
47234
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47231
diff changeset
   276
        b'docket': False,
47231
4d1c893b9095 revlog: unify flag processing when loading index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47230
diff changeset
   277
    },
4d1c893b9095 revlog: unify flag processing when loading index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47230
diff changeset
   278
    REVLOGV1: {
4d1c893b9095 revlog: unify flag processing when loading index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47230
diff changeset
   279
        b'inline': _from_flag(FLAG_INLINE_DATA),
4d1c893b9095 revlog: unify flag processing when loading index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47230
diff changeset
   280
        b'generaldelta': _from_flag(FLAG_GENERALDELTA),
4d1c893b9095 revlog: unify flag processing when loading index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47230
diff changeset
   281
        b'sidedata': False,
47234
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47231
diff changeset
   282
        b'docket': False,
47231
4d1c893b9095 revlog: unify flag processing when loading index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47230
diff changeset
   283
    },
4d1c893b9095 revlog: unify flag processing when loading index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47230
diff changeset
   284
    REVLOGV2: {
47234
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47231
diff changeset
   285
        # The point of inline-revlog is to reduce the number of files used in
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47231
diff changeset
   286
        # the store. Using a docket defeat this purpose. So we needs other
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47231
diff changeset
   287
        # means to reduce the number of files for revlogv2.
47231
4d1c893b9095 revlog: unify flag processing when loading index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47230
diff changeset
   288
        b'inline': _no,
4d1c893b9095 revlog: unify flag processing when loading index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47230
diff changeset
   289
        b'generaldelta': _yes,
4d1c893b9095 revlog: unify flag processing when loading index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47230
diff changeset
   290
        b'sidedata': True,
47234
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47231
diff changeset
   291
        b'docket': True,
47231
4d1c893b9095 revlog: unify flag processing when loading index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47230
diff changeset
   292
    },
47266
921648d31553 changelogv2: use a dedicated version number
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47255
diff changeset
   293
    CHANGELOGV2: {
921648d31553 changelogv2: use a dedicated version number
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47255
diff changeset
   294
        b'inline': _no,
921648d31553 changelogv2: use a dedicated version number
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47255
diff changeset
   295
        # General delta is useless for changelog since we don't do any delta
921648d31553 changelogv2: use a dedicated version number
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47255
diff changeset
   296
        b'generaldelta': _no,
921648d31553 changelogv2: use a dedicated version number
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47255
diff changeset
   297
        b'sidedata': True,
921648d31553 changelogv2: use a dedicated version number
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47255
diff changeset
   298
        b'docket': True,
921648d31553 changelogv2: use a dedicated version number
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47255
diff changeset
   299
    },
47231
4d1c893b9095 revlog: unify flag processing when loading index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47230
diff changeset
   300
}
4d1c893b9095 revlog: unify flag processing when loading index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47230
diff changeset
   301
47249
130c9f7ed914 revlog: add a "data compression mode" entry in the index tuple
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47234
diff changeset
   302
39506
b66ea3fc3a86 sparse-revlog: set max delta chain length to on thousand
Boris Feld <boris.feld@octobus.net>
parents: 39330
diff changeset
   303
SPARSE_REVLOG_MAX_CHAIN_LENGTH = 1000
49677
05db41701ece find-delta: pass the cache-delta usage policy alongside the cache-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48875
diff changeset
   304
05db41701ece find-delta: pass the cache-delta usage policy alongside the cache-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48875
diff changeset
   305
### What should be done with a cached delta and its base ?
05db41701ece find-delta: pass the cache-delta usage policy alongside the cache-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48875
diff changeset
   306
05db41701ece find-delta: pass the cache-delta usage policy alongside the cache-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48875
diff changeset
   307
# Ignore the cache when considering candidates.
05db41701ece find-delta: pass the cache-delta usage policy alongside the cache-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48875
diff changeset
   308
#
05db41701ece find-delta: pass the cache-delta usage policy alongside the cache-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48875
diff changeset
   309
# The cached delta might be used, but the delta base will not be scheduled for
05db41701ece find-delta: pass the cache-delta usage policy alongside the cache-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48875
diff changeset
   310
# usage earlier than in "normal" order.
05db41701ece find-delta: pass the cache-delta usage policy alongside the cache-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48875
diff changeset
   311
DELTA_BASE_REUSE_NO = 0
05db41701ece find-delta: pass the cache-delta usage policy alongside the cache-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48875
diff changeset
   312
05db41701ece find-delta: pass the cache-delta usage policy alongside the cache-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48875
diff changeset
   313
# Prioritize trying the cached delta base
05db41701ece find-delta: pass the cache-delta usage policy alongside the cache-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48875
diff changeset
   314
#
05db41701ece find-delta: pass the cache-delta usage policy alongside the cache-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48875
diff changeset
   315
# The delta base will be tested for validy first. So that the cached deltas get
05db41701ece find-delta: pass the cache-delta usage policy alongside the cache-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48875
diff changeset
   316
# used when possible.
05db41701ece find-delta: pass the cache-delta usage policy alongside the cache-delta
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48875
diff changeset
   317
DELTA_BASE_REUSE_TRY = 1
49768
bcae90c53def delta-find: add a delta-reuse policy that blindly accepts incoming deltas
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49677
diff changeset
   318
DELTA_BASE_REUSE_FORCE = 2