annotate mercurial/revlogutils/sidedata.py @ 50825:973fbb27ab45

backout: migrate `opts` to native kwargs It will take a bit to unwind `cmdutil.commit`, so there's a conversion to byteskwargs there, without changing the type of `opts` in this function. That's also useful to flag it as needing to be changed.
author Matt Harbison <matt_harbison@yahoo.com>
date Sun, 20 Aug 2023 00:27:27 -0400
parents 6000f5b25c9b
children f4733654f144
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
43033
21025a4107d4 sidedata: add a new module with basic documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
1 # sidedata.py - Logic around store extra data alongside revlog revisions
21025a4107d4 sidedata: add a new module with basic documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
2 #
21025a4107d4 sidedata: add a new module with basic documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
3 # Copyright 2019 Pierre-Yves David <pierre-yves.david@octobus.net)
21025a4107d4 sidedata: add a new module with basic documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
4 #
21025a4107d4 sidedata: add a new module with basic documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
5 # This software may be used and distributed according to the terms of the
21025a4107d4 sidedata: add a new module with basic documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
6 # GNU General Public License version 2 or any later version.
21025a4107d4 sidedata: add a new module with basic documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
7 """core code for "sidedata" support
21025a4107d4 sidedata: add a new module with basic documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
8
21025a4107d4 sidedata: add a new module with basic documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
9 The "sidedata" are stored alongside the revision without actually being part of
21025a4107d4 sidedata: add a new module with basic documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
10 its content and not affecting its hash. It's main use cases is to cache
21025a4107d4 sidedata: add a new module with basic documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
11 important information related to a changesets.
21025a4107d4 sidedata: add a new module with basic documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
12
21025a4107d4 sidedata: add a new module with basic documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
13 The current implementation is experimental and subject to changes. Do not rely
21025a4107d4 sidedata: add a new module with basic documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
14 on it in production.
21025a4107d4 sidedata: add a new module with basic documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
15
46709
3d740058b467 sidedata: move to new sidedata storage in revlogv2
Raphaël Gomès <rgomes@octobus.net>
parents: 46195
diff changeset
16 Sidedata are stored in the revlog itself, thanks to a new version of the
3d740058b467 sidedata: move to new sidedata storage in revlogv2
Raphaël Gomès <rgomes@octobus.net>
parents: 46195
diff changeset
17 revlog. The following format is currently used::
43033
21025a4107d4 sidedata: add a new module with basic documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
18
21025a4107d4 sidedata: add a new module with basic documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
19 initial header:
21025a4107d4 sidedata: add a new module with basic documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
20 <number of sidedata; 2 bytes>
21025a4107d4 sidedata: add a new module with basic documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
21 sidedata (repeated N times):
21025a4107d4 sidedata: add a new module with basic documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
22 <sidedata-key; 2 bytes>
21025a4107d4 sidedata: add a new module with basic documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
23 <sidedata-entry-length: 4 bytes>
21025a4107d4 sidedata: add a new module with basic documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
24 <sidedata-content-sha1-digest: 20 bytes>
21025a4107d4 sidedata: add a new module with basic documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
25 <sidedata-content; X bytes>
21025a4107d4 sidedata: add a new module with basic documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
26 normal raw text:
21025a4107d4 sidedata: add a new module with basic documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
27 <all bytes remaining in the rawtext>
21025a4107d4 sidedata: add a new module with basic documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
28
46195
d6a9e690d620 comments: fix typos
Joerg Sonnenberger <joerg@bec.de>
parents: 45634
diff changeset
29 This is a simple and effective format. It should be enough to experiment with
43033
21025a4107d4 sidedata: add a new module with basic documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
30 the concept.
21025a4107d4 sidedata: add a new module with basic documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
31 """
21025a4107d4 sidedata: add a new module with basic documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
32
43034
294afb982a88 sidedata: add a function to read sidedata from revlog raw text
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43033
diff changeset
33
47085
3aab2330b7d3 sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents: 46709
diff changeset
34 import collections
43034
294afb982a88 sidedata: add a function to read sidedata from revlog raw text
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43033
diff changeset
35 import struct
294afb982a88 sidedata: add a function to read sidedata from revlog raw text
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43033
diff changeset
36
47085
3aab2330b7d3 sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents: 46709
diff changeset
37 from .. import error, requirements as requirementsmod
3aab2330b7d3 sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents: 46709
diff changeset
38 from ..revlogutils import constants, flagutil
44060
a61287a95dc3 core: migrate uses of hashlib.sha1 to hashutil.sha1
Augie Fackler <augie@google.com>
parents: 43506
diff changeset
39 from ..utils import hashutil
43034
294afb982a88 sidedata: add a function to read sidedata from revlog raw text
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43033
diff changeset
40
43040
ba4072c0a911 sidedata: test we can successfully write sidedata
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43037
diff changeset
41 ## sidedata type constant
ba4072c0a911 sidedata: test we can successfully write sidedata
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43037
diff changeset
42 # reserve a block for testing purposes.
ba4072c0a911 sidedata: test we can successfully write sidedata
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43037
diff changeset
43 SD_TEST1 = 1
ba4072c0a911 sidedata: test we can successfully write sidedata
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43037
diff changeset
44 SD_TEST2 = 2
ba4072c0a911 sidedata: test we can successfully write sidedata
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43037
diff changeset
45 SD_TEST3 = 3
ba4072c0a911 sidedata: test we can successfully write sidedata
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43037
diff changeset
46 SD_TEST4 = 4
ba4072c0a911 sidedata: test we can successfully write sidedata
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43037
diff changeset
47 SD_TEST5 = 5
ba4072c0a911 sidedata: test we can successfully write sidedata
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43037
diff changeset
48 SD_TEST6 = 6
ba4072c0a911 sidedata: test we can successfully write sidedata
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43037
diff changeset
49 SD_TEST7 = 7
ba4072c0a911 sidedata: test we can successfully write sidedata
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43037
diff changeset
50
43142
beed7ce61681 sidedatacopies: write copies information in sidedata when applicable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43077
diff changeset
51 # key to store copies related information
beed7ce61681 sidedatacopies: write copies information in sidedata when applicable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43077
diff changeset
52 SD_P1COPIES = 8
beed7ce61681 sidedatacopies: write copies information in sidedata when applicable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43077
diff changeset
53 SD_P2COPIES = 9
beed7ce61681 sidedatacopies: write copies information in sidedata when applicable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43077
diff changeset
54 SD_FILESADDED = 10
beed7ce61681 sidedatacopies: write copies information in sidedata when applicable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43077
diff changeset
55 SD_FILESREMOVED = 11
45634
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44060
diff changeset
56 SD_FILES = 12
43142
beed7ce61681 sidedatacopies: write copies information in sidedata when applicable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43077
diff changeset
57
43040
ba4072c0a911 sidedata: test we can successfully write sidedata
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43037
diff changeset
58 # internal format constant
43506
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43142
diff changeset
59 SIDEDATA_HEADER = struct.Struct('>H')
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43142
diff changeset
60 SIDEDATA_ENTRY = struct.Struct('>HL20s')
43034
294afb982a88 sidedata: add a function to read sidedata from revlog raw text
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43033
diff changeset
61
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43048
diff changeset
62
46709
3d740058b467 sidedata: move to new sidedata storage in revlogv2
Raphaël Gomès <rgomes@octobus.net>
parents: 46195
diff changeset
63 def serialize_sidedata(sidedata):
43035
ea83abf95630 sidedata: add a function to write sidedata into a raw text
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43034
diff changeset
64 sidedata = list(sidedata.items())
ea83abf95630 sidedata: add a function to write sidedata into a raw text
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43034
diff changeset
65 sidedata.sort()
46709
3d740058b467 sidedata: move to new sidedata storage in revlogv2
Raphaël Gomès <rgomes@octobus.net>
parents: 46195
diff changeset
66 buf = [SIDEDATA_HEADER.pack(len(sidedata))]
43035
ea83abf95630 sidedata: add a function to write sidedata into a raw text
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43034
diff changeset
67 for key, value in sidedata:
44060
a61287a95dc3 core: migrate uses of hashlib.sha1 to hashutil.sha1
Augie Fackler <augie@google.com>
parents: 43506
diff changeset
68 digest = hashutil.sha1(value).digest()
46709
3d740058b467 sidedata: move to new sidedata storage in revlogv2
Raphaël Gomès <rgomes@octobus.net>
parents: 46195
diff changeset
69 buf.append(SIDEDATA_ENTRY.pack(key, len(value), digest))
43035
ea83abf95630 sidedata: add a function to write sidedata into a raw text
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43034
diff changeset
70 for key, value in sidedata:
46709
3d740058b467 sidedata: move to new sidedata storage in revlogv2
Raphaël Gomès <rgomes@octobus.net>
parents: 46195
diff changeset
71 buf.append(value)
3d740058b467 sidedata: move to new sidedata storage in revlogv2
Raphaël Gomès <rgomes@octobus.net>
parents: 46195
diff changeset
72 buf = b''.join(buf)
3d740058b467 sidedata: move to new sidedata storage in revlogv2
Raphaël Gomès <rgomes@octobus.net>
parents: 46195
diff changeset
73 return buf
43035
ea83abf95630 sidedata: add a function to write sidedata into a raw text
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43034
diff changeset
74
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43048
diff changeset
75
46709
3d740058b467 sidedata: move to new sidedata storage in revlogv2
Raphaël Gomès <rgomes@octobus.net>
parents: 46195
diff changeset
76 def deserialize_sidedata(blob):
43034
294afb982a88 sidedata: add a function to read sidedata from revlog raw text
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43033
diff changeset
77 sidedata = {}
294afb982a88 sidedata: add a function to read sidedata from revlog raw text
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43033
diff changeset
78 offset = 0
46709
3d740058b467 sidedata: move to new sidedata storage in revlogv2
Raphaël Gomès <rgomes@octobus.net>
parents: 46195
diff changeset
79 (nbentry,) = SIDEDATA_HEADER.unpack(blob[: SIDEDATA_HEADER.size])
43034
294afb982a88 sidedata: add a function to read sidedata from revlog raw text
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43033
diff changeset
80 offset += SIDEDATA_HEADER.size
294afb982a88 sidedata: add a function to read sidedata from revlog raw text
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43033
diff changeset
81 dataoffset = SIDEDATA_HEADER.size + (SIDEDATA_ENTRY.size * nbentry)
294afb982a88 sidedata: add a function to read sidedata from revlog raw text
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43033
diff changeset
82 for i in range(nbentry):
294afb982a88 sidedata: add a function to read sidedata from revlog raw text
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43033
diff changeset
83 nextoffset = offset + SIDEDATA_ENTRY.size
46709
3d740058b467 sidedata: move to new sidedata storage in revlogv2
Raphaël Gomès <rgomes@octobus.net>
parents: 46195
diff changeset
84 key, size, storeddigest = SIDEDATA_ENTRY.unpack(blob[offset:nextoffset])
43034
294afb982a88 sidedata: add a function to read sidedata from revlog raw text
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43033
diff changeset
85 offset = nextoffset
294afb982a88 sidedata: add a function to read sidedata from revlog raw text
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43033
diff changeset
86 # read the data associated with that entry
294afb982a88 sidedata: add a function to read sidedata from revlog raw text
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43033
diff changeset
87 nextdataoffset = dataoffset + size
46709
3d740058b467 sidedata: move to new sidedata storage in revlogv2
Raphaël Gomès <rgomes@octobus.net>
parents: 46195
diff changeset
88 entrytext = bytes(blob[dataoffset:nextdataoffset])
44060
a61287a95dc3 core: migrate uses of hashlib.sha1 to hashutil.sha1
Augie Fackler <augie@google.com>
parents: 43506
diff changeset
89 readdigest = hashutil.sha1(entrytext).digest()
43034
294afb982a88 sidedata: add a function to read sidedata from revlog raw text
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43033
diff changeset
90 if storeddigest != readdigest:
294afb982a88 sidedata: add a function to read sidedata from revlog raw text
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43033
diff changeset
91 raise error.SidedataHashError(key, storeddigest, readdigest)
294afb982a88 sidedata: add a function to read sidedata from revlog raw text
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43033
diff changeset
92 sidedata[key] = entrytext
294afb982a88 sidedata: add a function to read sidedata from revlog raw text
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43033
diff changeset
93 dataoffset = nextdataoffset
46709
3d740058b467 sidedata: move to new sidedata storage in revlogv2
Raphaël Gomès <rgomes@octobus.net>
parents: 46195
diff changeset
94 return sidedata
47085
3aab2330b7d3 sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents: 46709
diff changeset
95
3aab2330b7d3 sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents: 46709
diff changeset
96
3aab2330b7d3 sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents: 46709
diff changeset
97 def get_sidedata_helpers(repo, remote_sd_categories, pull=False):
47086
8bd769b5c941 sidedata: move documentation about sidedata helpers to sidedata module
Raphaël Gomès <rgomes@octobus.net>
parents: 47085
diff changeset
98 """
8bd769b5c941 sidedata: move documentation about sidedata helpers to sidedata module
Raphaël Gomès <rgomes@octobus.net>
parents: 47085
diff changeset
99 Returns a dictionary mapping revlog types to tuples of
8bd769b5c941 sidedata: move documentation about sidedata helpers to sidedata module
Raphaël Gomès <rgomes@octobus.net>
parents: 47085
diff changeset
100 `(repo, computers, removers)`:
8bd769b5c941 sidedata: move documentation about sidedata helpers to sidedata module
Raphaël Gomès <rgomes@octobus.net>
parents: 47085
diff changeset
101 * `repo` is used as an argument for computers
8bd769b5c941 sidedata: move documentation about sidedata helpers to sidedata module
Raphaël Gomès <rgomes@octobus.net>
parents: 47085
diff changeset
102 * `computers` is a list of `(category, (keys, computer, flags)` that
8bd769b5c941 sidedata: move documentation about sidedata helpers to sidedata module
Raphaël Gomès <rgomes@octobus.net>
parents: 47085
diff changeset
103 compute the missing sidedata categories that were asked:
8bd769b5c941 sidedata: move documentation about sidedata helpers to sidedata module
Raphaël Gomès <rgomes@octobus.net>
parents: 47085
diff changeset
104 * `category` is the sidedata category
8bd769b5c941 sidedata: move documentation about sidedata helpers to sidedata module
Raphaël Gomès <rgomes@octobus.net>
parents: 47085
diff changeset
105 * `keys` are the sidedata keys to be affected
8bd769b5c941 sidedata: move documentation about sidedata helpers to sidedata module
Raphaël Gomès <rgomes@octobus.net>
parents: 47085
diff changeset
106 * `flags` is a bitmask (an integer) of flags to remove when
8bd769b5c941 sidedata: move documentation about sidedata helpers to sidedata module
Raphaël Gomès <rgomes@octobus.net>
parents: 47085
diff changeset
107 removing the category.
8bd769b5c941 sidedata: move documentation about sidedata helpers to sidedata module
Raphaël Gomès <rgomes@octobus.net>
parents: 47085
diff changeset
108 * `computer` is the function `(repo, store, rev, sidedata)` that
8bd769b5c941 sidedata: move documentation about sidedata helpers to sidedata module
Raphaël Gomès <rgomes@octobus.net>
parents: 47085
diff changeset
109 returns a tuple of
8bd769b5c941 sidedata: move documentation about sidedata helpers to sidedata module
Raphaël Gomès <rgomes@octobus.net>
parents: 47085
diff changeset
110 `(new sidedata dict, (flags to add, flags to remove))`.
8bd769b5c941 sidedata: move documentation about sidedata helpers to sidedata module
Raphaël Gomès <rgomes@octobus.net>
parents: 47085
diff changeset
111 For example, it will return `({}, (0, 1 << 15))` to return no
8bd769b5c941 sidedata: move documentation about sidedata helpers to sidedata module
Raphaël Gomès <rgomes@octobus.net>
parents: 47085
diff changeset
112 sidedata, with no flags to add and one flag to remove.
8bd769b5c941 sidedata: move documentation about sidedata helpers to sidedata module
Raphaël Gomès <rgomes@octobus.net>
parents: 47085
diff changeset
113 * `removers` will remove the keys corresponding to the categories
8bd769b5c941 sidedata: move documentation about sidedata helpers to sidedata module
Raphaël Gomès <rgomes@octobus.net>
parents: 47085
diff changeset
114 that are present, but not needed.
8bd769b5c941 sidedata: move documentation about sidedata helpers to sidedata module
Raphaël Gomès <rgomes@octobus.net>
parents: 47085
diff changeset
115 If both `computers` and `removers` are empty, sidedata will simply not
8bd769b5c941 sidedata: move documentation about sidedata helpers to sidedata module
Raphaël Gomès <rgomes@octobus.net>
parents: 47085
diff changeset
116 be transformed.
8bd769b5c941 sidedata: move documentation about sidedata helpers to sidedata module
Raphaël Gomès <rgomes@octobus.net>
parents: 47085
diff changeset
117 """
47085
3aab2330b7d3 sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents: 46709
diff changeset
118 # Computers for computing sidedata on-the-fly
3aab2330b7d3 sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents: 46709
diff changeset
119 sd_computers = collections.defaultdict(list)
3aab2330b7d3 sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents: 46709
diff changeset
120 # Computers for categories to remove from sidedata
3aab2330b7d3 sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents: 46709
diff changeset
121 sd_removers = collections.defaultdict(list)
3aab2330b7d3 sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents: 46709
diff changeset
122 to_generate = remote_sd_categories - repo._wanted_sidedata
3aab2330b7d3 sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents: 46709
diff changeset
123 to_remove = repo._wanted_sidedata - remote_sd_categories
3aab2330b7d3 sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents: 46709
diff changeset
124 if pull:
3aab2330b7d3 sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents: 46709
diff changeset
125 to_generate, to_remove = to_remove, to_generate
3aab2330b7d3 sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents: 46709
diff changeset
126
3aab2330b7d3 sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents: 46709
diff changeset
127 for revlog_kind, computers in repo._sidedata_computers.items():
3aab2330b7d3 sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents: 46709
diff changeset
128 for category, computer in computers.items():
3aab2330b7d3 sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents: 46709
diff changeset
129 if category in to_generate:
3aab2330b7d3 sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents: 46709
diff changeset
130 sd_computers[revlog_kind].append(computer)
3aab2330b7d3 sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents: 46709
diff changeset
131 if category in to_remove:
3aab2330b7d3 sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents: 46709
diff changeset
132 sd_removers[revlog_kind].append(computer)
3aab2330b7d3 sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents: 46709
diff changeset
133
3aab2330b7d3 sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents: 46709
diff changeset
134 sidedata_helpers = (repo, sd_computers, sd_removers)
3aab2330b7d3 sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents: 46709
diff changeset
135 return sidedata_helpers
3aab2330b7d3 sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents: 46709
diff changeset
136
3aab2330b7d3 sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents: 46709
diff changeset
137
3aab2330b7d3 sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents: 46709
diff changeset
138 def run_sidedata_helpers(store, sidedata_helpers, sidedata, rev):
3aab2330b7d3 sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents: 46709
diff changeset
139 """Returns the sidedata for the given revision after running through
3aab2330b7d3 sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents: 46709
diff changeset
140 the given helpers.
3aab2330b7d3 sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents: 46709
diff changeset
141 - `store`: the revlog this applies to (changelog, manifest, or filelog
3aab2330b7d3 sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents: 46709
diff changeset
142 instance)
47086
8bd769b5c941 sidedata: move documentation about sidedata helpers to sidedata module
Raphaël Gomès <rgomes@octobus.net>
parents: 47085
diff changeset
143 - `sidedata_helpers`: see `get_sidedata_helpers`
47085
3aab2330b7d3 sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents: 46709
diff changeset
144 - `sidedata`: previous sidedata at the given rev, if any
3aab2330b7d3 sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents: 46709
diff changeset
145 - `rev`: affected rev of `store`
3aab2330b7d3 sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents: 46709
diff changeset
146 """
3aab2330b7d3 sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents: 46709
diff changeset
147 repo, sd_computers, sd_removers = sidedata_helpers
3aab2330b7d3 sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents: 46709
diff changeset
148 kind = store.revlog_kind
3aab2330b7d3 sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents: 46709
diff changeset
149 flags_to_add = 0
3aab2330b7d3 sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents: 46709
diff changeset
150 flags_to_remove = 0
3aab2330b7d3 sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents: 46709
diff changeset
151 for _keys, sd_computer, _flags in sd_computers.get(kind, []):
3aab2330b7d3 sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents: 46709
diff changeset
152 sidedata, flags = sd_computer(repo, store, rev, sidedata)
3aab2330b7d3 sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents: 46709
diff changeset
153 flags_to_add |= flags[0]
3aab2330b7d3 sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents: 46709
diff changeset
154 flags_to_remove |= flags[1]
3aab2330b7d3 sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents: 46709
diff changeset
155 for keys, _computer, flags in sd_removers.get(kind, []):
3aab2330b7d3 sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents: 46709
diff changeset
156 for key in keys:
3aab2330b7d3 sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents: 46709
diff changeset
157 sidedata.pop(key, None)
3aab2330b7d3 sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents: 46709
diff changeset
158 flags_to_remove |= flags
3aab2330b7d3 sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents: 46709
diff changeset
159 return sidedata, (flags_to_add, flags_to_remove)
3aab2330b7d3 sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents: 46709
diff changeset
160
3aab2330b7d3 sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents: 46709
diff changeset
161
3aab2330b7d3 sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents: 46709
diff changeset
162 def set_sidedata_spec_for_repo(repo):
3aab2330b7d3 sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents: 46709
diff changeset
163 # prevent cycle metadata -> revlogutils.sidedata -> metadata
3aab2330b7d3 sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents: 46709
diff changeset
164 from .. import metadata
3aab2330b7d3 sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents: 46709
diff changeset
165
3aab2330b7d3 sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents: 46709
diff changeset
166 if requirementsmod.COPIESSDC_REQUIREMENT in repo.requirements:
3aab2330b7d3 sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents: 46709
diff changeset
167 repo.register_wanted_sidedata(SD_FILES)
3aab2330b7d3 sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents: 46709
diff changeset
168 repo.register_sidedata_computer(
3aab2330b7d3 sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents: 46709
diff changeset
169 constants.KIND_CHANGELOG,
3aab2330b7d3 sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents: 46709
diff changeset
170 SD_FILES,
3aab2330b7d3 sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents: 46709
diff changeset
171 (SD_FILES,),
3aab2330b7d3 sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents: 46709
diff changeset
172 metadata.copies_sidedata_computer,
3aab2330b7d3 sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents: 46709
diff changeset
173 flagutil.REVIDX_HASCOPIESINFO,
3aab2330b7d3 sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents: 46709
diff changeset
174 )