Mercurial > hg
annotate mercurial/revlogutils/sidedata.py @ 47819:c02ce6def30c stable
issue6528: implement _is_revision_affected_fast using callback
The delta comming from a bundle/stream does not exists in the revlog yet, so we
will need other way to retrieve the same information.
To prepare for this we split the function to use callbacks in the core logic.
Differential Revision: https://phab.mercurial-scm.org/D11268
author | Pierre-Yves David <pierre-yves.david@octobus.net> |
---|---|
date | Sat, 07 Aug 2021 12:39:01 +0200 |
parents | 8bd769b5c941 |
children | 6000f5b25c9b |
rev | line source |
---|---|
43033
21025a4107d4
sidedata: add a new module with basic documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
1 # sidedata.py - Logic around store extra data alongside revlog revisions |
21025a4107d4
sidedata: add a new module with basic documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
2 # |
21025a4107d4
sidedata: add a new module with basic documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
3 # Copyright 2019 Pierre-Yves David <pierre-yves.david@octobus.net) |
21025a4107d4
sidedata: add a new module with basic documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
4 # |
21025a4107d4
sidedata: add a new module with basic documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
5 # This software may be used and distributed according to the terms of the |
21025a4107d4
sidedata: add a new module with basic documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
6 # GNU General Public License version 2 or any later version. |
21025a4107d4
sidedata: add a new module with basic documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
7 """core code for "sidedata" support |
21025a4107d4
sidedata: add a new module with basic documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
8 |
21025a4107d4
sidedata: add a new module with basic documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
9 The "sidedata" are stored alongside the revision without actually being part of |
21025a4107d4
sidedata: add a new module with basic documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
10 its content and not affecting its hash. It's main use cases is to cache |
21025a4107d4
sidedata: add a new module with basic documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
11 important information related to a changesets. |
21025a4107d4
sidedata: add a new module with basic documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
12 |
21025a4107d4
sidedata: add a new module with basic documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
13 The current implementation is experimental and subject to changes. Do not rely |
21025a4107d4
sidedata: add a new module with basic documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
14 on it in production. |
21025a4107d4
sidedata: add a new module with basic documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
15 |
46709
3d740058b467
sidedata: move to new sidedata storage in revlogv2
Raphaël Gomès <rgomes@octobus.net>
parents:
46195
diff
changeset
|
16 Sidedata are stored in the revlog itself, thanks to a new version of the |
3d740058b467
sidedata: move to new sidedata storage in revlogv2
Raphaël Gomès <rgomes@octobus.net>
parents:
46195
diff
changeset
|
17 revlog. The following format is currently used:: |
43033
21025a4107d4
sidedata: add a new module with basic documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
18 |
21025a4107d4
sidedata: add a new module with basic documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
19 initial header: |
21025a4107d4
sidedata: add a new module with basic documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
20 <number of sidedata; 2 bytes> |
21025a4107d4
sidedata: add a new module with basic documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
21 sidedata (repeated N times): |
21025a4107d4
sidedata: add a new module with basic documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
22 <sidedata-key; 2 bytes> |
21025a4107d4
sidedata: add a new module with basic documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
23 <sidedata-entry-length: 4 bytes> |
21025a4107d4
sidedata: add a new module with basic documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
24 <sidedata-content-sha1-digest: 20 bytes> |
21025a4107d4
sidedata: add a new module with basic documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
25 <sidedata-content; X bytes> |
21025a4107d4
sidedata: add a new module with basic documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
26 normal raw text: |
21025a4107d4
sidedata: add a new module with basic documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
27 <all bytes remaining in the rawtext> |
21025a4107d4
sidedata: add a new module with basic documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
28 |
46195 | 29 This is a simple and effective format. It should be enough to experiment with |
43033
21025a4107d4
sidedata: add a new module with basic documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
30 the concept. |
21025a4107d4
sidedata: add a new module with basic documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
31 """ |
21025a4107d4
sidedata: add a new module with basic documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
32 |
21025a4107d4
sidedata: add a new module with basic documentation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
33 from __future__ import absolute_import |
43034
294afb982a88
sidedata: add a function to read sidedata from revlog raw text
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
43033
diff
changeset
|
34 |
47085
3aab2330b7d3
sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents:
46709
diff
changeset
|
35 import collections |
43034
294afb982a88
sidedata: add a function to read sidedata from revlog raw text
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
43033
diff
changeset
|
36 import struct |
294afb982a88
sidedata: add a function to read sidedata from revlog raw text
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
43033
diff
changeset
|
37 |
47085
3aab2330b7d3
sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents:
46709
diff
changeset
|
38 from .. import error, requirements as requirementsmod |
3aab2330b7d3
sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents:
46709
diff
changeset
|
39 from ..revlogutils import constants, flagutil |
44060
a61287a95dc3
core: migrate uses of hashlib.sha1 to hashutil.sha1
Augie Fackler <augie@google.com>
parents:
43506
diff
changeset
|
40 from ..utils import hashutil |
43034
294afb982a88
sidedata: add a function to read sidedata from revlog raw text
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
43033
diff
changeset
|
41 |
43040
ba4072c0a911
sidedata: test we can successfully write sidedata
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
43037
diff
changeset
|
42 ## sidedata type constant |
ba4072c0a911
sidedata: test we can successfully write sidedata
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
43037
diff
changeset
|
43 # reserve a block for testing purposes. |
ba4072c0a911
sidedata: test we can successfully write sidedata
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
43037
diff
changeset
|
44 SD_TEST1 = 1 |
ba4072c0a911
sidedata: test we can successfully write sidedata
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
43037
diff
changeset
|
45 SD_TEST2 = 2 |
ba4072c0a911
sidedata: test we can successfully write sidedata
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
43037
diff
changeset
|
46 SD_TEST3 = 3 |
ba4072c0a911
sidedata: test we can successfully write sidedata
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
43037
diff
changeset
|
47 SD_TEST4 = 4 |
ba4072c0a911
sidedata: test we can successfully write sidedata
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
43037
diff
changeset
|
48 SD_TEST5 = 5 |
ba4072c0a911
sidedata: test we can successfully write sidedata
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
43037
diff
changeset
|
49 SD_TEST6 = 6 |
ba4072c0a911
sidedata: test we can successfully write sidedata
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
43037
diff
changeset
|
50 SD_TEST7 = 7 |
ba4072c0a911
sidedata: test we can successfully write sidedata
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
43037
diff
changeset
|
51 |
43142
beed7ce61681
sidedatacopies: write copies information in sidedata when applicable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
43077
diff
changeset
|
52 # key to store copies related information |
beed7ce61681
sidedatacopies: write copies information in sidedata when applicable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
43077
diff
changeset
|
53 SD_P1COPIES = 8 |
beed7ce61681
sidedatacopies: write copies information in sidedata when applicable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
43077
diff
changeset
|
54 SD_P2COPIES = 9 |
beed7ce61681
sidedatacopies: write copies information in sidedata when applicable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
43077
diff
changeset
|
55 SD_FILESADDED = 10 |
beed7ce61681
sidedatacopies: write copies information in sidedata when applicable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
43077
diff
changeset
|
56 SD_FILESREMOVED = 11 |
45634
9a6b409b8ebc
changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
44060
diff
changeset
|
57 SD_FILES = 12 |
43142
beed7ce61681
sidedatacopies: write copies information in sidedata when applicable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
43077
diff
changeset
|
58 |
43040
ba4072c0a911
sidedata: test we can successfully write sidedata
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
43037
diff
changeset
|
59 # internal format constant |
43506
9f70512ae2cf
cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents:
43142
diff
changeset
|
60 SIDEDATA_HEADER = struct.Struct('>H') |
9f70512ae2cf
cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents:
43142
diff
changeset
|
61 SIDEDATA_ENTRY = struct.Struct('>HL20s') |
43034
294afb982a88
sidedata: add a function to read sidedata from revlog raw text
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
43033
diff
changeset
|
62 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43048
diff
changeset
|
63 |
46709
3d740058b467
sidedata: move to new sidedata storage in revlogv2
Raphaël Gomès <rgomes@octobus.net>
parents:
46195
diff
changeset
|
64 def serialize_sidedata(sidedata): |
43035
ea83abf95630
sidedata: add a function to write sidedata into a raw text
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
43034
diff
changeset
|
65 sidedata = list(sidedata.items()) |
ea83abf95630
sidedata: add a function to write sidedata into a raw text
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
43034
diff
changeset
|
66 sidedata.sort() |
46709
3d740058b467
sidedata: move to new sidedata storage in revlogv2
Raphaël Gomès <rgomes@octobus.net>
parents:
46195
diff
changeset
|
67 buf = [SIDEDATA_HEADER.pack(len(sidedata))] |
43035
ea83abf95630
sidedata: add a function to write sidedata into a raw text
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
43034
diff
changeset
|
68 for key, value in sidedata: |
44060
a61287a95dc3
core: migrate uses of hashlib.sha1 to hashutil.sha1
Augie Fackler <augie@google.com>
parents:
43506
diff
changeset
|
69 digest = hashutil.sha1(value).digest() |
46709
3d740058b467
sidedata: move to new sidedata storage in revlogv2
Raphaël Gomès <rgomes@octobus.net>
parents:
46195
diff
changeset
|
70 buf.append(SIDEDATA_ENTRY.pack(key, len(value), digest)) |
43035
ea83abf95630
sidedata: add a function to write sidedata into a raw text
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
43034
diff
changeset
|
71 for key, value in sidedata: |
46709
3d740058b467
sidedata: move to new sidedata storage in revlogv2
Raphaël Gomès <rgomes@octobus.net>
parents:
46195
diff
changeset
|
72 buf.append(value) |
3d740058b467
sidedata: move to new sidedata storage in revlogv2
Raphaël Gomès <rgomes@octobus.net>
parents:
46195
diff
changeset
|
73 buf = b''.join(buf) |
3d740058b467
sidedata: move to new sidedata storage in revlogv2
Raphaël Gomès <rgomes@octobus.net>
parents:
46195
diff
changeset
|
74 return buf |
43035
ea83abf95630
sidedata: add a function to write sidedata into a raw text
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
43034
diff
changeset
|
75 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43048
diff
changeset
|
76 |
46709
3d740058b467
sidedata: move to new sidedata storage in revlogv2
Raphaël Gomès <rgomes@octobus.net>
parents:
46195
diff
changeset
|
77 def deserialize_sidedata(blob): |
43034
294afb982a88
sidedata: add a function to read sidedata from revlog raw text
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
43033
diff
changeset
|
78 sidedata = {} |
294afb982a88
sidedata: add a function to read sidedata from revlog raw text
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
43033
diff
changeset
|
79 offset = 0 |
46709
3d740058b467
sidedata: move to new sidedata storage in revlogv2
Raphaël Gomès <rgomes@octobus.net>
parents:
46195
diff
changeset
|
80 (nbentry,) = SIDEDATA_HEADER.unpack(blob[: SIDEDATA_HEADER.size]) |
43034
294afb982a88
sidedata: add a function to read sidedata from revlog raw text
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
43033
diff
changeset
|
81 offset += SIDEDATA_HEADER.size |
294afb982a88
sidedata: add a function to read sidedata from revlog raw text
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
43033
diff
changeset
|
82 dataoffset = SIDEDATA_HEADER.size + (SIDEDATA_ENTRY.size * nbentry) |
294afb982a88
sidedata: add a function to read sidedata from revlog raw text
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
43033
diff
changeset
|
83 for i in range(nbentry): |
294afb982a88
sidedata: add a function to read sidedata from revlog raw text
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
43033
diff
changeset
|
84 nextoffset = offset + SIDEDATA_ENTRY.size |
46709
3d740058b467
sidedata: move to new sidedata storage in revlogv2
Raphaël Gomès <rgomes@octobus.net>
parents:
46195
diff
changeset
|
85 key, size, storeddigest = SIDEDATA_ENTRY.unpack(blob[offset:nextoffset]) |
43034
294afb982a88
sidedata: add a function to read sidedata from revlog raw text
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
43033
diff
changeset
|
86 offset = nextoffset |
294afb982a88
sidedata: add a function to read sidedata from revlog raw text
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
43033
diff
changeset
|
87 # read the data associated with that entry |
294afb982a88
sidedata: add a function to read sidedata from revlog raw text
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
43033
diff
changeset
|
88 nextdataoffset = dataoffset + size |
46709
3d740058b467
sidedata: move to new sidedata storage in revlogv2
Raphaël Gomès <rgomes@octobus.net>
parents:
46195
diff
changeset
|
89 entrytext = bytes(blob[dataoffset:nextdataoffset]) |
44060
a61287a95dc3
core: migrate uses of hashlib.sha1 to hashutil.sha1
Augie Fackler <augie@google.com>
parents:
43506
diff
changeset
|
90 readdigest = hashutil.sha1(entrytext).digest() |
43034
294afb982a88
sidedata: add a function to read sidedata from revlog raw text
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
43033
diff
changeset
|
91 if storeddigest != readdigest: |
294afb982a88
sidedata: add a function to read sidedata from revlog raw text
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
43033
diff
changeset
|
92 raise error.SidedataHashError(key, storeddigest, readdigest) |
294afb982a88
sidedata: add a function to read sidedata from revlog raw text
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
43033
diff
changeset
|
93 sidedata[key] = entrytext |
294afb982a88
sidedata: add a function to read sidedata from revlog raw text
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
43033
diff
changeset
|
94 dataoffset = nextdataoffset |
46709
3d740058b467
sidedata: move to new sidedata storage in revlogv2
Raphaël Gomès <rgomes@octobus.net>
parents:
46195
diff
changeset
|
95 return sidedata |
47085
3aab2330b7d3
sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents:
46709
diff
changeset
|
96 |
3aab2330b7d3
sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents:
46709
diff
changeset
|
97 |
3aab2330b7d3
sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents:
46709
diff
changeset
|
98 def get_sidedata_helpers(repo, remote_sd_categories, pull=False): |
47086
8bd769b5c941
sidedata: move documentation about sidedata helpers to sidedata module
Raphaël Gomès <rgomes@octobus.net>
parents:
47085
diff
changeset
|
99 """ |
8bd769b5c941
sidedata: move documentation about sidedata helpers to sidedata module
Raphaël Gomès <rgomes@octobus.net>
parents:
47085
diff
changeset
|
100 Returns a dictionary mapping revlog types to tuples of |
8bd769b5c941
sidedata: move documentation about sidedata helpers to sidedata module
Raphaël Gomès <rgomes@octobus.net>
parents:
47085
diff
changeset
|
101 `(repo, computers, removers)`: |
8bd769b5c941
sidedata: move documentation about sidedata helpers to sidedata module
Raphaël Gomès <rgomes@octobus.net>
parents:
47085
diff
changeset
|
102 * `repo` is used as an argument for computers |
8bd769b5c941
sidedata: move documentation about sidedata helpers to sidedata module
Raphaël Gomès <rgomes@octobus.net>
parents:
47085
diff
changeset
|
103 * `computers` is a list of `(category, (keys, computer, flags)` that |
8bd769b5c941
sidedata: move documentation about sidedata helpers to sidedata module
Raphaël Gomès <rgomes@octobus.net>
parents:
47085
diff
changeset
|
104 compute the missing sidedata categories that were asked: |
8bd769b5c941
sidedata: move documentation about sidedata helpers to sidedata module
Raphaël Gomès <rgomes@octobus.net>
parents:
47085
diff
changeset
|
105 * `category` is the sidedata category |
8bd769b5c941
sidedata: move documentation about sidedata helpers to sidedata module
Raphaël Gomès <rgomes@octobus.net>
parents:
47085
diff
changeset
|
106 * `keys` are the sidedata keys to be affected |
8bd769b5c941
sidedata: move documentation about sidedata helpers to sidedata module
Raphaël Gomès <rgomes@octobus.net>
parents:
47085
diff
changeset
|
107 * `flags` is a bitmask (an integer) of flags to remove when |
8bd769b5c941
sidedata: move documentation about sidedata helpers to sidedata module
Raphaël Gomès <rgomes@octobus.net>
parents:
47085
diff
changeset
|
108 removing the category. |
8bd769b5c941
sidedata: move documentation about sidedata helpers to sidedata module
Raphaël Gomès <rgomes@octobus.net>
parents:
47085
diff
changeset
|
109 * `computer` is the function `(repo, store, rev, sidedata)` that |
8bd769b5c941
sidedata: move documentation about sidedata helpers to sidedata module
Raphaël Gomès <rgomes@octobus.net>
parents:
47085
diff
changeset
|
110 returns a tuple of |
8bd769b5c941
sidedata: move documentation about sidedata helpers to sidedata module
Raphaël Gomès <rgomes@octobus.net>
parents:
47085
diff
changeset
|
111 `(new sidedata dict, (flags to add, flags to remove))`. |
8bd769b5c941
sidedata: move documentation about sidedata helpers to sidedata module
Raphaël Gomès <rgomes@octobus.net>
parents:
47085
diff
changeset
|
112 For example, it will return `({}, (0, 1 << 15))` to return no |
8bd769b5c941
sidedata: move documentation about sidedata helpers to sidedata module
Raphaël Gomès <rgomes@octobus.net>
parents:
47085
diff
changeset
|
113 sidedata, with no flags to add and one flag to remove. |
8bd769b5c941
sidedata: move documentation about sidedata helpers to sidedata module
Raphaël Gomès <rgomes@octobus.net>
parents:
47085
diff
changeset
|
114 * `removers` will remove the keys corresponding to the categories |
8bd769b5c941
sidedata: move documentation about sidedata helpers to sidedata module
Raphaël Gomès <rgomes@octobus.net>
parents:
47085
diff
changeset
|
115 that are present, but not needed. |
8bd769b5c941
sidedata: move documentation about sidedata helpers to sidedata module
Raphaël Gomès <rgomes@octobus.net>
parents:
47085
diff
changeset
|
116 If both `computers` and `removers` are empty, sidedata will simply not |
8bd769b5c941
sidedata: move documentation about sidedata helpers to sidedata module
Raphaël Gomès <rgomes@octobus.net>
parents:
47085
diff
changeset
|
117 be transformed. |
8bd769b5c941
sidedata: move documentation about sidedata helpers to sidedata module
Raphaël Gomès <rgomes@octobus.net>
parents:
47085
diff
changeset
|
118 """ |
47085
3aab2330b7d3
sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents:
46709
diff
changeset
|
119 # Computers for computing sidedata on-the-fly |
3aab2330b7d3
sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents:
46709
diff
changeset
|
120 sd_computers = collections.defaultdict(list) |
3aab2330b7d3
sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents:
46709
diff
changeset
|
121 # Computers for categories to remove from sidedata |
3aab2330b7d3
sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents:
46709
diff
changeset
|
122 sd_removers = collections.defaultdict(list) |
3aab2330b7d3
sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents:
46709
diff
changeset
|
123 to_generate = remote_sd_categories - repo._wanted_sidedata |
3aab2330b7d3
sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents:
46709
diff
changeset
|
124 to_remove = repo._wanted_sidedata - remote_sd_categories |
3aab2330b7d3
sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents:
46709
diff
changeset
|
125 if pull: |
3aab2330b7d3
sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents:
46709
diff
changeset
|
126 to_generate, to_remove = to_remove, to_generate |
3aab2330b7d3
sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents:
46709
diff
changeset
|
127 |
3aab2330b7d3
sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents:
46709
diff
changeset
|
128 for revlog_kind, computers in repo._sidedata_computers.items(): |
3aab2330b7d3
sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents:
46709
diff
changeset
|
129 for category, computer in computers.items(): |
3aab2330b7d3
sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents:
46709
diff
changeset
|
130 if category in to_generate: |
3aab2330b7d3
sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents:
46709
diff
changeset
|
131 sd_computers[revlog_kind].append(computer) |
3aab2330b7d3
sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents:
46709
diff
changeset
|
132 if category in to_remove: |
3aab2330b7d3
sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents:
46709
diff
changeset
|
133 sd_removers[revlog_kind].append(computer) |
3aab2330b7d3
sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents:
46709
diff
changeset
|
134 |
3aab2330b7d3
sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents:
46709
diff
changeset
|
135 sidedata_helpers = (repo, sd_computers, sd_removers) |
3aab2330b7d3
sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents:
46709
diff
changeset
|
136 return sidedata_helpers |
3aab2330b7d3
sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents:
46709
diff
changeset
|
137 |
3aab2330b7d3
sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents:
46709
diff
changeset
|
138 |
3aab2330b7d3
sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents:
46709
diff
changeset
|
139 def run_sidedata_helpers(store, sidedata_helpers, sidedata, rev): |
3aab2330b7d3
sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents:
46709
diff
changeset
|
140 """Returns the sidedata for the given revision after running through |
3aab2330b7d3
sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents:
46709
diff
changeset
|
141 the given helpers. |
3aab2330b7d3
sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents:
46709
diff
changeset
|
142 - `store`: the revlog this applies to (changelog, manifest, or filelog |
3aab2330b7d3
sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents:
46709
diff
changeset
|
143 instance) |
47086
8bd769b5c941
sidedata: move documentation about sidedata helpers to sidedata module
Raphaël Gomès <rgomes@octobus.net>
parents:
47085
diff
changeset
|
144 - `sidedata_helpers`: see `get_sidedata_helpers` |
47085
3aab2330b7d3
sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents:
46709
diff
changeset
|
145 - `sidedata`: previous sidedata at the given rev, if any |
3aab2330b7d3
sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents:
46709
diff
changeset
|
146 - `rev`: affected rev of `store` |
3aab2330b7d3
sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents:
46709
diff
changeset
|
147 """ |
3aab2330b7d3
sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents:
46709
diff
changeset
|
148 repo, sd_computers, sd_removers = sidedata_helpers |
3aab2330b7d3
sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents:
46709
diff
changeset
|
149 kind = store.revlog_kind |
3aab2330b7d3
sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents:
46709
diff
changeset
|
150 flags_to_add = 0 |
3aab2330b7d3
sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents:
46709
diff
changeset
|
151 flags_to_remove = 0 |
3aab2330b7d3
sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents:
46709
diff
changeset
|
152 for _keys, sd_computer, _flags in sd_computers.get(kind, []): |
3aab2330b7d3
sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents:
46709
diff
changeset
|
153 sidedata, flags = sd_computer(repo, store, rev, sidedata) |
3aab2330b7d3
sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents:
46709
diff
changeset
|
154 flags_to_add |= flags[0] |
3aab2330b7d3
sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents:
46709
diff
changeset
|
155 flags_to_remove |= flags[1] |
3aab2330b7d3
sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents:
46709
diff
changeset
|
156 for keys, _computer, flags in sd_removers.get(kind, []): |
3aab2330b7d3
sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents:
46709
diff
changeset
|
157 for key in keys: |
3aab2330b7d3
sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents:
46709
diff
changeset
|
158 sidedata.pop(key, None) |
3aab2330b7d3
sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents:
46709
diff
changeset
|
159 flags_to_remove |= flags |
3aab2330b7d3
sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents:
46709
diff
changeset
|
160 return sidedata, (flags_to_add, flags_to_remove) |
3aab2330b7d3
sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents:
46709
diff
changeset
|
161 |
3aab2330b7d3
sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents:
46709
diff
changeset
|
162 |
3aab2330b7d3
sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents:
46709
diff
changeset
|
163 def set_sidedata_spec_for_repo(repo): |
3aab2330b7d3
sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents:
46709
diff
changeset
|
164 # prevent cycle metadata -> revlogutils.sidedata -> metadata |
3aab2330b7d3
sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents:
46709
diff
changeset
|
165 from .. import metadata |
3aab2330b7d3
sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents:
46709
diff
changeset
|
166 |
3aab2330b7d3
sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents:
46709
diff
changeset
|
167 if requirementsmod.COPIESSDC_REQUIREMENT in repo.requirements: |
3aab2330b7d3
sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents:
46709
diff
changeset
|
168 repo.register_wanted_sidedata(SD_FILES) |
3aab2330b7d3
sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents:
46709
diff
changeset
|
169 repo.register_sidedata_computer( |
3aab2330b7d3
sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents:
46709
diff
changeset
|
170 constants.KIND_CHANGELOG, |
3aab2330b7d3
sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents:
46709
diff
changeset
|
171 SD_FILES, |
3aab2330b7d3
sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents:
46709
diff
changeset
|
172 (SD_FILES,), |
3aab2330b7d3
sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents:
46709
diff
changeset
|
173 metadata.copies_sidedata_computer, |
3aab2330b7d3
sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents:
46709
diff
changeset
|
174 flagutil.REVIDX_HASCOPIESINFO, |
3aab2330b7d3
sidedata: move sidedata-related utils to the dedicated module
Raphaël Gomès <rgomes@octobus.net>
parents:
46709
diff
changeset
|
175 ) |