annotate mercurial/metadata.py @ 45548:25e365d5aa8f

rebase: add dryrun property to rebaseruntime I want to be able to check the property in `rebaseruntime._rebasenode()`. Passing the value via the runtime is a convenient way. Differential Revision: https://phab.mercurial-scm.org/D9074
author Martin von Zweigbergk <martinvonz@google.com>
date Fri, 18 Sep 2020 15:40:26 -0700
parents df87821081ee
children 64d18e9e8508
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
44940
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
1 # metadata.py -- code related to various metadata computation and access.
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
2 #
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
3 # Copyright 2019 Google, Inc <martinvonz@google.com>
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
4 # Copyright 2020 Pierre-Yves David <pierre-yves.david@octobus.net>
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
5 #
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
6 # This software may be used and distributed according to the terms of the
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
7 # GNU General Public License version 2 or any later version.
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
8 from __future__ import absolute_import, print_function
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
9
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
10 import multiprocessing
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
11
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
12 from . import (
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
13 error,
44941
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
14 node,
44940
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
15 pycompat,
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
16 util,
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
17 )
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
18
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
19 from .revlogutils import (
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
20 flagutil as sidedataflag,
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
21 sidedata as sidedatamod,
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
22 )
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
23
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
24
45323
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
25 class ChangingFiles(object):
45506
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
26 """A class recording the changes made to a file by a changeset
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
27
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
28 Actions performed on files are gathered into 3 sets:
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
29
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
30 - added: files actively added in the changeset.
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
31 - removed: files removed in the revision
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
32 - touched: files affected by the merge
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
33
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
34 and copies information is held by 2 mappings
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
35
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
36 - copied_from_p1: {"<new-name>": "<source-name-in-p1>"} mapping for copies
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
37 - copied_from_p2: {"<new-name>": "<source-name-in-p2>"} mapping for copies
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
38
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
39 See their inline help for details.
45323
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
40 """
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
41
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
42 def __init__(
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
43 self, touched=(), added=(), removed=(), p1_copies=(), p2_copies=(),
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
44 ):
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
45 self._added = set(added)
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
46 self._removed = set(removed)
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
47 self._touched = set(touched)
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
48 self._touched.update(self._added)
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
49 self._touched.update(self._removed)
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
50 self._p1_copies = dict(p1_copies)
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
51 self._p2_copies = dict(p2_copies)
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
52
45507
df87821081ee changing-files: implement equality checking
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45506
diff changeset
53 def __eq__(self, other):
df87821081ee changing-files: implement equality checking
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45506
diff changeset
54 return (
df87821081ee changing-files: implement equality checking
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45506
diff changeset
55 self.added == other.added
df87821081ee changing-files: implement equality checking
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45506
diff changeset
56 and self.removed == other.removed
df87821081ee changing-files: implement equality checking
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45506
diff changeset
57 and self.touched == other.touched
df87821081ee changing-files: implement equality checking
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45506
diff changeset
58 and self.copied_from_p1 == other.copied_from_p1
df87821081ee changing-files: implement equality checking
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45506
diff changeset
59 and self.copied_from_p2 == other.copied_from_p2
df87821081ee changing-files: implement equality checking
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45506
diff changeset
60 )
df87821081ee changing-files: implement equality checking
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45506
diff changeset
61
45323
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
62 @property
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
63 def added(self):
45506
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
64 """files actively added in the changeset
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
65
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
66 Any file present in that revision that was absent in all the changeset's
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
67 parents.
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
68
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
69 In case of merge, this means a file absent in one of the parents but
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
70 existing in the other will *not* be contained in this set. (They were
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
71 added by an ancestor)
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
72 """
45323
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
73 return frozenset(self._added)
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
74
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
75 def mark_added(self, filename):
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
76 self._added.add(filename)
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
77 self._touched.add(filename)
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
78
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
79 def update_added(self, filenames):
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
80 for f in filenames:
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
81 self.mark_added(f)
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
82
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
83 @property
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
84 def removed(self):
45506
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
85 """files actively removed by the changeset
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
86
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
87 In case of merge this will only contain the set of files removing "new"
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
88 content. For any file absent in the current changeset:
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
89
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
90 a) If the file exists in both parents, it is clearly "actively" removed
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
91 by this changeset.
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
92
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
93 b) If a file exists in only one parent and in none of the common
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
94 ancestors, then the file was newly added in one of the merged branches
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
95 and then got "actively" removed.
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
96
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
97 c) If a file exists in only one parent and at least one of the common
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
98 ancestors using the same filenode, then the file was unchanged on one
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
99 side and deleted on the other side. The merge "passively" propagated
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
100 that deletion, but didn't "actively" remove the file. In this case the
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
101 file is *not* included in the `removed` set.
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
102
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
103 d) If a file exists in only one parent and at least one of the common
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
104 ancestors using a different filenode, then the file was changed on one
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
105 side and removed on the other side. The merge process "actively"
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
106 decided to drop the new change and delete the file. Unlike in the
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
107 previous case, (c), the file included in the `removed` set.
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
108
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
109 Summary table for merge:
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
110
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
111 case | exists in parents | exists in gca || removed
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
112 (a) | both | * || yes
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
113 (b) | one | none || yes
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
114 (c) | one | same filenode || no
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
115 (d) | one | new filenode || yes
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
116 """
45323
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
117 return frozenset(self._removed)
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
118
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
119 def mark_removed(self, filename):
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
120 self._removed.add(filename)
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
121 self._touched.add(filename)
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
122
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
123 def update_removed(self, filenames):
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
124 for f in filenames:
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
125 self.mark_removed(f)
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
126
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
127 @property
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
128 def touched(self):
45506
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
129 """files either actively modified, added or removed"""
45323
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
130 return frozenset(self._touched)
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
131
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
132 def mark_touched(self, filename):
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
133 self._touched.add(filename)
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
134
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
135 def update_touched(self, filenames):
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
136 for f in filenames:
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
137 self.mark_touched(f)
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
138
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
139 @property
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
140 def copied_from_p1(self):
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
141 return self._p1_copies.copy()
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
142
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
143 def mark_copied_from_p1(self, source, dest):
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
144 self._p1_copies[dest] = source
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
145
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
146 def update_copies_from_p1(self, copies):
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
147 for dest, source in copies.items():
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
148 self.mark_copied_from_p1(source, dest)
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
149
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
150 @property
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
151 def copied_from_p2(self):
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
152 return self._p2_copies.copy()
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
153
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
154 def mark_copied_from_p2(self, source, dest):
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
155 self._p2_copies[dest] = source
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
156
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
157 def update_copies_from_p2(self, copies):
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
158 for dest, source in copies.items():
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
159 self.mark_copied_from_p2(source, dest)
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
160
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
161
44940
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
162 def computechangesetfilesadded(ctx):
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
163 """return the list of files added in a changeset
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
164 """
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
165 added = []
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
166 for f in ctx.files():
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
167 if not any(f in p for p in ctx.parents()):
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
168 added.append(f)
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
169 return added
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
170
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
171
44941
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
172 def get_removal_filter(ctx, x=None):
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
173 """return a function to detect files "wrongly" detected as `removed`
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
174
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
175 When a file is removed relative to p1 in a merge, this
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
176 function determines whether the absence is due to a
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
177 deletion from a parent, or whether the merge commit
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
178 itself deletes the file. We decide this by doing a
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
179 simplified three way merge of the manifest entry for
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
180 the file. There are two ways we decide the merge
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
181 itself didn't delete a file:
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
182 - neither parent (nor the merge) contain the file
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
183 - exactly one parent contains the file, and that
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
184 parent has the same filelog entry as the merge
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
185 ancestor (or all of them if there two). In other
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
186 words, that parent left the file unchanged while the
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
187 other one deleted it.
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
188 One way to think about this is that deleting a file is
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
189 similar to emptying it, so the list of changed files
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
190 should be similar either way. The computation
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
191 described above is not done directly in _filecommit
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
192 when creating the list of changed files, however
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
193 it does something very similar by comparing filelog
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
194 nodes.
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
195 """
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
196
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
197 if x is not None:
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
198 p1, p2, m1, m2 = x
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
199 else:
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
200 p1 = ctx.p1()
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
201 p2 = ctx.p2()
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
202 m1 = p1.manifest()
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
203 m2 = p2.manifest()
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
204
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
205 @util.cachefunc
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
206 def mas():
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
207 p1n = p1.node()
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
208 p2n = p2.node()
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
209 cahs = ctx.repo().changelog.commonancestorsheads(p1n, p2n)
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
210 if not cahs:
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
211 cahs = [node.nullrev]
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
212 return [ctx.repo()[r].manifest() for r in cahs]
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
213
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
214 def deletionfromparent(f):
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
215 if f in m1:
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
216 return f not in m2 and all(
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
217 f in ma and ma.find(f) == m1.find(f) for ma in mas()
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
218 )
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
219 elif f in m2:
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
220 return all(f in ma and ma.find(f) == m2.find(f) for ma in mas())
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
221 else:
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
222 return True
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
223
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
224 return deletionfromparent
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
225
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
226
44940
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
227 def computechangesetfilesremoved(ctx):
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
228 """return the list of files removed in a changeset
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
229 """
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
230 removed = []
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
231 for f in ctx.files():
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
232 if f not in ctx:
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
233 removed.append(f)
44942
25512a65cefd metadata: filter the `removed` set to only contains relevant data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44941
diff changeset
234 if removed:
25512a65cefd metadata: filter the `removed` set to only contains relevant data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44941
diff changeset
235 rf = get_removal_filter(ctx)
25512a65cefd metadata: filter the `removed` set to only contains relevant data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44941
diff changeset
236 removed = [r for r in removed if not rf(r)]
44940
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
237 return removed
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
238
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
239
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
240 def computechangesetcopies(ctx):
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
241 """return the copies data for a changeset
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
242
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
243 The copies data are returned as a pair of dictionnary (p1copies, p2copies).
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
244
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
245 Each dictionnary are in the form: `{newname: oldname}`
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
246 """
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
247 p1copies = {}
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
248 p2copies = {}
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
249 p1 = ctx.p1()
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
250 p2 = ctx.p2()
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
251 narrowmatch = ctx._repo.narrowmatch()
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
252 for dst in ctx.files():
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
253 if not narrowmatch(dst) or dst not in ctx:
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
254 continue
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
255 copied = ctx[dst].renamed()
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
256 if not copied:
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
257 continue
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
258 src, srcnode = copied
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
259 if src in p1 and p1[src].filenode() == srcnode:
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
260 p1copies[dst] = src
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
261 elif src in p2 and p2[src].filenode() == srcnode:
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
262 p2copies[dst] = src
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
263 return p1copies, p2copies
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
264
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
265
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
266 def encodecopies(files, copies):
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
267 items = []
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
268 for i, dst in enumerate(files):
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
269 if dst in copies:
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
270 items.append(b'%d\0%s' % (i, copies[dst]))
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
271 if len(items) != len(copies):
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
272 raise error.ProgrammingError(
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
273 b'some copy targets missing from file list'
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
274 )
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
275 return b"\n".join(items)
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
276
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
277
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
278 def decodecopies(files, data):
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
279 try:
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
280 copies = {}
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
281 if not data:
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
282 return copies
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
283 for l in data.split(b'\n'):
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
284 strindex, src = l.split(b'\0')
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
285 i = int(strindex)
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
286 dst = files[i]
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
287 copies[dst] = src
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
288 return copies
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
289 except (ValueError, IndexError):
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
290 # Perhaps someone had chosen the same key name (e.g. "p1copies") and
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
291 # used different syntax for the value.
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
292 return None
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
293
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
294
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
295 def encodefileindices(files, subset):
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
296 subset = set(subset)
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
297 indices = []
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
298 for i, f in enumerate(files):
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
299 if f in subset:
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
300 indices.append(b'%d' % i)
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
301 return b'\n'.join(indices)
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
302
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
303
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
304 def decodefileindices(files, data):
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
305 try:
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
306 subset = []
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
307 if not data:
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
308 return subset
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
309 for strindex in data.split(b'\n'):
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
310 i = int(strindex)
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
311 if i < 0 or i >= len(files):
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
312 return None
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
313 subset.append(files[i])
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
314 return subset
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
315 except (ValueError, IndexError):
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
316 # Perhaps someone had chosen the same key name (e.g. "added") and
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
317 # used different syntax for the value.
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
318 return None
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
319
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
320
45325
c6eea5804551 commitctx: extract sidedata encoding inside its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45323
diff changeset
321 def encode_copies_sidedata(files):
c6eea5804551 commitctx: extract sidedata encoding inside its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45323
diff changeset
322 sortedfiles = sorted(files.touched)
c6eea5804551 commitctx: extract sidedata encoding inside its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45323
diff changeset
323 sidedata = {}
c6eea5804551 commitctx: extract sidedata encoding inside its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45323
diff changeset
324 p1copies = files.copied_from_p1
c6eea5804551 commitctx: extract sidedata encoding inside its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45323
diff changeset
325 if p1copies:
c6eea5804551 commitctx: extract sidedata encoding inside its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45323
diff changeset
326 p1copies = encodecopies(sortedfiles, p1copies)
c6eea5804551 commitctx: extract sidedata encoding inside its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45323
diff changeset
327 sidedata[sidedatamod.SD_P1COPIES] = p1copies
c6eea5804551 commitctx: extract sidedata encoding inside its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45323
diff changeset
328 p2copies = files.copied_from_p2
c6eea5804551 commitctx: extract sidedata encoding inside its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45323
diff changeset
329 if p2copies:
c6eea5804551 commitctx: extract sidedata encoding inside its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45323
diff changeset
330 p2copies = encodecopies(sortedfiles, p2copies)
c6eea5804551 commitctx: extract sidedata encoding inside its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45323
diff changeset
331 sidedata[sidedatamod.SD_P2COPIES] = p2copies
c6eea5804551 commitctx: extract sidedata encoding inside its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45323
diff changeset
332 filesadded = files.added
c6eea5804551 commitctx: extract sidedata encoding inside its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45323
diff changeset
333 if filesadded:
c6eea5804551 commitctx: extract sidedata encoding inside its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45323
diff changeset
334 filesadded = encodefileindices(sortedfiles, filesadded)
c6eea5804551 commitctx: extract sidedata encoding inside its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45323
diff changeset
335 sidedata[sidedatamod.SD_FILESADDED] = filesadded
c6eea5804551 commitctx: extract sidedata encoding inside its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45323
diff changeset
336 filesremoved = files.removed
c6eea5804551 commitctx: extract sidedata encoding inside its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45323
diff changeset
337 if filesremoved:
c6eea5804551 commitctx: extract sidedata encoding inside its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45323
diff changeset
338 filesremoved = encodefileindices(sortedfiles, filesremoved)
c6eea5804551 commitctx: extract sidedata encoding inside its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45323
diff changeset
339 sidedata[sidedatamod.SD_FILESREMOVED] = filesremoved
c6eea5804551 commitctx: extract sidedata encoding inside its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45323
diff changeset
340 if not sidedata:
c6eea5804551 commitctx: extract sidedata encoding inside its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45323
diff changeset
341 sidedata = None
c6eea5804551 commitctx: extract sidedata encoding inside its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45323
diff changeset
342 return sidedata
c6eea5804551 commitctx: extract sidedata encoding inside its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45323
diff changeset
343
c6eea5804551 commitctx: extract sidedata encoding inside its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45323
diff changeset
344
44940
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
345 def _getsidedata(srcrepo, rev):
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
346 ctx = srcrepo[rev]
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
347 filescopies = computechangesetcopies(ctx)
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
348 filesadded = computechangesetfilesadded(ctx)
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
349 filesremoved = computechangesetfilesremoved(ctx)
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
350 sidedata = {}
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
351 if any([filescopies, filesadded, filesremoved]):
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
352 sortedfiles = sorted(ctx.files())
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
353 p1copies, p2copies = filescopies
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
354 p1copies = encodecopies(sortedfiles, p1copies)
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
355 p2copies = encodecopies(sortedfiles, p2copies)
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
356 filesadded = encodefileindices(sortedfiles, filesadded)
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
357 filesremoved = encodefileindices(sortedfiles, filesremoved)
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
358 if p1copies:
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
359 sidedata[sidedatamod.SD_P1COPIES] = p1copies
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
360 if p2copies:
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
361 sidedata[sidedatamod.SD_P2COPIES] = p2copies
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
362 if filesadded:
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
363 sidedata[sidedatamod.SD_FILESADDED] = filesadded
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
364 if filesremoved:
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
365 sidedata[sidedatamod.SD_FILESREMOVED] = filesremoved
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
366 return sidedata
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
367
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
368
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
369 def getsidedataadder(srcrepo, destrepo):
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
370 use_w = srcrepo.ui.configbool(b'experimental', b'worker.repository-upgrade')
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
371 if pycompat.iswindows or not use_w:
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
372 return _get_simple_sidedata_adder(srcrepo, destrepo)
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
373 else:
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
374 return _get_worker_sidedata_adder(srcrepo, destrepo)
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
375
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
376
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
377 def _sidedata_worker(srcrepo, revs_queue, sidedata_queue, tokens):
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
378 """The function used by worker precomputing sidedata
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
379
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
380 It read an input queue containing revision numbers
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
381 It write in an output queue containing (rev, <sidedata-map>)
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
382
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
383 The `None` input value is used as a stop signal.
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
384
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
385 The `tokens` semaphore is user to avoid having too many unprocessed
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
386 entries. The workers needs to acquire one token before fetching a task.
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
387 They will be released by the consumer of the produced data.
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
388 """
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
389 tokens.acquire()
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
390 rev = revs_queue.get()
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
391 while rev is not None:
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
392 data = _getsidedata(srcrepo, rev)
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
393 sidedata_queue.put((rev, data))
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
394 tokens.acquire()
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
395 rev = revs_queue.get()
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
396 # processing of `None` is completed, release the token.
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
397 tokens.release()
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
398
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
399
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
400 BUFF_PER_WORKER = 50
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
401
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
402
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
403 def _get_worker_sidedata_adder(srcrepo, destrepo):
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
404 """The parallel version of the sidedata computation
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
405
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
406 This code spawn a pool of worker that precompute a buffer of sidedata
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
407 before we actually need them"""
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
408 # avoid circular import copies -> scmutil -> worker -> copies
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
409 from . import worker
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
410
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
411 nbworkers = worker._numworkers(srcrepo.ui)
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
412
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
413 tokens = multiprocessing.BoundedSemaphore(nbworkers * BUFF_PER_WORKER)
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
414 revsq = multiprocessing.Queue()
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
415 sidedataq = multiprocessing.Queue()
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
416
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
417 assert srcrepo.filtername is None
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
418 # queue all tasks beforehand, revision numbers are small and it make
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
419 # synchronisation simpler
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
420 #
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
421 # Since the computation for each node can be quite expensive, the overhead
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
422 # of using a single queue is not revelant. In practice, most computation
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
423 # are fast but some are very expensive and dominate all the other smaller
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
424 # cost.
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
425 for r in srcrepo.changelog.revs():
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
426 revsq.put(r)
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
427 # queue the "no more tasks" markers
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
428 for i in range(nbworkers):
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
429 revsq.put(None)
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
430
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
431 allworkers = []
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
432 for i in range(nbworkers):
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
433 args = (srcrepo, revsq, sidedataq, tokens)
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
434 w = multiprocessing.Process(target=_sidedata_worker, args=args)
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
435 allworkers.append(w)
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
436 w.start()
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
437
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
438 # dictionnary to store results for revision higher than we one we are
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
439 # looking for. For example, if we need the sidedatamap for 42, and 43 is
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
440 # received, when shelve 43 for later use.
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
441 staging = {}
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
442
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
443 def sidedata_companion(revlog, rev):
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
444 sidedata = {}
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
445 if util.safehasattr(revlog, b'filteredrevs'): # this is a changelog
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
446 # Is the data previously shelved ?
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
447 sidedata = staging.pop(rev, None)
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
448 if sidedata is None:
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
449 # look at the queued result until we find the one we are lookig
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
450 # for (shelve the other ones)
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
451 r, sidedata = sidedataq.get()
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
452 while r != rev:
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
453 staging[r] = sidedata
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
454 r, sidedata = sidedataq.get()
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
455 tokens.release()
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
456 return False, (), sidedata
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
457
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
458 return sidedata_companion
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
459
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
460
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
461 def _get_simple_sidedata_adder(srcrepo, destrepo):
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
462 """The simple version of the sidedata computation
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
463
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
464 It just compute it in the same thread on request"""
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
465
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
466 def sidedatacompanion(revlog, rev):
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
467 sidedata = {}
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
468 if util.safehasattr(revlog, 'filteredrevs'): # this is a changelog
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
469 sidedata = _getsidedata(srcrepo, rev)
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
470 return False, (), sidedata
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
471
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
472 return sidedatacompanion
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
473
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
474
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
475 def getsidedataremover(srcrepo, destrepo):
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
476 def sidedatacompanion(revlog, rev):
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
477 f = ()
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
478 if util.safehasattr(revlog, 'filteredrevs'): # this is a changelog
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
479 if revlog.flags(rev) & sidedataflag.REVIDX_SIDEDATA:
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
480 f = (
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
481 sidedatamod.SD_P1COPIES,
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
482 sidedatamod.SD_P2COPIES,
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
483 sidedatamod.SD_FILESADDED,
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
484 sidedatamod.SD_FILESREMOVED,
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
485 )
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
486 return False, f, {}
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
487
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
488 return sidedatacompanion