Mercurial > hg
annotate mercurial/narrowspec.py @ 40626:87a872555e90
revlog: detect incomplete revlog reads
_readsegment() is supposed to return N bytes of revlog revision
data starting at a file offset. Surprisingly, its behavior before
this patch never verified that it actually read and returned N
bytes! Instead, it would perform the read(), then return whatever
data was available. And even more surprisingly, nothing in the
call chain appears to have been validating that it received all
the data it was expecting.
This behavior could lead to partial or incomplete revision chunks
being operated on. This could result in e.g. cached deltas being
applied against incomplete base revisions. The delta application
process would happily perform this operation. Only hash
verification would detect the corruption and save us.
This commit changes the behavior of raw revlog reading to validate
that we actually read() the number of bytes that were requested.
We will raise a more specific error faster, rather than possibly
have it go undetected or manifest later in the call stack, at
delta application or hash verification.
Differential Revision: https://phab.mercurial-scm.org/D5266
author | Gregory Szorc <gregory.szorc@gmail.com> |
---|---|
date | Tue, 13 Nov 2018 12:30:59 -0800 |
parents | ae20f52437e9 |
children | efd0f79246e3 |
rev | line source |
---|---|
36079
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
1 # narrowspec.py - methods for working with a narrow view of a repository |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
2 # |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
3 # Copyright 2017 Google, Inc. |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
4 # |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
5 # This software may be used and distributed according to the terms of the |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
6 # GNU General Public License version 2 or any later version. |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
7 |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
8 from __future__ import absolute_import |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
9 |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
10 import errno |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
11 |
36160
9fd8c2a3db5a
narrowspec: move module into core
Gregory Szorc <gregory.szorc@gmail.com>
parents:
36159
diff
changeset
|
12 from .i18n import _ |
9fd8c2a3db5a
narrowspec: move module into core
Gregory Szorc <gregory.szorc@gmail.com>
parents:
36159
diff
changeset
|
13 from . import ( |
36079
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
14 error, |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
15 match as matchmod, |
38869
ad24b581e4d9
narrow: call narrowspec.{save,restore,clear}backup directly
Martin von Zweigbergk <martinvonz@google.com>
parents:
38840
diff
changeset
|
16 repository, |
38839
f64ebe7d2259
narrowspec: use sparse.parseconfig() to parse narrowspec file (BC)
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
38836
diff
changeset
|
17 sparse, |
36079
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
18 util, |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
19 ) |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
20 |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
21 FILENAME = 'narrowspec' |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
22 |
39531
0d572769046a
narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39525
diff
changeset
|
23 # Pattern prefixes that are allowed in narrow patterns. This list MUST |
0d572769046a
narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39525
diff
changeset
|
24 # only contain patterns that are fast and safe to evaluate. Keep in mind |
0d572769046a
narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39525
diff
changeset
|
25 # that patterns are supplied by clients and executed on remote servers |
39811
ae20f52437e9
wireprotov2: advertise recognized path filter prefixes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39556
diff
changeset
|
26 # as part of wire protocol commands. That means that changes to this |
ae20f52437e9
wireprotov2: advertise recognized path filter prefixes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39556
diff
changeset
|
27 # data structure influence the wire protocol and should not be taken |
ae20f52437e9
wireprotov2: advertise recognized path filter prefixes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39556
diff
changeset
|
28 # lightly - especially removals. |
39531
0d572769046a
narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39525
diff
changeset
|
29 VALID_PREFIXES = ( |
0d572769046a
narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39525
diff
changeset
|
30 b'path:', |
0d572769046a
narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39525
diff
changeset
|
31 b'rootfilesin:', |
0d572769046a
narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39525
diff
changeset
|
32 ) |
0d572769046a
narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39525
diff
changeset
|
33 |
36079
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
34 def normalizesplitpattern(kind, pat): |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
35 """Returns the normalized version of a pattern and kind. |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
36 |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
37 Returns a tuple with the normalized kind and normalized pattern. |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
38 """ |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
39 pat = pat.rstrip('/') |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
40 _validatepattern(pat) |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
41 return kind, pat |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
42 |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
43 def _numlines(s): |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
44 """Returns the number of lines in s, including ending empty lines.""" |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
45 # We use splitlines because it is Unicode-friendly and thus Python 3 |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
46 # compatible. However, it does not count empty lines at the end, so trick |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
47 # it by adding a character at the end. |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
48 return len((s + 'x').splitlines()) |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
49 |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
50 def _validatepattern(pat): |
36098
9c55bbc29dcf
narrowspec: document constraints when validating patterns
Augie Fackler <augie@google.com>
parents:
36079
diff
changeset
|
51 """Validates the pattern and aborts if it is invalid. |
9c55bbc29dcf
narrowspec: document constraints when validating patterns
Augie Fackler <augie@google.com>
parents:
36079
diff
changeset
|
52 |
9c55bbc29dcf
narrowspec: document constraints when validating patterns
Augie Fackler <augie@google.com>
parents:
36079
diff
changeset
|
53 Patterns are stored in the narrowspec as newline-separated |
9c55bbc29dcf
narrowspec: document constraints when validating patterns
Augie Fackler <augie@google.com>
parents:
36079
diff
changeset
|
54 POSIX-style bytestring paths. There's no escaping. |
9c55bbc29dcf
narrowspec: document constraints when validating patterns
Augie Fackler <augie@google.com>
parents:
36079
diff
changeset
|
55 """ |
36079
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
56 |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
57 # We use newlines as separators in the narrowspec file, so don't allow them |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
58 # in patterns. |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
59 if _numlines(pat) > 1: |
36160
9fd8c2a3db5a
narrowspec: move module into core
Gregory Szorc <gregory.szorc@gmail.com>
parents:
36159
diff
changeset
|
60 raise error.Abort(_('newlines are not allowed in narrowspec paths')) |
36079
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
61 |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
62 components = pat.split('/') |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
63 if '.' in components or '..' in components: |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
64 raise error.Abort(_('"." and ".." are not allowed in narrowspec paths')) |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
65 |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
66 def normalizepattern(pattern, defaultkind='path'): |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
67 """Returns the normalized version of a text-format pattern. |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
68 |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
69 If the pattern has no kind, the default will be added. |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
70 """ |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
71 kind, pat = matchmod._patsplit(pattern, defaultkind) |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
72 return '%s:%s' % normalizesplitpattern(kind, pat) |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
73 |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
74 def parsepatterns(pats): |
39531
0d572769046a
narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39525
diff
changeset
|
75 """Parses an iterable of patterns into a typed pattern set. |
0d572769046a
narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39525
diff
changeset
|
76 |
0d572769046a
narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39525
diff
changeset
|
77 Patterns are assumed to be ``path:`` if no prefix is present. |
0d572769046a
narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39525
diff
changeset
|
78 For safety and performance reasons, only some prefixes are allowed. |
0d572769046a
narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39525
diff
changeset
|
79 See ``validatepatterns()``. |
0d572769046a
narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39525
diff
changeset
|
80 |
0d572769046a
narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39525
diff
changeset
|
81 This function should be used on patterns that come from the user to |
0d572769046a
narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39525
diff
changeset
|
82 normalize and validate them to the internal data structure used for |
0d572769046a
narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39525
diff
changeset
|
83 representing patterns. |
0d572769046a
narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39525
diff
changeset
|
84 """ |
0d572769046a
narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39525
diff
changeset
|
85 res = {normalizepattern(orig) for orig in pats} |
0d572769046a
narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39525
diff
changeset
|
86 validatepatterns(res) |
0d572769046a
narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39525
diff
changeset
|
87 return res |
0d572769046a
narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39525
diff
changeset
|
88 |
0d572769046a
narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39525
diff
changeset
|
89 def validatepatterns(pats): |
0d572769046a
narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39525
diff
changeset
|
90 """Validate that patterns are in the expected data structure and format. |
0d572769046a
narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39525
diff
changeset
|
91 |
0d572769046a
narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39525
diff
changeset
|
92 And that is a set of normalized patterns beginning with ``path:`` or |
0d572769046a
narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39525
diff
changeset
|
93 ``rootfilesin:``. |
0d572769046a
narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39525
diff
changeset
|
94 |
0d572769046a
narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39525
diff
changeset
|
95 This function should be used to validate internal data structures |
0d572769046a
narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39525
diff
changeset
|
96 and patterns that are loaded from sources that use the internal, |
0d572769046a
narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39525
diff
changeset
|
97 prefixed pattern representation (but can't necessarily be fully trusted). |
0d572769046a
narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39525
diff
changeset
|
98 """ |
0d572769046a
narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39525
diff
changeset
|
99 if not isinstance(pats, set): |
0d572769046a
narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39525
diff
changeset
|
100 raise error.ProgrammingError('narrow patterns should be a set; ' |
0d572769046a
narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39525
diff
changeset
|
101 'got %r' % pats) |
0d572769046a
narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39525
diff
changeset
|
102 |
0d572769046a
narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39525
diff
changeset
|
103 for pat in pats: |
0d572769046a
narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39525
diff
changeset
|
104 if not pat.startswith(VALID_PREFIXES): |
0d572769046a
narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39525
diff
changeset
|
105 # Use a Mercurial exception because this can happen due to user |
0d572769046a
narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39525
diff
changeset
|
106 # bugs (e.g. manually updating spec file). |
0d572769046a
narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39525
diff
changeset
|
107 raise error.Abort(_('invalid prefix on narrow pattern: %s') % pat, |
0d572769046a
narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39525
diff
changeset
|
108 hint=_('narrow patterns must begin with one of ' |
0d572769046a
narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39525
diff
changeset
|
109 'the following: %s') % |
0d572769046a
narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39525
diff
changeset
|
110 ', '.join(VALID_PREFIXES)) |
36079
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
111 |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
112 def format(includes, excludes): |
38839
f64ebe7d2259
narrowspec: use sparse.parseconfig() to parse narrowspec file (BC)
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
38836
diff
changeset
|
113 output = '[include]\n' |
36079
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
114 for i in sorted(includes - excludes): |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
115 output += i + '\n' |
38839
f64ebe7d2259
narrowspec: use sparse.parseconfig() to parse narrowspec file (BC)
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
38836
diff
changeset
|
116 output += '[exclude]\n' |
36079
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
117 for e in sorted(excludes): |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
118 output += e + '\n' |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
119 return output |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
120 |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
121 def match(root, include=None, exclude=None): |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
122 if not include: |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
123 # Passing empty include and empty exclude to matchmod.match() |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
124 # gives a matcher that matches everything, so explicitly use |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
125 # the nevermatcher. |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
126 return matchmod.never(root, '') |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
127 return matchmod.match(root, '', [], include=include or [], |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
128 exclude=exclude or []) |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
129 |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
130 def load(repo): |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
131 try: |
38872
576eef1ab43d
narrow: move .hg/narrowspec to .hg/store/narrowspec (BC)
Martin von Zweigbergk <martinvonz@google.com>
parents:
38871
diff
changeset
|
132 spec = repo.svfs.read(FILENAME) |
36079
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
133 except IOError as e: |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
134 # Treat "narrowspec does not exist" the same as "narrowspec file exists |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
135 # and is empty". |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
136 if e.errno == errno.ENOENT: |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
137 return set(), set() |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
138 raise |
38839
f64ebe7d2259
narrowspec: use sparse.parseconfig() to parse narrowspec file (BC)
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
38836
diff
changeset
|
139 # maybe we should care about the profiles returned too |
f64ebe7d2259
narrowspec: use sparse.parseconfig() to parse narrowspec file (BC)
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
38836
diff
changeset
|
140 includepats, excludepats, profiles = sparse.parseconfig(repo.ui, spec, |
f64ebe7d2259
narrowspec: use sparse.parseconfig() to parse narrowspec file (BC)
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
38836
diff
changeset
|
141 'narrow') |
f64ebe7d2259
narrowspec: use sparse.parseconfig() to parse narrowspec file (BC)
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
38836
diff
changeset
|
142 if profiles: |
f64ebe7d2259
narrowspec: use sparse.parseconfig() to parse narrowspec file (BC)
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
38836
diff
changeset
|
143 raise error.Abort(_("including other spec files using '%include' is not" |
39525
2675d561f5cb
narrowspec: fix a typoed 'supported'
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
38872
diff
changeset
|
144 " supported in narrowspec")) |
39539
8d8e61df8259
narrowspec: validate patterns when loading and saving spec file
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39531
diff
changeset
|
145 |
8d8e61df8259
narrowspec: validate patterns when loading and saving spec file
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39531
diff
changeset
|
146 validatepatterns(includepats) |
8d8e61df8259
narrowspec: validate patterns when loading and saving spec file
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39531
diff
changeset
|
147 validatepatterns(excludepats) |
8d8e61df8259
narrowspec: validate patterns when loading and saving spec file
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39531
diff
changeset
|
148 |
38839
f64ebe7d2259
narrowspec: use sparse.parseconfig() to parse narrowspec file (BC)
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
38836
diff
changeset
|
149 return includepats, excludepats |
36079
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
150 |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
151 def save(repo, includepats, excludepats): |
39539
8d8e61df8259
narrowspec: validate patterns when loading and saving spec file
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39531
diff
changeset
|
152 validatepatterns(includepats) |
8d8e61df8259
narrowspec: validate patterns when loading and saving spec file
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39531
diff
changeset
|
153 validatepatterns(excludepats) |
36079
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
154 spec = format(includepats, excludepats) |
38872
576eef1ab43d
narrow: move .hg/narrowspec to .hg/store/narrowspec (BC)
Martin von Zweigbergk <martinvonz@google.com>
parents:
38871
diff
changeset
|
155 repo.svfs.write(FILENAME, spec) |
36079
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
156 |
38869
ad24b581e4d9
narrow: call narrowspec.{save,restore,clear}backup directly
Martin von Zweigbergk <martinvonz@google.com>
parents:
38840
diff
changeset
|
157 def savebackup(repo, backupname): |
ad24b581e4d9
narrow: call narrowspec.{save,restore,clear}backup directly
Martin von Zweigbergk <martinvonz@google.com>
parents:
38840
diff
changeset
|
158 if repository.NARROW_REQUIREMENT not in repo.requirements: |
ad24b581e4d9
narrow: call narrowspec.{save,restore,clear}backup directly
Martin von Zweigbergk <martinvonz@google.com>
parents:
38840
diff
changeset
|
159 return |
ad24b581e4d9
narrow: call narrowspec.{save,restore,clear}backup directly
Martin von Zweigbergk <martinvonz@google.com>
parents:
38840
diff
changeset
|
160 vfs = repo.vfs |
38836
fed6fe856333
narrow: extract part of narrowspec backup to core
Martin von Zweigbergk <martinvonz@google.com>
parents:
36470
diff
changeset
|
161 vfs.tryunlink(backupname) |
38872
576eef1ab43d
narrow: move .hg/narrowspec to .hg/store/narrowspec (BC)
Martin von Zweigbergk <martinvonz@google.com>
parents:
38871
diff
changeset
|
162 util.copyfile(repo.svfs.join(FILENAME), vfs.join(backupname), hardlink=True) |
38836
fed6fe856333
narrow: extract part of narrowspec backup to core
Martin von Zweigbergk <martinvonz@google.com>
parents:
36470
diff
changeset
|
163 |
38869
ad24b581e4d9
narrow: call narrowspec.{save,restore,clear}backup directly
Martin von Zweigbergk <martinvonz@google.com>
parents:
38840
diff
changeset
|
164 def restorebackup(repo, backupname): |
ad24b581e4d9
narrow: call narrowspec.{save,restore,clear}backup directly
Martin von Zweigbergk <martinvonz@google.com>
parents:
38840
diff
changeset
|
165 if repository.NARROW_REQUIREMENT not in repo.requirements: |
ad24b581e4d9
narrow: call narrowspec.{save,restore,clear}backup directly
Martin von Zweigbergk <martinvonz@google.com>
parents:
38840
diff
changeset
|
166 return |
38872
576eef1ab43d
narrow: move .hg/narrowspec to .hg/store/narrowspec (BC)
Martin von Zweigbergk <martinvonz@google.com>
parents:
38871
diff
changeset
|
167 util.rename(repo.vfs.join(backupname), repo.svfs.join(FILENAME)) |
38836
fed6fe856333
narrow: extract part of narrowspec backup to core
Martin von Zweigbergk <martinvonz@google.com>
parents:
36470
diff
changeset
|
168 |
38869
ad24b581e4d9
narrow: call narrowspec.{save,restore,clear}backup directly
Martin von Zweigbergk <martinvonz@google.com>
parents:
38840
diff
changeset
|
169 def clearbackup(repo, backupname): |
ad24b581e4d9
narrow: call narrowspec.{save,restore,clear}backup directly
Martin von Zweigbergk <martinvonz@google.com>
parents:
38840
diff
changeset
|
170 if repository.NARROW_REQUIREMENT not in repo.requirements: |
ad24b581e4d9
narrow: call narrowspec.{save,restore,clear}backup directly
Martin von Zweigbergk <martinvonz@google.com>
parents:
38840
diff
changeset
|
171 return |
ad24b581e4d9
narrow: call narrowspec.{save,restore,clear}backup directly
Martin von Zweigbergk <martinvonz@google.com>
parents:
38840
diff
changeset
|
172 repo.vfs.unlink(backupname) |
38836
fed6fe856333
narrow: extract part of narrowspec backup to core
Martin von Zweigbergk <martinvonz@google.com>
parents:
36470
diff
changeset
|
173 |
36100
8fd0a9e2d7e9
narrow: make restrictpatterns a little more idiomatic
Augie Fackler <augie@google.com>
parents:
36099
diff
changeset
|
174 def restrictpatterns(req_includes, req_excludes, repo_includes, repo_excludes): |
36079
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
175 r""" Restricts the patterns according to repo settings, |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
176 results in a logical AND operation |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
177 |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
178 :param req_includes: requested includes |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
179 :param req_excludes: requested excludes |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
180 :param repo_includes: repo includes |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
181 :param repo_excludes: repo excludes |
36100
8fd0a9e2d7e9
narrow: make restrictpatterns a little more idiomatic
Augie Fackler <augie@google.com>
parents:
36099
diff
changeset
|
182 :return: include patterns, exclude patterns, and invalid include patterns. |
36079
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
183 |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
184 >>> restrictpatterns({'f1','f2'}, {}, ['f1'], []) |
36100
8fd0a9e2d7e9
narrow: make restrictpatterns a little more idiomatic
Augie Fackler <augie@google.com>
parents:
36099
diff
changeset
|
185 (set(['f1']), {}, []) |
36079
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
186 >>> restrictpatterns({'f1'}, {}, ['f1','f2'], []) |
36100
8fd0a9e2d7e9
narrow: make restrictpatterns a little more idiomatic
Augie Fackler <augie@google.com>
parents:
36099
diff
changeset
|
187 (set(['f1']), {}, []) |
36079
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
188 >>> restrictpatterns({'f1/fc1', 'f3/fc3'}, {}, ['f1','f2'], []) |
36100
8fd0a9e2d7e9
narrow: make restrictpatterns a little more idiomatic
Augie Fackler <augie@google.com>
parents:
36099
diff
changeset
|
189 (set(['f1/fc1']), {}, []) |
36079
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
190 >>> restrictpatterns({'f1_fc1'}, {}, ['f1','f2'], []) |
36100
8fd0a9e2d7e9
narrow: make restrictpatterns a little more idiomatic
Augie Fackler <augie@google.com>
parents:
36099
diff
changeset
|
191 ([], set(['path:.']), []) |
36079
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
192 >>> restrictpatterns({'f1/../f2/fc2'}, {}, ['f1','f2'], []) |
36100
8fd0a9e2d7e9
narrow: make restrictpatterns a little more idiomatic
Augie Fackler <augie@google.com>
parents:
36099
diff
changeset
|
193 (set(['f2/fc2']), {}, []) |
36079
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
194 >>> restrictpatterns({'f1/../f3/fc3'}, {}, ['f1','f2'], []) |
36100
8fd0a9e2d7e9
narrow: make restrictpatterns a little more idiomatic
Augie Fackler <augie@google.com>
parents:
36099
diff
changeset
|
195 ([], set(['path:.']), []) |
36079
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
196 >>> restrictpatterns({'f1/$non_exitent_var'}, {}, ['f1','f2'], []) |
36100
8fd0a9e2d7e9
narrow: make restrictpatterns a little more idiomatic
Augie Fackler <augie@google.com>
parents:
36099
diff
changeset
|
197 (set(['f1/$non_exitent_var']), {}, []) |
36079
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
198 """ |
36099
b8bbe589fd47
narrowspec: consistently use set() to copy sets
Augie Fackler <augie@google.com>
parents:
36098
diff
changeset
|
199 res_excludes = set(req_excludes) |
36079
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
200 res_excludes.update(repo_excludes) |
36100
8fd0a9e2d7e9
narrow: make restrictpatterns a little more idiomatic
Augie Fackler <augie@google.com>
parents:
36099
diff
changeset
|
201 invalid_includes = [] |
36079
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
202 if not req_includes: |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
203 res_includes = set(repo_includes) |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
204 elif 'path:.' not in repo_includes: |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
205 res_includes = [] |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
206 for req_include in req_includes: |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
207 req_include = util.expandpath(util.normpath(req_include)) |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
208 if req_include in repo_includes: |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
209 res_includes.append(req_include) |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
210 continue |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
211 valid = False |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
212 for repo_include in repo_includes: |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
213 if req_include.startswith(repo_include + '/'): |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
214 valid = True |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
215 res_includes.append(req_include) |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
216 break |
36100
8fd0a9e2d7e9
narrow: make restrictpatterns a little more idiomatic
Augie Fackler <augie@google.com>
parents:
36099
diff
changeset
|
217 if not valid: |
36079
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
218 invalid_includes.append(req_include) |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
219 if len(res_includes) == 0: |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
220 res_excludes = {'path:.'} |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
221 else: |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
222 res_includes = set(res_includes) |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
223 else: |
a2a6e724d61a
narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff
changeset
|
224 res_includes = set(req_includes) |
36100
8fd0a9e2d7e9
narrow: make restrictpatterns a little more idiomatic
Augie Fackler <augie@google.com>
parents:
36099
diff
changeset
|
225 return res_includes, res_excludes, invalid_includes |