Mercurial > hg
annotate hgext/schemes.py @ 47799:3fffb48539ee stable
rust-nodemap: falling back to C impl as mitigation
This is a mitigation for https://bz.mercurial-scm.org/show_bug.cgi?id=6554
We see sometimes almost all data except the most recent revisions
removed from the persistent nodemap, but we don't know how to
reproduce yet.
This has sadly repercussions beyond just needing to reconstruct the
persistent nodemap: for instance, this automatically filters out
all bookmarks pointing to revisions that the nodemap cannot resolve.
If such filtering happens in a transaction, the update of the
bookmarks file that happens at the end of transaction loses all
bookmarks that have been affected. There may be similar consequences
for other data.
So this is a data loss, something that we have to prevent as soon as
possible.
As a mitigation measure, we will now fallback to the C implementation
in case nodemap lookups failed. This will add some latency, e.g., in
discovery, yet less than disabling the persistent nodemap entirely.
We considered implementing the fallback directly on the Python
side, but `revlog.get_rev()` is not systematically used, there are
also several direct calls to the index method (`self.index.rev()` for
a `revlog` instance). It is therefore more direct to implement the
mitigation in the rust-cpython wrapper.
Differential Revision: https://phab.mercurial-scm.org/D11238
author | Georges Racinet <georges.racinet@octobus.net> |
---|---|
date | Sun, 01 Aug 2021 14:39:38 +0200 |
parents | ffd3e823a7e5 |
children | 6000f5b25c9b |
rev | line source |
---|---|
9964 | 1 # Copyright 2009, Alexander Solovyov <piranha@piranha.org.ua> |
2 # | |
3 # This software may be used and distributed according to the terms of the | |
10263 | 4 # GNU General Public License version 2 or any later version. |
9964 | 5 |
6 """extend schemes with shortcuts to repository swarms | |
7 | |
8 This extension allows you to specify shortcuts for parent URLs with a | |
9 lot of repositories to act like a scheme, for example:: | |
10 | |
11 [schemes] | |
12 py = http://code.python.org/hg/ | |
13 | |
14 After that you can use it like:: | |
15 | |
16 hg clone py://trunk/ | |
17 | |
18 Additionally there is support for some more complex schemas, for | |
19 example used by Google Code:: | |
20 | |
21 [schemes] | |
22 gcode = http://{1}.googlecode.com/hg/ | |
23 | |
24 The syntax is taken from Mercurial templates, and you have unlimited | |
25 number of variables, starting with ``{1}`` and continuing with | |
26 ``{2}``, ``{3}`` and so on. This variables will receive parts of URL | |
27 supplied, split by ``/``. Anything not specified as ``{part}`` will be | |
28 just appended to an URL. | |
29 | |
30 For convenience, the extension adds these schemes by default:: | |
31 | |
32 [schemes] | |
33 py = http://hg.python.org/ | |
34 bb = https://bitbucket.org/ | |
35 bb+ssh = ssh://hg@bitbucket.org/ | |
36 gcode = https://{1}.googlecode.com/hg/ | |
10777
bdc3256a318e
schemes: add Kiln On Demand to default schemes
Benjamin Pollack <benjamin@bitquabit.com>
parents:
10282
diff
changeset
|
37 kiln = https://{1}.kilnhg.com/Repo/ |
9964 | 38 |
9965
963ed04a8fde
schemes: fixed typos in module docstring
Martin Geisler <mg@lazybytes.net>
parents:
9964
diff
changeset
|
39 You can override a predefined scheme by defining a new scheme with the |
963ed04a8fde
schemes: fixed typos in module docstring
Martin Geisler <mg@lazybytes.net>
parents:
9964
diff
changeset
|
40 same name. |
9964 | 41 """ |
28379
27a9032374a7
schemas: use absolute_import
timeless <timeless@mozdev.org>
parents:
27982
diff
changeset
|
42 from __future__ import absolute_import |
9964 | 43 |
28379
27a9032374a7
schemas: use absolute_import
timeless <timeless@mozdev.org>
parents:
27982
diff
changeset
|
44 import os |
27a9032374a7
schemas: use absolute_import
timeless <timeless@mozdev.org>
parents:
27982
diff
changeset
|
45 import re |
29205
a0939666b836
py3: move up symbol imports to enforce import-checker rules
Yuya Nishihara <yuya@tcha.org>
parents:
28379
diff
changeset
|
46 |
a0939666b836
py3: move up symbol imports to enforce import-checker rules
Yuya Nishihara <yuya@tcha.org>
parents:
28379
diff
changeset
|
47 from mercurial.i18n import _ |
28379
27a9032374a7
schemas: use absolute_import
timeless <timeless@mozdev.org>
parents:
27982
diff
changeset
|
48 from mercurial import ( |
27a9032374a7
schemas: use absolute_import
timeless <timeless@mozdev.org>
parents:
27982
diff
changeset
|
49 error, |
27a9032374a7
schemas: use absolute_import
timeless <timeless@mozdev.org>
parents:
27982
diff
changeset
|
50 extensions, |
27a9032374a7
schemas: use absolute_import
timeless <timeless@mozdev.org>
parents:
27982
diff
changeset
|
51 hg, |
30640
7a3e67bfa417
py3: replace os.name with pycompat.osname (part 2 of 2)
Pulkit Goyal <7895pulkit@gmail.com>
parents:
29841
diff
changeset
|
52 pycompat, |
32337
46ba2cdda476
registrar: move cmdutil.command to registrar module (API)
Yuya Nishihara <yuya@tcha.org>
parents:
31288
diff
changeset
|
53 registrar, |
28379
27a9032374a7
schemas: use absolute_import
timeless <timeless@mozdev.org>
parents:
27982
diff
changeset
|
54 templater, |
46907
ffd3e823a7e5
urlutil: extract `url` related code from `util` into the new module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
45942
diff
changeset
|
55 ) |
ffd3e823a7e5
urlutil: extract `url` related code from `util` into the new module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
45942
diff
changeset
|
56 from mercurial.utils import ( |
ffd3e823a7e5
urlutil: extract `url` related code from `util` into the new module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
45942
diff
changeset
|
57 urlutil, |
28379
27a9032374a7
schemas: use absolute_import
timeless <timeless@mozdev.org>
parents:
27982
diff
changeset
|
58 ) |
9964 | 59 |
27982
bf1d5c223ac0
schemes: add debugexpandscheme command, resolving a scheme to canonical form
Jason R. Coombs <jaraco@jaraco.com>
parents:
27981
diff
changeset
|
60 cmdtable = {} |
32337
46ba2cdda476
registrar: move cmdutil.command to registrar module (API)
Yuya Nishihara <yuya@tcha.org>
parents:
31288
diff
changeset
|
61 command = registrar.command(cmdtable) |
29841
d5883fd055c6
extensions: change magic "shipped with hg" string
Augie Fackler <augie@google.com>
parents:
29205
diff
changeset
|
62 # Note for extension authors: ONLY specify testedwith = 'ships-with-hg-core' for |
25186
80c5b2666a96
extensions: document that `testedwith = 'internal'` is special
Augie Fackler <augie@google.com>
parents:
18910
diff
changeset
|
63 # extensions which SHIP WITH MERCURIAL. Non-mainline extensions should |
80c5b2666a96
extensions: document that `testedwith = 'internal'` is special
Augie Fackler <augie@google.com>
parents:
18910
diff
changeset
|
64 # be specifying the version(s) of Mercurial they are tested with, or |
80c5b2666a96
extensions: document that `testedwith = 'internal'` is special
Augie Fackler <augie@google.com>
parents:
18910
diff
changeset
|
65 # leave the attribute unspecified. |
43077
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
66 testedwith = b'ships-with-hg-core' |
16743
38caf405d010
hgext: mark all first-party extensions as such
Augie Fackler <raf@durin42.com>
parents:
15609
diff
changeset
|
67 |
44022
c1ccefb513e4
cleanup: drop redundant character escapes outside of `[]`
Matt Harbison <matt_harbison@yahoo.com>
parents:
43077
diff
changeset
|
68 _partre = re.compile(br'{(\d+)\}') |
9964 | 69 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
39549
diff
changeset
|
70 |
9964 | 71 class ShortRepository(object): |
72 def __init__(self, url, scheme, templater): | |
73 self.scheme = scheme | |
74 self.templater = templater | |
75 self.url = url | |
76 try: | |
31181
150cd5125722
schemes: move re construction to module-level and python3-ify
Augie Fackler <raf@durin42.com>
parents:
30640
diff
changeset
|
77 self.parts = max(map(int, _partre.findall(self.url))) |
9964 | 78 except ValueError: |
79 self.parts = 0 | |
80 | |
81 def __repr__(self): | |
43077
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
82 return b'<ShortRepository: %s>' % self.scheme |
9964 | 83 |
39549
089fc0db0954
hg: allow extra arguments to be passed to repo creation (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38354
diff
changeset
|
84 def instance(self, ui, url, create, intents=None, createopts=None): |
27981
d630eac3a5db
schemes: extract scheme expansion as its own method on ShortRepository
Jason R. Coombs <jaraco@jaraco.com>
parents:
26587
diff
changeset
|
85 url = self.resolve(url) |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
39549
diff
changeset
|
86 return hg._peerlookup(url).instance( |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
39549
diff
changeset
|
87 ui, url, create, intents=intents, createopts=createopts |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
39549
diff
changeset
|
88 ) |
27981
d630eac3a5db
schemes: extract scheme expansion as its own method on ShortRepository
Jason R. Coombs <jaraco@jaraco.com>
parents:
26587
diff
changeset
|
89 |
d630eac3a5db
schemes: extract scheme expansion as its own method on ShortRepository
Jason R. Coombs <jaraco@jaraco.com>
parents:
26587
diff
changeset
|
90 def resolve(self, url): |
46907
ffd3e823a7e5
urlutil: extract `url` related code from `util` into the new module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
45942
diff
changeset
|
91 # Should this use the urlutil.url class, or is manual parsing better? |
18910
b52404a914a9
scheme: don't crash on invalid URLs
Mads Kiilerich <madski@unity3d.com>
parents:
17425
diff
changeset
|
92 try: |
43077
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
93 url = url.split(b'://', 1)[1] |
18910
b52404a914a9
scheme: don't crash on invalid URLs
Mads Kiilerich <madski@unity3d.com>
parents:
17425
diff
changeset
|
94 except IndexError: |
43077
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
95 raise error.Abort(_(b"no '://' in scheme url '%s'") % url) |
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
96 parts = url.split(b'/', self.parts) |
9964 | 97 if len(parts) > self.parts: |
98 tail = parts[-1] | |
99 parts = parts[:-1] | |
100 else: | |
43077
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
101 tail = b'' |
44452
9d2b2df2c2ba
cleanup: run pyupgrade on our source tree to clean up varying things
Augie Fackler <augie@google.com>
parents:
44022
diff
changeset
|
102 context = {b'%d' % (i + 1): v for i, v in enumerate(parts)} |
43077
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
103 return b''.join(self.templater.process(self.url, context)) + tail |
9964 | 104 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
39549
diff
changeset
|
105 |
13827
f1823b9f073b
url: nuke some newly-introduced underbars in identifiers
Matt Mackall <mpm@selenic.com>
parents:
13822
diff
changeset
|
106 def hasdriveletter(orig, path): |
15609
8f4bad72d8b1
util: fix url.__str__() for windows file URLs
Patrick Mezard <pmezard@gmail.com>
parents:
14606
diff
changeset
|
107 if path: |
8f4bad72d8b1
util: fix url.__str__() for windows file URLs
Patrick Mezard <pmezard@gmail.com>
parents:
14606
diff
changeset
|
108 for scheme in schemes: |
43077
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
109 if path.startswith(scheme + b':'): |
15609
8f4bad72d8b1
util: fix url.__str__() for windows file URLs
Patrick Mezard <pmezard@gmail.com>
parents:
14606
diff
changeset
|
110 return False |
13822
fbf32a6c903e
schemes: prevent one letter schemes from being interpreted as drive letters
Brodie Rao <brodie@bitheap.org>
parents:
10777
diff
changeset
|
111 return orig(path) |
fbf32a6c903e
schemes: prevent one letter schemes from being interpreted as drive letters
Brodie Rao <brodie@bitheap.org>
parents:
10777
diff
changeset
|
112 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
39549
diff
changeset
|
113 |
9964 | 114 schemes = { |
43077
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
115 b'py': b'http://hg.python.org/', |
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
116 b'bb': b'https://bitbucket.org/', |
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
117 b'bb+ssh': b'ssh://hg@bitbucket.org/', |
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
118 b'gcode': b'https://{1}.googlecode.com/hg/', |
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
119 b'kiln': b'https://{1}.kilnhg.com/Repo/', |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
39549
diff
changeset
|
120 } |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
39549
diff
changeset
|
121 |
9964 | 122 |
123 def extsetup(ui): | |
43077
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
124 schemes.update(dict(ui.configitems(b'schemes'))) |
38354
e637dc0b3b1f
templater: parse template string to tree by templater class
Yuya Nishihara <yuya@tcha.org>
parents:
37717
diff
changeset
|
125 t = templater.engine(templater.parse) |
9964 | 126 for scheme, url in schemes.items(): |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
39549
diff
changeset
|
127 if ( |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
39549
diff
changeset
|
128 pycompat.iswindows |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
39549
diff
changeset
|
129 and len(scheme) == 1 |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
39549
diff
changeset
|
130 and scheme.isalpha() |
43077
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
131 and os.path.exists(b'%s:\\' % scheme) |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
39549
diff
changeset
|
132 ): |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
39549
diff
changeset
|
133 raise error.Abort( |
43077
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
134 _( |
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
135 b'custom scheme %s:// conflicts with drive ' |
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
136 b'letter %s:\\\n' |
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
137 ) |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
39549
diff
changeset
|
138 % (scheme, scheme.upper()) |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
39549
diff
changeset
|
139 ) |
14606
6e631c24c6d9
hg: move peerschemes back to schemes
Matt Mackall <mpm@selenic.com>
parents:
14605
diff
changeset
|
140 hg.schemes[scheme] = ShortRepository(url, scheme, t) |
13822
fbf32a6c903e
schemes: prevent one letter schemes from being interpreted as drive letters
Brodie Rao <brodie@bitheap.org>
parents:
10777
diff
changeset
|
141 |
46907
ffd3e823a7e5
urlutil: extract `url` related code from `util` into the new module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
45942
diff
changeset
|
142 extensions.wrapfunction(urlutil, b'hasdriveletter', hasdriveletter) |
27982
bf1d5c223ac0
schemes: add debugexpandscheme command, resolving a scheme to canonical form
Jason R. Coombs <jaraco@jaraco.com>
parents:
27981
diff
changeset
|
143 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
39549
diff
changeset
|
144 |
43077
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
145 @command(b'debugexpandscheme', norepo=True) |
27982
bf1d5c223ac0
schemes: add debugexpandscheme command, resolving a scheme to canonical form
Jason R. Coombs <jaraco@jaraco.com>
parents:
27981
diff
changeset
|
146 def expandscheme(ui, url, **opts): |
45942
89a2afe31e82
formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents:
44452
diff
changeset
|
147 """given a repo path, provide the scheme-expanded path""" |
27982
bf1d5c223ac0
schemes: add debugexpandscheme command, resolving a scheme to canonical form
Jason R. Coombs <jaraco@jaraco.com>
parents:
27981
diff
changeset
|
148 repo = hg._peerlookup(url) |
bf1d5c223ac0
schemes: add debugexpandscheme command, resolving a scheme to canonical form
Jason R. Coombs <jaraco@jaraco.com>
parents:
27981
diff
changeset
|
149 if isinstance(repo, ShortRepository): |
bf1d5c223ac0
schemes: add debugexpandscheme command, resolving a scheme to canonical form
Jason R. Coombs <jaraco@jaraco.com>
parents:
27981
diff
changeset
|
150 url = repo.resolve(url) |
43077
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
151 ui.write(url + b'\n') |