Mercurial > hg
annotate hgext/clonebundles.py @ 26645:2faa7671a4b3
clonebundles: filter on SNI requirement
Server Name Indication (SNI) is commonly used in CDNs and other hosted
environments. Unfortunately, Python <2.7.9 does not support SNI and when
these older Python versions attempt to negotiate TLS to an SNI server,
they raise an opaque error like
"_ssl.c:507: error:14094410:SSL routines:SSL3_READ_BYTES:sslv3 alert
handshake failure."
We introduce a manifest attribute to denote the URL requires SNI and
have clients without SNI support filter these entries.
author | Gregory Szorc <gregory.szorc@gmail.com> |
---|---|
date | Tue, 13 Oct 2015 10:59:41 -0700 |
parents | 74de1c59f71c |
children | 23c0da28c034 |
rev | line source |
---|---|
26623
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
1 # This software may be used and distributed according to the terms of the |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
2 # GNU General Public License version 2 or any later version. |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
3 |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
4 """server side extension to advertise pre-generated bundles to seed clones. |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
5 |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
6 The extension essentially serves the content of a .hg/clonebundles.manifest |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
7 file to clients that request it. |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
8 |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
9 The clonebundles.manifest file contains a list of URLs and attributes. URLs |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
10 hold pre-generated bundles that a client fetches and applies. After applying |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
11 the pre-generated bundle, the client will connect back to the original server |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
12 and pull data not in the pre-generated bundle. |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
13 |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
14 Manifest File Format: |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
15 |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
16 The manifest file contains a newline (\n) delimited list of entries. |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
17 |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
18 Each line in this file defines an available bundle. Lines have the format: |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
19 |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
20 <URL> [<key>=<value] |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
21 |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
22 That is, a URL followed by extra metadata describing it. Metadata keys and |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
23 values should be URL encoded. |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
24 |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
25 This metadata is optional. It is up to server operators to populate this |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
26 metadata. |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
27 |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
28 Keys in UPPERCASE are reserved for use by Mercurial. All non-uppercase keys |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
29 can be used by site installations. |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
30 |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
31 The server operator is responsible for generating the bundle manifest file. |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
32 |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
33 Metadata Attributes: |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
34 |
26644
74de1c59f71c
clonebundles: filter on bundle specification
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26623
diff
changeset
|
35 BUNDLESPEC |
74de1c59f71c
clonebundles: filter on bundle specification
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26623
diff
changeset
|
36 A "bundle specification" string that describes the type of the bundle. |
74de1c59f71c
clonebundles: filter on bundle specification
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26623
diff
changeset
|
37 |
74de1c59f71c
clonebundles: filter on bundle specification
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26623
diff
changeset
|
38 These are string values that are accepted by the "--type" argument of |
74de1c59f71c
clonebundles: filter on bundle specification
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26623
diff
changeset
|
39 `hg bundle`. |
74de1c59f71c
clonebundles: filter on bundle specification
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26623
diff
changeset
|
40 |
74de1c59f71c
clonebundles: filter on bundle specification
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26623
diff
changeset
|
41 The values are parsed in strict mode, which means they must be of the |
74de1c59f71c
clonebundles: filter on bundle specification
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26623
diff
changeset
|
42 "<compression>-<type>" form. See |
74de1c59f71c
clonebundles: filter on bundle specification
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26623
diff
changeset
|
43 mercurial.exchange.parsebundlespec() for more details. |
74de1c59f71c
clonebundles: filter on bundle specification
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26623
diff
changeset
|
44 |
74de1c59f71c
clonebundles: filter on bundle specification
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26623
diff
changeset
|
45 Clients will automatically filter out specifications that are unknown or |
74de1c59f71c
clonebundles: filter on bundle specification
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26623
diff
changeset
|
46 unsupported so they won't attempt to download something that likely won't |
74de1c59f71c
clonebundles: filter on bundle specification
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26623
diff
changeset
|
47 apply. |
74de1c59f71c
clonebundles: filter on bundle specification
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26623
diff
changeset
|
48 |
74de1c59f71c
clonebundles: filter on bundle specification
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26623
diff
changeset
|
49 The actual value doesn't impact client behavior beyond filtering: |
74de1c59f71c
clonebundles: filter on bundle specification
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26623
diff
changeset
|
50 clients will still sniff the bundle type from the header of downloaded |
74de1c59f71c
clonebundles: filter on bundle specification
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26623
diff
changeset
|
51 files. |
26645
2faa7671a4b3
clonebundles: filter on SNI requirement
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26644
diff
changeset
|
52 |
2faa7671a4b3
clonebundles: filter on SNI requirement
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26644
diff
changeset
|
53 REQUIRESNI |
2faa7671a4b3
clonebundles: filter on SNI requirement
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26644
diff
changeset
|
54 Whether Server Name Indication (SNI) is required to connect to the URL. |
2faa7671a4b3
clonebundles: filter on SNI requirement
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26644
diff
changeset
|
55 SNI allows servers to use multiple certificates on the same IP. It is |
2faa7671a4b3
clonebundles: filter on SNI requirement
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26644
diff
changeset
|
56 somewhat common in CDNs and other hosting providers. Older Python |
2faa7671a4b3
clonebundles: filter on SNI requirement
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26644
diff
changeset
|
57 versions do not support SNI. Defining this attribute enables clients |
2faa7671a4b3
clonebundles: filter on SNI requirement
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26644
diff
changeset
|
58 with older Python versions to filter this entry. |
2faa7671a4b3
clonebundles: filter on SNI requirement
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26644
diff
changeset
|
59 |
2faa7671a4b3
clonebundles: filter on SNI requirement
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26644
diff
changeset
|
60 If this is defined, it is important to advertise a non-SNI fallback |
2faa7671a4b3
clonebundles: filter on SNI requirement
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26644
diff
changeset
|
61 URL or clients running old Python releases may not be able to clone |
2faa7671a4b3
clonebundles: filter on SNI requirement
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26644
diff
changeset
|
62 with the clonebundles facility. |
2faa7671a4b3
clonebundles: filter on SNI requirement
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26644
diff
changeset
|
63 |
2faa7671a4b3
clonebundles: filter on SNI requirement
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26644
diff
changeset
|
64 Value should be "true". |
26623
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
65 """ |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
66 |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
67 from mercurial import ( |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
68 extensions, |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
69 wireproto, |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
70 ) |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
71 |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
72 testedwith = 'internal' |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
73 |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
74 def capabilities(orig, repo, proto): |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
75 caps = orig(repo, proto) |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
76 |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
77 # Only advertise if a manifest exists. This does add some I/O to requests. |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
78 # But this should be cheaper than a wasted network round trip due to |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
79 # missing file. |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
80 if repo.opener.exists('clonebundles.manifest'): |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
81 caps.append('clonebundles') |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
82 |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
83 return caps |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
84 |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
85 @wireproto.wireprotocommand('clonebundles', '') |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
86 def bundles(repo, proto): |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
87 """Server command for returning info for available bundles to seed clones. |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
88 |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
89 Clients will parse this response and determine what bundle to fetch. |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
90 |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
91 Other extensions may wrap this command to filter or dynamically emit |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
92 data depending on the request. e.g. you could advertise URLs for |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
93 the closest data center given the client's IP address. |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
94 """ |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
95 return repo.opener.tryread('clonebundles.manifest') |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
96 |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
97 def extsetup(ui): |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
98 extensions.wrapfunction(wireproto, '_capabilities', capabilities) |