Mercurial > hg
annotate mercurial/httpconnection.py @ 26623:5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Cloning can be an expensive operation for servers because the server
generates a bundle from existing repository data at request time. For
a large repository like mozilla-central, this consumes 4+ minutes
of CPU time on the server. It also results in significant network
utilization. Multiplied by hundreds or even thousands of clients and
the ensuing load can result in difficulties scaling the Mercurial server.
Despite generation of bundles being deterministic until the next
changeset is added, the generation of bundles to service a clone request
is not cached. Each clone thus performs redundant work. This is
wasteful.
This patch introduces the "clonebundles" extension and related
client-side functionality to help alleviate this deficiency. The
client-side feature is behind an experimental flag and is not enabled by
default.
It works as follows:
1) Server operator generates a bundle and makes it available on a
server (likely HTTP).
2) Server operator defines the URL of a bundle file in a
.hg/clonebundles.manifest file.
3) Client `hg clone`ing sees the server is advertising bundle URLs.
4) Client fetches and applies the advertised bundle.
5) Client performs equivalent of `hg pull` to fetch changes made since
the bundle was created.
Essentially, the server performs the expensive work of generating a
bundle once and all subsequent clones fetch a static file from
somewhere. Scaling static file serving is a much more manageable
problem than scaling a Python application like Mercurial. Assuming your
repository grows less than 1% per day, the end result is 99+% of CPU
and network load from clones is eliminated, allowing Mercurial servers
to scale more easily. Serving static files also means data can be
transferred to clients as fast as they can consume it, rather than as
fast as servers can generate it. This makes clones faster.
Mozilla has implemented similar functionality of this patch on
hg.mozilla.org using a custom extension. We are hosting bundle files in
Amazon S3 and CloudFront (a CDN) and have successfully offloaded
>1 TB/day in data transfer from hg.mozilla.org, freeing up significant
bandwidth and CPU resources. The positive impact has been stellar and
I believe it has proved its value to be included in Mercurial core. I
feel it is important for the client-side support to be enabled in core
by default because it means that clients will get faster, more reliable
clones and will enable server operators to reduce load without
requiring any client-side configuration changes (assuming clients are
up to date, of course).
The scope of this feature is narrowly and specifically tailored to
cloning, despite "serve pulls from pre-generated bundles" being a valid
and useful feature. I would eventually like for Mercurial servers to
support transferring *all* repository data via statically hosted files.
You could imagine a server that siphons all pushed data to bundle files
and instructs clients to apply a stream of bundles to reconstruct all
repository data. This feature, while useful and powerful, is
significantly more work to implement because it requires the server
component have awareness of discovery and a mapping of which changesets
are in which files. Full, clone bundles, by contrast, are much simpler.
The wire protocol command is named "clonebundles" instead of something
more generic like "staticbundles" to leave the door open for a new, more
powerful and more generic server-side component with minimal backwards
compatibility implications. The name "bundleclone" is used by Mozilla's
extension and would cause problems since there are subtle differences
in Mozilla's extension.
Mozilla's experience with this idea has taught us that some form of
"content negotiation" is required. Not all clients will support all
bundle formats or even URLs (advanced TLS requirements, etc). To ensure
the highest uptake possible, a server needs to advertise multiple
versions of bundles and clients need to be able to choose the most
appropriate from that list one. The "attributes" in each
server-advertised entry facilitate this filtering and sorting. Their
use will become apparent in subsequent patches.
Initial inspiration and credit for the idea of cloning from static files
belongs to Augie Fackler and his "lookaside clone" extension proof of
concept.
author | Gregory Szorc <gregory.szorc@gmail.com> |
---|---|
date | Fri, 09 Oct 2015 11:22:01 -0700 |
parents | e9a35411bbbc |
children | b1adf32b0605 |
rev | line source |
---|---|
14244
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
1 # httpconnection.py - urllib2 handler for new http support |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
2 # |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
3 # Copyright 2005, 2006, 2007, 2008 Matt Mackall <mpm@selenic.com> |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
4 # Copyright 2006, 2007 Alexis S. L. Carvalho <alexis@cecm.usp.br> |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
5 # Copyright 2006 Vadim Gelfer <vadim.gelfer@gmail.com> |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
6 # Copyright 2011 Google, Inc. |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
7 # |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
8 # This software may be used and distributed according to the terms of the |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
9 # GNU General Public License version 2 or any later version. |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
10 import logging |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
11 import socket |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
12 import urllib |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
13 import urllib2 |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
14 import os |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
15 |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
16 from mercurial import httpclient |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
17 from mercurial import sslutil |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
18 from mercurial import util |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
19 from mercurial.i18n import _ |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
20 |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
21 # moved here from url.py to avoid a cycle |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
22 class httpsendfile(object): |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
23 """This is a wrapper around the objects returned by python's "open". |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
24 |
15152
94b200a11cf7
http: handle push of bundles > 2 GB again (issue3017)
Mads Kiilerich <mads@kiilerich.com>
parents:
15025
diff
changeset
|
25 Its purpose is to send file-like objects via HTTP. |
94b200a11cf7
http: handle push of bundles > 2 GB again (issue3017)
Mads Kiilerich <mads@kiilerich.com>
parents:
15025
diff
changeset
|
26 It do however not define a __len__ attribute because the length |
94b200a11cf7
http: handle push of bundles > 2 GB again (issue3017)
Mads Kiilerich <mads@kiilerich.com>
parents:
15025
diff
changeset
|
27 might be more than Py_ssize_t can handle. |
14244
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
28 """ |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
29 |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
30 def __init__(self, ui, *args, **kwargs): |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
31 # We can't just "self._data = open(*args, **kwargs)" here because there |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
32 # is an "open" function defined in this module that shadows the global |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
33 # one |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
34 self.ui = ui |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
35 self._data = open(*args, **kwargs) |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
36 self.seek = self._data.seek |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
37 self.close = self._data.close |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
38 self.write = self._data.write |
15152
94b200a11cf7
http: handle push of bundles > 2 GB again (issue3017)
Mads Kiilerich <mads@kiilerich.com>
parents:
15025
diff
changeset
|
39 self.length = os.fstat(self._data.fileno()).st_size |
14244
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
40 self._pos = 0 |
15791
a814f8fcc65a
Use explicit integer division
Martin Geisler <mg@aragost.com>
parents:
15288
diff
changeset
|
41 self._total = self.length // 1024 * 2 |
14244
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
42 |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
43 def read(self, *args, **kwargs): |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
44 try: |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
45 ret = self._data.read(*args, **kwargs) |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
46 except EOFError: |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
47 self.ui.progress(_('sending'), None) |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
48 self._pos += len(ret) |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
49 # We pass double the max for total because we currently have |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
50 # to send the bundle twice in the case of a server that |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
51 # requires authentication. Since we can't know until we try |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
52 # once whether authentication will be required, just lie to |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
53 # the user and maybe the push succeeds suddenly at 50%. |
15791
a814f8fcc65a
Use explicit integer division
Martin Geisler <mg@aragost.com>
parents:
15288
diff
changeset
|
54 self.ui.progress(_('sending'), self._pos // 1024, |
14244
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
55 unit=_('kb'), total=self._total) |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
56 return ret |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
57 |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
58 # moved here from url.py to avoid a cycle |
15025
0593e8f81c71
http: pass user to readauthforuri() (fix 4a43e23b8c55)
Patrick Mezard <pmezard@gmail.com>
parents:
15005
diff
changeset
|
59 def readauthforuri(ui, uri, user): |
14244
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
60 # Read configuration |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
61 config = dict() |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
62 for key, val in ui.configitems('auth'): |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
63 if '.' not in key: |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
64 ui.warn(_("ignoring invalid [auth] key '%s'\n") % key) |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
65 continue |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
66 group, setting = key.rsplit('.', 1) |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
67 gdict = config.setdefault(group, dict()) |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
68 if setting in ('username', 'cert', 'key'): |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
69 val = util.expandpath(val) |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
70 gdict[setting] = val |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
71 |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
72 # Find the best match |
25206
18a032704f0a
httpconnection: drop Python 2.4 specify hack
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
19809
diff
changeset
|
73 scheme, hostpath = uri.split('://', 1) |
15005
4a43e23b8c55
hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents:
14430
diff
changeset
|
74 bestuser = None |
14244
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
75 bestlen = 0 |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
76 bestauth = None |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
77 for group, auth in config.iteritems(): |
15005
4a43e23b8c55
hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents:
14430
diff
changeset
|
78 if user and user != auth.get('username', user): |
4a43e23b8c55
hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents:
14430
diff
changeset
|
79 # If a username was set in the URI, the entry username |
4a43e23b8c55
hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents:
14430
diff
changeset
|
80 # must either match it or be unset |
4a43e23b8c55
hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents:
14430
diff
changeset
|
81 continue |
14244
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
82 prefix = auth.get('prefix') |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
83 if not prefix: |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
84 continue |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
85 p = prefix.split('://', 1) |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
86 if len(p) > 1: |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
87 schemes, prefix = [p[0]], p[1] |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
88 else: |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
89 schemes = (auth.get('schemes') or 'https').split() |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
90 if (prefix == '*' or hostpath.startswith(prefix)) and \ |
15005
4a43e23b8c55
hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents:
14430
diff
changeset
|
91 (len(prefix) > bestlen or (len(prefix) == bestlen and \ |
4a43e23b8c55
hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents:
14430
diff
changeset
|
92 not bestuser and 'username' in auth)) \ |
4a43e23b8c55
hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents:
14430
diff
changeset
|
93 and scheme in schemes: |
14244
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
94 bestlen = len(prefix) |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
95 bestauth = group, auth |
15005
4a43e23b8c55
hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents:
14430
diff
changeset
|
96 bestuser = auth.get('username') |
4a43e23b8c55
hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents:
14430
diff
changeset
|
97 if user and not bestuser: |
4a43e23b8c55
hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents:
14430
diff
changeset
|
98 auth['username'] = user |
14244
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
99 return bestauth |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
100 |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
101 # Mercurial (at least until we can remove the old codepath) requires |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
102 # that the http response object be sufficiently file-like, so we |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
103 # provide a close() method here. |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
104 class HTTPResponse(httpclient.HTTPResponse): |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
105 def close(self): |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
106 pass |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
107 |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
108 class HTTPConnection(httpclient.HTTPConnection): |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
109 response_class = HTTPResponse |
26347
e9a35411bbbc
httpconnection: remove a mutable default argument
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25837
diff
changeset
|
110 def request(self, method, uri, body=None, headers=None): |
e9a35411bbbc
httpconnection: remove a mutable default argument
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25837
diff
changeset
|
111 if headers is None: |
e9a35411bbbc
httpconnection: remove a mutable default argument
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25837
diff
changeset
|
112 headers = {} |
14244
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
113 if isinstance(body, httpsendfile): |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
114 body.seek(0) |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
115 httpclient.HTTPConnection.request(self, method, uri, body=body, |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
116 headers=headers) |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
117 |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
118 |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
119 _configuredlogging = False |
14375
436e5379d7ba
httpconnection: improved logging formatting
Augie Fackler <durin42@gmail.com>
parents:
14346
diff
changeset
|
120 LOGFMT = '%(levelname)s:%(name)s:%(lineno)d:%(message)s' |
14244
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
121 # Subclass BOTH of these because otherwise urllib2 "helpfully" |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
122 # reinserts them since it notices we don't include any subclasses of |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
123 # them. |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
124 class http2handler(urllib2.HTTPHandler, urllib2.HTTPSHandler): |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
125 def __init__(self, ui, pwmgr): |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
126 global _configuredlogging |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
127 urllib2.AbstractHTTPHandler.__init__(self) |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
128 self.ui = ui |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
129 self.pwmgr = pwmgr |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
130 self._connections = {} |
25837
d343806dcf68
http2: mark experimental and developer options
Matt Mackall <mpm@selenic.com>
parents:
25660
diff
changeset
|
131 # developer config: ui.http2debuglevel |
14244
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
132 loglevel = ui.config('ui', 'http2debuglevel', default=None) |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
133 if loglevel and not _configuredlogging: |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
134 _configuredlogging = True |
14294
84256ba2fbf7
httpconnection: fix debug logging option for httpclient
Augie Fackler <durin42@gmail.com>
parents:
14244
diff
changeset
|
135 logger = logging.getLogger('mercurial.httpclient') |
14244
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
136 logger.setLevel(getattr(logging, loglevel.upper())) |
14375
436e5379d7ba
httpconnection: improved logging formatting
Augie Fackler <durin42@gmail.com>
parents:
14346
diff
changeset
|
137 handler = logging.StreamHandler() |
436e5379d7ba
httpconnection: improved logging formatting
Augie Fackler <durin42@gmail.com>
parents:
14346
diff
changeset
|
138 handler.setFormatter(logging.Formatter(LOGFMT)) |
436e5379d7ba
httpconnection: improved logging formatting
Augie Fackler <durin42@gmail.com>
parents:
14346
diff
changeset
|
139 logger.addHandler(handler) |
14244
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
140 |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
141 def close_all(self): |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
142 """Close and remove all connection objects being kept for reuse.""" |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
143 for openconns in self._connections.values(): |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
144 for conn in openconns: |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
145 conn.close() |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
146 self._connections = {} |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
147 |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
148 # shamelessly borrowed from urllib2.AbstractHTTPHandler |
14346
bf85c2639700
httpconnection: correctly handle redirects from http to https
Augie Fackler <durin42@gmail.com>
parents:
14294
diff
changeset
|
149 def do_open(self, http_class, req, use_ssl): |
14244
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
150 """Return an addinfourl object for the request, using http_class. |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
151 |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
152 http_class must implement the HTTPConnection API from httplib. |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
153 The addinfourl return value is a file-like object. It also |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
154 has methods and attributes including: |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
155 - info(): return a mimetools.Message object for the headers |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
156 - geturl(): return the original request URL |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
157 - code: HTTP status code |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
158 """ |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
159 # If using a proxy, the host returned by get_host() is |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
160 # actually the proxy. On Python 2.6.1, the real destination |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
161 # hostname is encoded in the URI in the urllib2 request |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
162 # object. On Python 2.6.5, it's stored in the _tunnel_host |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
163 # attribute which has no accessor. |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
164 tunhost = getattr(req, '_tunnel_host', None) |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
165 host = req.get_host() |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
166 if tunhost: |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
167 proxyhost = host |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
168 host = tunhost |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
169 elif req.has_proxy(): |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
170 proxyhost = req.get_host() |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
171 host = req.get_selector().split('://', 1)[1].split('/', 1)[0] |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
172 else: |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
173 proxyhost = None |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
174 |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
175 if proxyhost: |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
176 if ':' in proxyhost: |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
177 # Note: this means we'll explode if we try and use an |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
178 # IPv6 http proxy. This isn't a regression, so we |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
179 # won't worry about it for now. |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
180 proxyhost, proxyport = proxyhost.rsplit(':', 1) |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
181 else: |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
182 proxyport = 3128 # squid default |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
183 proxy = (proxyhost, proxyport) |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
184 else: |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
185 proxy = None |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
186 |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
187 if not host: |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
188 raise urllib2.URLError('no host given') |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
189 |
14346
bf85c2639700
httpconnection: correctly handle redirects from http to https
Augie Fackler <durin42@gmail.com>
parents:
14294
diff
changeset
|
190 connkey = use_ssl, host, proxy |
bf85c2639700
httpconnection: correctly handle redirects from http to https
Augie Fackler <durin42@gmail.com>
parents:
14294
diff
changeset
|
191 allconns = self._connections.get(connkey, []) |
14244
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
192 conns = [c for c in allconns if not c.busy()] |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
193 if conns: |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
194 h = conns[0] |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
195 else: |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
196 if allconns: |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
197 self.ui.debug('all connections for %s busy, making a new ' |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
198 'one\n' % host) |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
199 timeout = None |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
200 if req.timeout is not socket._GLOBAL_DEFAULT_TIMEOUT: |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
201 timeout = req.timeout |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
202 h = http_class(host, timeout=timeout, proxy_hostport=proxy) |
14346
bf85c2639700
httpconnection: correctly handle redirects from http to https
Augie Fackler <durin42@gmail.com>
parents:
14294
diff
changeset
|
203 self._connections.setdefault(connkey, []).append(h) |
14244
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
204 |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
205 headers = dict(req.headers) |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
206 headers.update(req.unredirected_hdrs) |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
207 headers = dict( |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
208 (name.title(), val) for name, val in headers.items()) |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
209 try: |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
210 path = req.get_selector() |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
211 if '://' in path: |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
212 path = path.split('://', 1)[1].split('/', 1)[1] |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
213 if path[0] != '/': |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
214 path = '/' + path |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
215 h.request(req.get_method(), path, req.data, headers) |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
216 r = h.getresponse() |
25660
328739ea70c3
global: mass rewrite to use modern exception syntax
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25429
diff
changeset
|
217 except socket.error as err: # XXX what error? |
14244
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
218 raise urllib2.URLError(err) |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
219 |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
220 # Pick apart the HTTPResponse object to get the addinfourl |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
221 # object initialized properly. |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
222 r.recv = r.read |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
223 |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
224 resp = urllib.addinfourl(r, r.headers, req.get_full_url()) |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
225 resp.code = r.status |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
226 resp.msg = r.reason |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
227 return resp |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
228 |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
229 # httplib always uses the given host/port as the socket connect |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
230 # target, and then allows full URIs in the request path, which it |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
231 # then observes and treats as a signal to do proxying instead. |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
232 def http_open(self, req): |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
233 if req.get_full_url().startswith('https'): |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
234 return self.https_open(req) |
17787
21503aa02d4f
http2: make it possible to connect w/o ssl on port 443
Augie Fackler <raf@durin42.com>
parents:
17428
diff
changeset
|
235 def makehttpcon(*args, **kwargs): |
17836
98347af64593
httpclient: fix calling convention violation
Matt Mackall <mpm@selenic.com>
parents:
17787
diff
changeset
|
236 k2 = dict(kwargs) |
98347af64593
httpclient: fix calling convention violation
Matt Mackall <mpm@selenic.com>
parents:
17787
diff
changeset
|
237 k2['use_ssl'] = False |
98347af64593
httpclient: fix calling convention violation
Matt Mackall <mpm@selenic.com>
parents:
17787
diff
changeset
|
238 return HTTPConnection(*args, **k2) |
17787
21503aa02d4f
http2: make it possible to connect w/o ssl on port 443
Augie Fackler <raf@durin42.com>
parents:
17428
diff
changeset
|
239 return self.do_open(makehttpcon, req, False) |
14244
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
240 |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
241 def https_open(self, req): |
15025
0593e8f81c71
http: pass user to readauthforuri() (fix 4a43e23b8c55)
Patrick Mezard <pmezard@gmail.com>
parents:
15005
diff
changeset
|
242 # req.get_full_url() does not contain credentials and we may |
0593e8f81c71
http: pass user to readauthforuri() (fix 4a43e23b8c55)
Patrick Mezard <pmezard@gmail.com>
parents:
15005
diff
changeset
|
243 # need them to match the certificates. |
0593e8f81c71
http: pass user to readauthforuri() (fix 4a43e23b8c55)
Patrick Mezard <pmezard@gmail.com>
parents:
15005
diff
changeset
|
244 url = req.get_full_url() |
0593e8f81c71
http: pass user to readauthforuri() (fix 4a43e23b8c55)
Patrick Mezard <pmezard@gmail.com>
parents:
15005
diff
changeset
|
245 user, password = self.pwmgr.find_stored_password(url) |
0593e8f81c71
http: pass user to readauthforuri() (fix 4a43e23b8c55)
Patrick Mezard <pmezard@gmail.com>
parents:
15005
diff
changeset
|
246 res = readauthforuri(self.ui, url, user) |
14244
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
247 if res: |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
248 group, auth = res |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
249 self.auth = auth |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
250 self.ui.debug("using auth.%s.* for authentication\n" % group) |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
251 else: |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
252 self.auth = None |
14346
bf85c2639700
httpconnection: correctly handle redirects from http to https
Augie Fackler <durin42@gmail.com>
parents:
14294
diff
changeset
|
253 return self.do_open(self._makesslconnection, req, True) |
14244
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
254 |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
255 def _makesslconnection(self, host, port=443, *args, **kwargs): |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
256 keyfile = None |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
257 certfile = None |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
258 |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
259 if args: # key_file |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
260 keyfile = args.pop(0) |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
261 if args: # cert_file |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
262 certfile = args.pop(0) |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
263 |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
264 # if the user has specified different key/cert files in |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
265 # hgrc, we prefer these |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
266 if self.auth and 'key' in self.auth and 'cert' in self.auth: |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
267 keyfile = self.auth['key'] |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
268 certfile = self.auth['cert'] |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
269 |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
270 # let host port take precedence |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
271 if ':' in host and '[' not in host or ']:' in host: |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
272 host, port = host.rsplit(':', 1) |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
273 port = int(port) |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
274 if '[' in host: |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
275 host = host[1:-1] |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
276 |
19809
50d721553198
httpconnection: properly inject ssl_wrap_socket into httpclient (issue4038)
Augie Fackler <raf@durin42.com>
parents:
17836
diff
changeset
|
277 kwargs['keyfile'] = keyfile |
50d721553198
httpconnection: properly inject ssl_wrap_socket into httpclient (issue4038)
Augie Fackler <raf@durin42.com>
parents:
17836
diff
changeset
|
278 kwargs['certfile'] = certfile |
14244
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
279 |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
280 kwargs.update(sslutil.sslkwargs(self.ui, host)) |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
281 |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
282 con = HTTPConnection(host, port, use_ssl=True, |
25429
9d1c61715939
ssl: rename ssl_wrap_socket() to conform to our naming convention
Yuya Nishihara <yuya@tcha.org>
parents:
25206
diff
changeset
|
283 ssl_wrap_socket=sslutil.wrapsocket, |
14244
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
284 ssl_validator=sslutil.validator(self.ui, host), |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
285 **kwargs) |
e7525a555a64
url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
286 return con |