annotate mercurial/url.py @ 44611:8fca7e8449a8 stable 5.3.2

histedit: add missing b prefix to a string If i18n is disabled (such as via HGPLAIN=1), `_()` doesn't convert from str to bytes, so this raises a TypeError on py3. Differential Revision: https://phab.mercurial-scm.org/D8354
author Kyle Lippincott <spectral@google.com>
date Wed, 01 Apr 2020 14:14:55 -0700
parents 9f70512ae2cf
children 9d2b2df2c2ba
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
1 # url.py - HTTP handling for mercurial
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
2 #
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
3 # Copyright 2005, 2006, 2007, 2008 Matt Mackall <mpm@selenic.com>
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
4 # Copyright 2006, 2007 Alexis S. L. Carvalho <alexis@cecm.usp.br>
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
5 # Copyright 2006 Vadim Gelfer <vadim.gelfer@gmail.com>
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
6 #
8225
46293a0c7e9f updated license to be explicit about GPL version 2
Martin Geisler <mg@lazybytes.net>
parents: 8208
diff changeset
7 # This software may be used and distributed according to the terms of the
10263
25e572394f5c Update license to GPLv2+
Matt Mackall <mpm@selenic.com>
parents: 9852
diff changeset
8 # GNU General Public License version 2 or any later version.
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
9
25990
c3efcdba08ea url: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25837
diff changeset
10 from __future__ import absolute_import
c3efcdba08ea url: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25837
diff changeset
11
c3efcdba08ea url: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25837
diff changeset
12 import base64
c3efcdba08ea url: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25837
diff changeset
13 import os
c3efcdba08ea url: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25837
diff changeset
14 import socket
37100
9e6d3465f17e url: make logginghttphandler compatible with Python 2.7.6
Yuya Nishihara <yuya@tcha.org>
parents: 37084
diff changeset
15 import sys
25990
c3efcdba08ea url: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25837
diff changeset
16
c3efcdba08ea url: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25837
diff changeset
17 from .i18n import _
43089
c59eb1560c44 py3: manually import getattr where it is needed
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43077
diff changeset
18 from .pycompat import getattr
25990
c3efcdba08ea url: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25837
diff changeset
19 from . import (
30820
6a70cf94d1b5 py3: replace pycompat.getenv with encoding.environ.get
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30763
diff changeset
20 encoding,
26587
56b2bcea2529 error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25990
diff changeset
21 error,
25990
c3efcdba08ea url: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25837
diff changeset
22 httpconnection as httpconnectionmod,
c3efcdba08ea url: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25837
diff changeset
23 keepalive,
34428
0ee9cf8d054a url: use native strings for header values
Augie Fackler <augie@google.com>
parents: 33499
diff changeset
24 pycompat,
25990
c3efcdba08ea url: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25837
diff changeset
25 sslutil,
34466
1232f7fa00c3 cleanup: use urllibcompat for renamed methods on urllib request objects
Augie Fackler <augie@google.com>
parents: 34428
diff changeset
26 urllibcompat,
25990
c3efcdba08ea url: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25837
diff changeset
27 util,
c3efcdba08ea url: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25837
diff changeset
28 )
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
29 from .utils import stringutil
29455
0c741fd6158a py3: conditionalize httplib import
Pulkit Goyal <7895pulkit@gmail.com>
parents: 29379
diff changeset
30
0c741fd6158a py3: conditionalize httplib import
Pulkit Goyal <7895pulkit@gmail.com>
parents: 29379
diff changeset
31 httplib = util.httplib
28861
86db5cb55d46 pycompat: switch to util.stringio for py3 compat
timeless <timeless@mozdev.org>
parents: 26806
diff changeset
32 stringio = util.stringio
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28861
diff changeset
33 urlerr = util.urlerr
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28861
diff changeset
34 urlreq = util.urlreq
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28861
diff changeset
35
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
36
34694
2976cf87a60a url: add cgi.escape equivalent for bytestrings
Augie Fackler <augie@google.com>
parents: 34466
diff changeset
37 def escape(s, quote=None):
2976cf87a60a url: add cgi.escape equivalent for bytestrings
Augie Fackler <augie@google.com>
parents: 34466
diff changeset
38 '''Replace special characters "&", "<" and ">" to HTML-safe sequences.
2976cf87a60a url: add cgi.escape equivalent for bytestrings
Augie Fackler <augie@google.com>
parents: 34466
diff changeset
39 If the optional flag quote is true, the quotation mark character (")
2976cf87a60a url: add cgi.escape equivalent for bytestrings
Augie Fackler <augie@google.com>
parents: 34466
diff changeset
40 is also translated.
2976cf87a60a url: add cgi.escape equivalent for bytestrings
Augie Fackler <augie@google.com>
parents: 34466
diff changeset
41
2976cf87a60a url: add cgi.escape equivalent for bytestrings
Augie Fackler <augie@google.com>
parents: 34466
diff changeset
42 This is the same as cgi.escape in Python, but always operates on
2976cf87a60a url: add cgi.escape equivalent for bytestrings
Augie Fackler <augie@google.com>
parents: 34466
diff changeset
43 bytes, whereas cgi.escape in Python 3 only works on unicodes.
2976cf87a60a url: add cgi.escape equivalent for bytestrings
Augie Fackler <augie@google.com>
parents: 34466
diff changeset
44 '''
2976cf87a60a url: add cgi.escape equivalent for bytestrings
Augie Fackler <augie@google.com>
parents: 34466
diff changeset
45 s = s.replace(b"&", b"&amp;")
2976cf87a60a url: add cgi.escape equivalent for bytestrings
Augie Fackler <augie@google.com>
parents: 34466
diff changeset
46 s = s.replace(b"<", b"&lt;")
2976cf87a60a url: add cgi.escape equivalent for bytestrings
Augie Fackler <augie@google.com>
parents: 34466
diff changeset
47 s = s.replace(b">", b"&gt;")
2976cf87a60a url: add cgi.escape equivalent for bytestrings
Augie Fackler <augie@google.com>
parents: 34466
diff changeset
48 if quote:
2976cf87a60a url: add cgi.escape equivalent for bytestrings
Augie Fackler <augie@google.com>
parents: 34466
diff changeset
49 s = s.replace(b'"', b"&quot;")
2976cf87a60a url: add cgi.escape equivalent for bytestrings
Augie Fackler <augie@google.com>
parents: 34466
diff changeset
50 return s
2976cf87a60a url: add cgi.escape equivalent for bytestrings
Augie Fackler <augie@google.com>
parents: 34466
diff changeset
51
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
52
29377
2c019aac6b99 url: extract password database from password manager
liscju <piotr.listkiewicz@gmail.com>
parents: 29252
diff changeset
53 class passwordmgr(object):
2c019aac6b99 url: extract password database from password manager
liscju <piotr.listkiewicz@gmail.com>
parents: 29252
diff changeset
54 def __init__(self, ui, passwddb):
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
55 self.ui = ui
29377
2c019aac6b99 url: extract password database from password manager
liscju <piotr.listkiewicz@gmail.com>
parents: 29252
diff changeset
56 self.passwddb = passwddb
2c019aac6b99 url: extract password database from password manager
liscju <piotr.listkiewicz@gmail.com>
parents: 29252
diff changeset
57
2c019aac6b99 url: extract password database from password manager
liscju <piotr.listkiewicz@gmail.com>
parents: 29252
diff changeset
58 def add_password(self, realm, uri, user, passwd):
2c019aac6b99 url: extract password database from password manager
liscju <piotr.listkiewicz@gmail.com>
parents: 29252
diff changeset
59 return self.passwddb.add_password(realm, uri, user, passwd)
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
60
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
61 def find_user_password(self, realm, authuri):
41449
bc776c31c093 url: add some defensive asserts on expected incoming types
Augie Fackler <augie@google.com>
parents: 40043
diff changeset
62 assert isinstance(realm, (type(None), str))
bc776c31c093 url: add some defensive asserts on expected incoming types
Augie Fackler <augie@google.com>
parents: 40043
diff changeset
63 assert isinstance(authuri, str)
29377
2c019aac6b99 url: extract password database from password manager
liscju <piotr.listkiewicz@gmail.com>
parents: 29252
diff changeset
64 authinfo = self.passwddb.find_user_password(realm, authuri)
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
65 user, passwd = authinfo
41450
d437d1e2a711 url: convert some variables back to bytes
Augie Fackler <augie@google.com>
parents: 41449
diff changeset
66 user, passwd = pycompat.bytesurl(user), pycompat.bytesurl(passwd)
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
67 if user and passwd:
8333
89c80c3dc584 allow http authentication information to be specified in the configuration
Sune Foldager <cryo@cyanite.org>
parents: 8225
diff changeset
68 self._writedebug(user, passwd)
41590
349c8879becd py3: ensure the HTTP password manager returns strings, not bytes
Matt Harbison <matt_harbison@yahoo.com>
parents: 41450
diff changeset
69 return (pycompat.strurl(user), pycompat.strurl(passwd))
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
70
15005
4a43e23b8c55 hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents: 14244
diff changeset
71 if not user or not passwd:
15025
0593e8f81c71 http: pass user to readauthforuri() (fix 4a43e23b8c55)
Patrick Mezard <pmezard@gmail.com>
parents: 15005
diff changeset
72 res = httpconnectionmod.readauthforuri(self.ui, authuri, user)
13372
5bced0d28a39 url: return the matched authentication group name from readauthforuri()
Steve Borho <steve@borho.org>
parents: 13371
diff changeset
73 if res:
5bced0d28a39 url: return the matched authentication group name from readauthforuri()
Steve Borho <steve@borho.org>
parents: 13371
diff changeset
74 group, auth = res
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
75 user, passwd = auth.get(b'username'), auth.get(b'password')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
76 self.ui.debug(b"using auth.%s.* for authentication\n" % group)
8333
89c80c3dc584 allow http authentication information to be specified in the configuration
Sune Foldager <cryo@cyanite.org>
parents: 8225
diff changeset
77 if not user or not passwd:
36652
8381126bf43c url: more bytes/unicodes fussing in url.py around auth handling
Augie Fackler <augie@google.com>
parents: 36454
diff changeset
78 u = util.url(pycompat.bytesurl(authuri))
20291
7d589d923b8a url: added authuri when login information is requested (issue3209)
Lucas Moscovicz <lmoscovicz@fb.com>
parents: 20174
diff changeset
79 u.query = None
8333
89c80c3dc584 allow http authentication information to be specified in the configuration
Sune Foldager <cryo@cyanite.org>
parents: 8225
diff changeset
80 if not self.ui.interactive():
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
81 raise error.Abort(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
82 _(b'http authorization required for %s')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
83 % util.hidepassword(bytes(u))
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
84 )
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
85
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
86 self.ui.write(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
87 _(b"http authorization required for %s\n")
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
88 % util.hidepassword(bytes(u))
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
89 )
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
90 self.ui.write(_(b"realm: %s\n") % pycompat.bytesurl(realm))
8333
89c80c3dc584 allow http authentication information to be specified in the configuration
Sune Foldager <cryo@cyanite.org>
parents: 8225
diff changeset
91 if user:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
92 self.ui.write(_(b"user: %s\n") % user)
8333
89c80c3dc584 allow http authentication information to be specified in the configuration
Sune Foldager <cryo@cyanite.org>
parents: 8225
diff changeset
93 else:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
94 user = self.ui.prompt(_(b"user:"), default=None)
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
95
8333
89c80c3dc584 allow http authentication information to be specified in the configuration
Sune Foldager <cryo@cyanite.org>
parents: 8225
diff changeset
96 if not passwd:
89c80c3dc584 allow http authentication information to be specified in the configuration
Sune Foldager <cryo@cyanite.org>
parents: 8225
diff changeset
97 passwd = self.ui.getpass()
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
98
29377
2c019aac6b99 url: extract password database from password manager
liscju <piotr.listkiewicz@gmail.com>
parents: 29252
diff changeset
99 self.passwddb.add_password(realm, authuri, user, passwd)
8333
89c80c3dc584 allow http authentication information to be specified in the configuration
Sune Foldager <cryo@cyanite.org>
parents: 8225
diff changeset
100 self._writedebug(user, passwd)
41590
349c8879becd py3: ensure the HTTP password manager returns strings, not bytes
Matt Harbison <matt_harbison@yahoo.com>
parents: 41450
diff changeset
101 return (pycompat.strurl(user), pycompat.strurl(passwd))
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
102
8333
89c80c3dc584 allow http authentication information to be specified in the configuration
Sune Foldager <cryo@cyanite.org>
parents: 8225
diff changeset
103 def _writedebug(self, user, passwd):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
104 msg = _(b'http auth: user %s, password %s\n')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
105 self.ui.debug(msg % (user, passwd and b'*' * len(passwd) or b'not set'))
8333
89c80c3dc584 allow http authentication information to be specified in the configuration
Sune Foldager <cryo@cyanite.org>
parents: 8225
diff changeset
106
15025
0593e8f81c71 http: pass user to readauthforuri() (fix 4a43e23b8c55)
Patrick Mezard <pmezard@gmail.com>
parents: 15005
diff changeset
107 def find_stored_password(self, authuri):
29377
2c019aac6b99 url: extract password database from password manager
liscju <piotr.listkiewicz@gmail.com>
parents: 29252
diff changeset
108 return self.passwddb.find_user_password(None, authuri)
15025
0593e8f81c71 http: pass user to readauthforuri() (fix 4a43e23b8c55)
Patrick Mezard <pmezard@gmail.com>
parents: 15005
diff changeset
109
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
110
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28861
diff changeset
111 class proxyhandler(urlreq.proxyhandler):
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
112 def __init__(self, ui):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
113 proxyurl = ui.config(b"http_proxy", b"host") or encoding.environ.get(
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
114 b'http_proxy'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
115 )
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
116 # XXX proxyauthinfo = None
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
117
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
118 if proxyurl:
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
119 # proxy can be proper url or host[:port]
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
120 if not (
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
121 proxyurl.startswith(b'http:') or proxyurl.startswith(b'https:')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
122 ):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
123 proxyurl = b'http://' + proxyurl + b'/'
14076
924c82157d46 url: move URL parsing functions into util to improve startup time
Brodie Rao <brodie@bitheap.org>
parents: 14071
diff changeset
124 proxy = util.url(proxyurl)
13820
65b89e80f892 url: use url.url in proxyhandler
Brodie Rao <brodie@bitheap.org>
parents: 13819
diff changeset
125 if not proxy.user:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
126 proxy.user = ui.config(b"http_proxy", b"user")
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
127 proxy.passwd = ui.config(b"http_proxy", b"passwd")
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
128
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
129 # see if we should use a proxy for this url
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
130 no_list = [b"localhost", b"127.0.0.1"]
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
131 no_list.extend(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
132 [p.lower() for p in ui.configlist(b"http_proxy", b"no")]
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
133 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
134 no_list.extend(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
135 [
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
136 p.strip().lower()
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
137 for p in encoding.environ.get(b"no_proxy", b'').split(b',')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
138 if p.strip()
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
139 ]
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
140 )
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
141 # "http_proxy.always" config is for running tests on localhost
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
142 if ui.configbool(b"http_proxy", b"always"):
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
143 self.no_list = []
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
144 else:
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
145 self.no_list = no_list
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
146
41710
4028897dfa05 url: always use str for proxy configuration
Gregory Szorc <gregory.szorc@gmail.com>
parents: 41708
diff changeset
147 # Keys and values need to be str because the standard library
4028897dfa05 url: always use str for proxy configuration
Gregory Szorc <gregory.szorc@gmail.com>
parents: 41708
diff changeset
148 # expects them to be.
4028897dfa05 url: always use str for proxy configuration
Gregory Szorc <gregory.szorc@gmail.com>
parents: 41708
diff changeset
149 proxyurl = str(proxy)
43506
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43106
diff changeset
150 proxies = {'http': proxyurl, 'https': proxyurl}
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
151 ui.debug(b'proxying through %s\n' % util.hidepassword(bytes(proxy)))
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
152 else:
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
153 proxies = {}
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
154
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28861
diff changeset
155 urlreq.proxyhandler.__init__(self, proxies)
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
156 self.ui = ui
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
157
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
158 def proxy_open(self, req, proxy, type_):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
159 host = pycompat.bytesurl(urllibcompat.gethost(req)).split(b':')[0]
19535
df2155ebf502 proxy: allow wildcards in the no proxy list (issue1821)
Matt Mackall <mpm@selenic.com>
parents: 18176
diff changeset
160 for e in self.no_list:
df2155ebf502 proxy: allow wildcards in the no proxy list (issue1821)
Matt Mackall <mpm@selenic.com>
parents: 18176
diff changeset
161 if host == e:
df2155ebf502 proxy: allow wildcards in the no proxy list (issue1821)
Matt Mackall <mpm@selenic.com>
parents: 18176
diff changeset
162 return None
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
163 if e.startswith(b'*.') and host.endswith(e[2:]):
19535
df2155ebf502 proxy: allow wildcards in the no proxy list (issue1821)
Matt Mackall <mpm@selenic.com>
parents: 18176
diff changeset
164 return None
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
165 if e.startswith(b'.') and host.endswith(e[1:]):
19535
df2155ebf502 proxy: allow wildcards in the no proxy list (issue1821)
Matt Mackall <mpm@selenic.com>
parents: 18176
diff changeset
166 return None
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
167
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28861
diff changeset
168 return urlreq.proxyhandler.proxy_open(self, req, proxy, type_)
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
169
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
170
13420
051f498628f7 url: refactor _gen_sendfile
Mads Kiilerich <mads@kiilerich.com>
parents: 13419
diff changeset
171 def _gen_sendfile(orgsend):
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
172 def _sendfile(self, data):
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
173 # send a file
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents: 14204
diff changeset
174 if isinstance(data, httpconnectionmod.httpsendfile):
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
175 # if auth required, some data sent twice, so rewind here
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
176 data.seek(0)
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
177 for chunk in util.filechunkiter(data):
13420
051f498628f7 url: refactor _gen_sendfile
Mads Kiilerich <mads@kiilerich.com>
parents: 13419
diff changeset
178 orgsend(self, chunk)
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
179 else:
13420
051f498628f7 url: refactor _gen_sendfile
Mads Kiilerich <mads@kiilerich.com>
parents: 13419
diff changeset
180 orgsend(self, data)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
181
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
182 return _sendfile
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
183
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
184
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
185 has_https = util.safehasattr(urlreq, b'httpshandler')
10409
4c94a3df4b10 url: SSL server certificate verification using web.cacerts file (issue1174)
Henrik Stuart <hg@hstuart.dk>
parents: 10408
diff changeset
186
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
187
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
188 class httpconnection(keepalive.HTTPConnection):
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
189 # must be able to send big bundle as stream.
13420
051f498628f7 url: refactor _gen_sendfile
Mads Kiilerich <mads@kiilerich.com>
parents: 13419
diff changeset
190 send = _gen_sendfile(keepalive.HTTPConnection.send)
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
191
8590
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967)
Henrik Stuart <hg@hstuart.dk>
parents: 8344
diff changeset
192 def getresponse(self):
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967)
Henrik Stuart <hg@hstuart.dk>
parents: 8344
diff changeset
193 proxyres = getattr(self, 'proxyres', None)
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967)
Henrik Stuart <hg@hstuart.dk>
parents: 8344
diff changeset
194 if proxyres:
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967)
Henrik Stuart <hg@hstuart.dk>
parents: 8344
diff changeset
195 if proxyres.will_close:
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967)
Henrik Stuart <hg@hstuart.dk>
parents: 8344
diff changeset
196 self.close()
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967)
Henrik Stuart <hg@hstuart.dk>
parents: 8344
diff changeset
197 self.proxyres = None
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967)
Henrik Stuart <hg@hstuart.dk>
parents: 8344
diff changeset
198 return proxyres
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967)
Henrik Stuart <hg@hstuart.dk>
parents: 8344
diff changeset
199 return keepalive.HTTPConnection.getresponse(self)
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967)
Henrik Stuart <hg@hstuart.dk>
parents: 8344
diff changeset
200
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
201
41708
d20f1594ff4a url: always access req._tunnel_host
Gregory Szorc <gregory.szorc@gmail.com>
parents: 41590
diff changeset
202 # Large parts of this function have their origin from before Python 2.6
d20f1594ff4a url: always access req._tunnel_host
Gregory Szorc <gregory.szorc@gmail.com>
parents: 41590
diff changeset
203 # and could potentially be removed.
9852
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
204 def _generic_start_transaction(handler, h, req):
41708
d20f1594ff4a url: always access req._tunnel_host
Gregory Szorc <gregory.szorc@gmail.com>
parents: 41590
diff changeset
205 tunnel_host = req._tunnel_host
14964
376c32a5ccdc url: replace uses of hasattr with safehasattr or getattr
Augie Fackler <durin42@gmail.com>
parents: 14244
diff changeset
206 if tunnel_host:
43506
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43106
diff changeset
207 if tunnel_host[:7] not in ['http://', 'https:/']:
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43106
diff changeset
208 tunnel_host = 'https://' + tunnel_host
9852
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
209 new_tunnel = True
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
210 else:
34466
1232f7fa00c3 cleanup: use urllibcompat for renamed methods on urllib request objects
Augie Fackler <augie@google.com>
parents: 34428
diff changeset
211 tunnel_host = urllibcompat.getselector(req)
9852
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
212 new_tunnel = False
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
213
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
214 if new_tunnel or tunnel_host == urllibcompat.getfullurl(req): # has proxy
41710
4028897dfa05 url: always use str for proxy configuration
Gregory Szorc <gregory.szorc@gmail.com>
parents: 41708
diff changeset
215 u = util.url(pycompat.bytesurl(tunnel_host))
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
216 if new_tunnel or u.scheme == b'https': # only use CONNECT for HTTPS
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
217 h.realhostport = b':'.join([u.host, (u.port or b'443')])
9852
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
218 h.headers = req.headers.copy()
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
219 h.headers.update(handler.parent.addheaders)
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
220 return
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
221
10415
677f15da38c1 url: proxy handling, simplify and correctly deal with IPv6
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 10411
diff changeset
222 h.realhostport = None
9852
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
223 h.headers = None
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
224
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
225
9852
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
226 def _generic_proxytunnel(self):
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
227 proxyheaders = dict(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
228 [
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
229 (x, self.headers[x])
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
230 for x in self.headers
43506
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43106
diff changeset
231 if x.lower().startswith('proxy-')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
232 ]
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
233 )
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
234 self.send(b'CONNECT %s HTTP/1.0\r\n' % self.realhostport)
43106
d783f945a701 py3: finish porting iteritems() to pycompat and remove source transformer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43089
diff changeset
235 for header in pycompat.iteritems(proxyheaders):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
236 self.send(b'%s: %s\r\n' % header)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
237 self.send(b'\r\n')
9852
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
238
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
239 # majority of the following code is duplicated from
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
240 # httplib.HTTPConnection as there are no adequate places to
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
241 # override functions to provide the needed functionality
41711
544035846830 url: don't pass strict argument on Python 3
Gregory Szorc <gregory.szorc@gmail.com>
parents: 41710
diff changeset
242 # strict was removed in Python 3.4.
544035846830 url: don't pass strict argument on Python 3
Gregory Szorc <gregory.szorc@gmail.com>
parents: 41710
diff changeset
243 kwargs = {}
544035846830 url: don't pass strict argument on Python 3
Gregory Szorc <gregory.szorc@gmail.com>
parents: 41710
diff changeset
244 if not pycompat.ispy3:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
245 kwargs[b'strict'] = self.strict
41711
544035846830 url: don't pass strict argument on Python 3
Gregory Szorc <gregory.szorc@gmail.com>
parents: 41710
diff changeset
246
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
247 res = self.response_class(self.sock, method=self._method, **kwargs)
9852
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
248
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
249 while True:
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
250 version, status, reason = res._read_status()
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
251 if status != httplib.CONTINUE:
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
252 break
29729
44ea12756fef url: use `iter(callable, sentinel)` instead of while True
Augie Fackler <augie@google.com>
parents: 29662
diff changeset
253 # skip lines that are all whitespace
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
254 list(iter(lambda: res.fp.readline().strip(), b''))
9852
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
255 res.status = status
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
256 res.reason = reason.strip()
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
257
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
258 if res.status == 200:
29729
44ea12756fef url: use `iter(callable, sentinel)` instead of while True
Augie Fackler <augie@google.com>
parents: 29662
diff changeset
259 # skip lines until we find a blank line
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
260 list(iter(res.fp.readline, b'\r\n'))
9852
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
261 return True
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
262
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
263 if version == b'HTTP/1.0':
9852
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
264 res.version = 10
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
265 elif version.startswith(b'HTTP/1.'):
9852
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
266 res.version = 11
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
267 elif version == b'HTTP/0.9':
9852
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
268 res.version = 9
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
269 else:
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
270 raise httplib.UnknownProtocol(version)
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
271
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
272 if res.version == 9:
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
273 res.length = None
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
274 res.chunked = 0
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
275 res.will_close = 1
28861
86db5cb55d46 pycompat: switch to util.stringio for py3 compat
timeless <timeless@mozdev.org>
parents: 26806
diff changeset
276 res.msg = httplib.HTTPMessage(stringio())
9852
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
277 return False
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
278
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
279 res.msg = httplib.HTTPMessage(res.fp)
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
280 res.msg.fp = None
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
281
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
282 # are we using the chunked-style of transfer encoding?
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
283 trenc = res.msg.getheader(b'transfer-encoding')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
284 if trenc and trenc.lower() == b"chunked":
9852
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
285 res.chunked = 1
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
286 res.chunk_left = None
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
287 else:
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
288 res.chunked = 0
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
289
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
290 # will the connection close at the end of the response?
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
291 res.will_close = res._check_close()
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
292
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
293 # do we have a Content-Length?
17428
72803c8edaa4 avoid using abbreviations that look like spelling errors
Mads Kiilerich <mads@kiilerich.com>
parents: 17424
diff changeset
294 # NOTE: RFC 2616, section 4.4, #3 says we ignore this if
72803c8edaa4 avoid using abbreviations that look like spelling errors
Mads Kiilerich <mads@kiilerich.com>
parents: 17424
diff changeset
295 # transfer-encoding is "chunked"
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
296 length = res.msg.getheader(b'content-length')
9852
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
297 if length and not res.chunked:
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
298 try:
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
299 res.length = int(length)
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
300 except ValueError:
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
301 res.length = None
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
302 else:
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
303 if res.length < 0: # ignore nonsensical negative lengths
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
304 res.length = None
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
305 else:
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
306 res.length = None
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
307
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
308 # does the body have a fixed length? (of zero)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
309 if (
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
310 status == httplib.NO_CONTENT
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
311 or status == httplib.NOT_MODIFIED
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
312 or 100 <= status < 200
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
313 or res._method == b'HEAD' # 1xx codes
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
314 ):
9852
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
315 res.length = 0
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
316
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
317 # if the connection remains open, and we aren't using chunked, and
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
318 # a content-length was not provided, then assume that the connection
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
319 # WILL close.
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
320 if not res.will_close and not res.chunked and res.length is None:
9852
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
321 res.will_close = 1
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
322
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
323 self.proxyres = res
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
324
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
325 return False
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
326
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
327
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
328 class httphandler(keepalive.HTTPHandler):
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
329 def http_open(self, req):
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
330 return self.do_open(httpconnection, req)
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
331
8590
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967)
Henrik Stuart <hg@hstuart.dk>
parents: 8344
diff changeset
332 def _start_transaction(self, h, req):
9852
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
333 _generic_start_transaction(self, h, req)
8590
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967)
Henrik Stuart <hg@hstuart.dk>
parents: 8344
diff changeset
334 return keepalive.HTTPHandler._start_transaction(self, h, req)
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967)
Henrik Stuart <hg@hstuart.dk>
parents: 8344
diff changeset
335
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
336
37011
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
337 class logginghttpconnection(keepalive.HTTPConnection):
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
338 def __init__(self, createconn, *args, **kwargs):
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
339 keepalive.HTTPConnection.__init__(self, *args, **kwargs)
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
340 self._create_connection = createconn
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
341
37100
9e6d3465f17e url: make logginghttphandler compatible with Python 2.7.6
Yuya Nishihara <yuya@tcha.org>
parents: 37084
diff changeset
342 if sys.version_info < (2, 7, 7):
9e6d3465f17e url: make logginghttphandler compatible with Python 2.7.6
Yuya Nishihara <yuya@tcha.org>
parents: 37084
diff changeset
343 # copied from 2.7.14, since old implementations directly call
9e6d3465f17e url: make logginghttphandler compatible with Python 2.7.6
Yuya Nishihara <yuya@tcha.org>
parents: 37084
diff changeset
344 # socket.create_connection()
9e6d3465f17e url: make logginghttphandler compatible with Python 2.7.6
Yuya Nishihara <yuya@tcha.org>
parents: 37084
diff changeset
345 def connect(self):
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
346 self.sock = self._create_connection(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
347 (self.host, self.port), self.timeout, self.source_address
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
348 )
37100
9e6d3465f17e url: make logginghttphandler compatible with Python 2.7.6
Yuya Nishihara <yuya@tcha.org>
parents: 37084
diff changeset
349 if self._tunnel_host:
9e6d3465f17e url: make logginghttphandler compatible with Python 2.7.6
Yuya Nishihara <yuya@tcha.org>
parents: 37084
diff changeset
350 self._tunnel()
9e6d3465f17e url: make logginghttphandler compatible with Python 2.7.6
Yuya Nishihara <yuya@tcha.org>
parents: 37084
diff changeset
351
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
352
37011
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
353 class logginghttphandler(httphandler):
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
354 """HTTP handler that logs socket I/O."""
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
355
40043
6509fcec830c url: allow to configure timeout on http connection
Cédric Krier <ced@b2ck.com>
parents: 40034
diff changeset
356 def __init__(self, logfh, name, observeropts, timeout=None):
6509fcec830c url: allow to configure timeout on http connection
Cédric Krier <ced@b2ck.com>
parents: 40034
diff changeset
357 super(logginghttphandler, self).__init__(timeout=timeout)
37011
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
358
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
359 self._logfh = logfh
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
360 self._logname = name
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
361 self._observeropts = observeropts
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
362
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
363 # do_open() calls the passed class to instantiate an HTTPConnection. We
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
364 # pass in a callable method that creates a custom HTTPConnection instance
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
365 # whose callback to create the socket knows how to proxy the socket.
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
366 def http_open(self, req):
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
367 return self.do_open(self._makeconnection, req)
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
368
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
369 def _makeconnection(self, *args, **kwargs):
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
370 def createconnection(*args, **kwargs):
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
371 sock = socket.create_connection(*args, **kwargs)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
372 return util.makeloggingsocket(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
373 self._logfh, sock, self._logname, **self._observeropts
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
374 )
37011
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
375
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
376 return logginghttpconnection(createconnection, *args, **kwargs)
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
377
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
378
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
379 if has_https:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
380
40031
f2dffa1359c6 url: have httpsconnection inherit from our custom HTTPConnection
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40018
diff changeset
381 class httpsconnection(keepalive.HTTPConnection):
13424
08f9c587141f url: merge BetterHTTPS with httpsconnection to get some proxy https validation
Mads Kiilerich <mads@kiilerich.com>
parents: 13422
diff changeset
382 response_class = keepalive.HTTPResponse
25414
f7ccbc2776b7 https: do not inherit httplib.HTTPSConnection that creates unused SSLContext
Yuya Nishihara <yuya@tcha.org>
parents: 25207
diff changeset
383 default_port = httplib.HTTPS_PORT
13424
08f9c587141f url: merge BetterHTTPS with httpsconnection to get some proxy https validation
Mads Kiilerich <mads@kiilerich.com>
parents: 13422
diff changeset
384 # must be able to send big bundle as stream.
08f9c587141f url: merge BetterHTTPS with httpsconnection to get some proxy https validation
Mads Kiilerich <mads@kiilerich.com>
parents: 13422
diff changeset
385 send = _gen_sendfile(keepalive.safesend)
25414
f7ccbc2776b7 https: do not inherit httplib.HTTPSConnection that creates unused SSLContext
Yuya Nishihara <yuya@tcha.org>
parents: 25207
diff changeset
386 getresponse = keepalive.wrapgetresponse(httplib.HTTPConnection)
f7ccbc2776b7 https: do not inherit httplib.HTTPSConnection that creates unused SSLContext
Yuya Nishihara <yuya@tcha.org>
parents: 25207
diff changeset
387
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
388 def __init__(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
389 self,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
390 host,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
391 port=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
392 key_file=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
393 cert_file=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
394 *args,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
395 **kwargs
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
396 ):
40031
f2dffa1359c6 url: have httpsconnection inherit from our custom HTTPConnection
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40018
diff changeset
397 keepalive.HTTPConnection.__init__(self, host, port, *args, **kwargs)
25414
f7ccbc2776b7 https: do not inherit httplib.HTTPSConnection that creates unused SSLContext
Yuya Nishihara <yuya@tcha.org>
parents: 25207
diff changeset
398 self.key_file = key_file
f7ccbc2776b7 https: do not inherit httplib.HTTPSConnection that creates unused SSLContext
Yuya Nishihara <yuya@tcha.org>
parents: 25207
diff changeset
399 self.cert_file = cert_file
9726
430e59ff3437 keepalive: handle broken pipes gracefully during large POSTs
Augie Fackler <durin42@gmail.com>
parents: 9467
diff changeset
400
10409
4c94a3df4b10 url: SSL server certificate verification using web.cacerts file (issue1174)
Henrik Stuart <hg@hstuart.dk>
parents: 10408
diff changeset
401 def connect(self):
29662
b76ea1384bf2 url: drop compatibility wrapper of socket.create_connection()
Yuya Nishihara <yuya@tcha.org>
parents: 29639
diff changeset
402 self.sock = socket.create_connection((self.host, self.port))
13422
ebce5196b9db url: always create BetterHTTPS connections the same way
Mads Kiilerich <mads@kiilerich.com>
parents: 13421
diff changeset
403
13421
bd8bfa85d5a5 url: refactor BetterHTTPS.connect
Mads Kiilerich <mads@kiilerich.com>
parents: 13420
diff changeset
404 host = self.host
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
405 if self.realhostport: # use CONNECT proxy
14064
e4bfb9c337f3 remove unused imports and variables
Alexander Solovyov <alexander@solovyov.net>
parents: 13902
diff changeset
406 _generic_proxytunnel(self)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
407 host = self.realhostport.rsplit(b':', 1)[0]
25429
9d1c61715939 ssl: rename ssl_wrap_socket() to conform to our naming convention
Yuya Nishihara <yuya@tcha.org>
parents: 25415
diff changeset
408 self.sock = sslutil.wrapsocket(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
409 self.sock,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
410 self.key_file,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
411 self.cert_file,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
412 ui=self.ui,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
413 serverhostname=host,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
414 )
29227
dffe78d80a6c sslutil: convert socket validation from a class to a function (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28883
diff changeset
415 sslutil.validatesocket(self.sock)
10409
4c94a3df4b10 url: SSL server certificate verification using web.cacerts file (issue1174)
Henrik Stuart <hg@hstuart.dk>
parents: 10408
diff changeset
416
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28861
diff changeset
417 class httpshandler(keepalive.KeepAliveHandler, urlreq.httpshandler):
40043
6509fcec830c url: allow to configure timeout on http connection
Cédric Krier <ced@b2ck.com>
parents: 40034
diff changeset
418 def __init__(self, ui, timeout=None):
6509fcec830c url: allow to configure timeout on http connection
Cédric Krier <ced@b2ck.com>
parents: 40034
diff changeset
419 keepalive.KeepAliveHandler.__init__(self, timeout=timeout)
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28861
diff changeset
420 urlreq.httpshandler.__init__(self)
8847
7951f385fcb7 url: support client certificate files over HTTPS (issue643)
Henrik Stuart <hg@hstuart.dk>
parents: 8590
diff changeset
421 self.ui = ui
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
422 self.pwmgr = passwordmgr(self.ui, self.ui.httppasswordmgrdb)
8847
7951f385fcb7 url: support client certificate files over HTTPS (issue643)
Henrik Stuart <hg@hstuart.dk>
parents: 8590
diff changeset
423
9852
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
424 def _start_transaction(self, h, req):
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
425 _generic_start_transaction(self, h, req)
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
426 return keepalive.KeepAliveHandler._start_transaction(self, h, req)
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
427
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
428 def https_open(self, req):
34466
1232f7fa00c3 cleanup: use urllibcompat for renamed methods on urllib request objects
Augie Fackler <augie@google.com>
parents: 34428
diff changeset
429 # urllibcompat.getfullurl() does not contain credentials
1232f7fa00c3 cleanup: use urllibcompat for renamed methods on urllib request objects
Augie Fackler <augie@google.com>
parents: 34428
diff changeset
430 # and we may need them to match the certificates.
1232f7fa00c3 cleanup: use urllibcompat for renamed methods on urllib request objects
Augie Fackler <augie@google.com>
parents: 34428
diff changeset
431 url = urllibcompat.getfullurl(req)
15025
0593e8f81c71 http: pass user to readauthforuri() (fix 4a43e23b8c55)
Patrick Mezard <pmezard@gmail.com>
parents: 15005
diff changeset
432 user, password = self.pwmgr.find_stored_password(url)
0593e8f81c71 http: pass user to readauthforuri() (fix 4a43e23b8c55)
Patrick Mezard <pmezard@gmail.com>
parents: 15005
diff changeset
433 res = httpconnectionmod.readauthforuri(self.ui, url, user)
13372
5bced0d28a39 url: return the matched authentication group name from readauthforuri()
Steve Borho <steve@borho.org>
parents: 13371
diff changeset
434 if res:
5bced0d28a39 url: return the matched authentication group name from readauthforuri()
Steve Borho <steve@borho.org>
parents: 13371
diff changeset
435 group, auth = res
5bced0d28a39 url: return the matched authentication group name from readauthforuri()
Steve Borho <steve@borho.org>
parents: 13371
diff changeset
436 self.auth = auth
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
437 self.ui.debug(b"using auth.%s.* for authentication\n" % group)
13372
5bced0d28a39 url: return the matched authentication group name from readauthforuri()
Steve Borho <steve@borho.org>
parents: 13371
diff changeset
438 else:
5bced0d28a39 url: return the matched authentication group name from readauthforuri()
Steve Borho <steve@borho.org>
parents: 13371
diff changeset
439 self.auth = None
8847
7951f385fcb7 url: support client certificate files over HTTPS (issue643)
Henrik Stuart <hg@hstuart.dk>
parents: 8590
diff changeset
440 return self.do_open(self._makeconnection, req)
7951f385fcb7 url: support client certificate files over HTTPS (issue643)
Henrik Stuart <hg@hstuart.dk>
parents: 8590
diff changeset
441
10408
50fb1fe143ff url: httplib.HTTPSConnection already handles IPv6 and port parsing fine
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 10282
diff changeset
442 def _makeconnection(self, host, port=None, *args, **kwargs):
8847
7951f385fcb7 url: support client certificate files over HTTPS (issue643)
Henrik Stuart <hg@hstuart.dk>
parents: 8590
diff changeset
443 keyfile = None
7951f385fcb7 url: support client certificate files over HTTPS (issue643)
Henrik Stuart <hg@hstuart.dk>
parents: 8590
diff changeset
444 certfile = None
7951f385fcb7 url: support client certificate files over HTTPS (issue643)
Henrik Stuart <hg@hstuart.dk>
parents: 8590
diff changeset
445
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
446 if len(args) >= 1: # key_file
10511
6f61c480f51c url: *args argument is a tuple, not a list (found by pylint)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 10484
diff changeset
447 keyfile = args[0]
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
448 if len(args) >= 2: # cert_file
10511
6f61c480f51c url: *args argument is a tuple, not a list (found by pylint)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 10484
diff changeset
449 certfile = args[1]
6f61c480f51c url: *args argument is a tuple, not a list (found by pylint)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 10484
diff changeset
450 args = args[2:]
8847
7951f385fcb7 url: support client certificate files over HTTPS (issue643)
Henrik Stuart <hg@hstuart.dk>
parents: 8590
diff changeset
451
7951f385fcb7 url: support client certificate files over HTTPS (issue643)
Henrik Stuart <hg@hstuart.dk>
parents: 8590
diff changeset
452 # if the user has specified different key/cert files in
7951f385fcb7 url: support client certificate files over HTTPS (issue643)
Henrik Stuart <hg@hstuart.dk>
parents: 8590
diff changeset
453 # hgrc, we prefer these
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
454 if self.auth and b'key' in self.auth and b'cert' in self.auth:
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
455 keyfile = self.auth[b'key']
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
456 certfile = self.auth[b'cert']
8847
7951f385fcb7 url: support client certificate files over HTTPS (issue643)
Henrik Stuart <hg@hstuart.dk>
parents: 8590
diff changeset
457
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
458 conn = httpsconnection(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
459 host, port, keyfile, certfile, *args, **kwargs
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
460 )
10409
4c94a3df4b10 url: SSL server certificate verification using web.cacerts file (issue1174)
Henrik Stuart <hg@hstuart.dk>
parents: 10408
diff changeset
461 conn.ui = self.ui
4c94a3df4b10 url: SSL server certificate verification using web.cacerts file (issue1174)
Henrik Stuart <hg@hstuart.dk>
parents: 10408
diff changeset
462 return conn
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
463
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
464
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28861
diff changeset
465 class httpdigestauthhandler(urlreq.httpdigestauthhandler):
11457
2ec346160783 http digest auth: reset redirect counter on new requests (issue2255)
Mads Kiilerich <mads@kiilerich.com>
parents: 11415
diff changeset
466 def __init__(self, *args, **kwargs):
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28861
diff changeset
467 urlreq.httpdigestauthhandler.__init__(self, *args, **kwargs)
11457
2ec346160783 http digest auth: reset redirect counter on new requests (issue2255)
Mads Kiilerich <mads@kiilerich.com>
parents: 11415
diff changeset
468 self.retried_req = None
2ec346160783 http digest auth: reset redirect counter on new requests (issue2255)
Mads Kiilerich <mads@kiilerich.com>
parents: 11415
diff changeset
469
2ec346160783 http digest auth: reset redirect counter on new requests (issue2255)
Mads Kiilerich <mads@kiilerich.com>
parents: 11415
diff changeset
470 def reset_retry_count(self):
2ec346160783 http digest auth: reset redirect counter on new requests (issue2255)
Mads Kiilerich <mads@kiilerich.com>
parents: 11415
diff changeset
471 # Python 2.6.5 will call this on 401 or 407 errors and thus loop
2ec346160783 http digest auth: reset redirect counter on new requests (issue2255)
Mads Kiilerich <mads@kiilerich.com>
parents: 11415
diff changeset
472 # forever. We disable reset_retry_count completely and reset in
2ec346160783 http digest auth: reset redirect counter on new requests (issue2255)
Mads Kiilerich <mads@kiilerich.com>
parents: 11415
diff changeset
473 # http_error_auth_reqed instead.
2ec346160783 http digest auth: reset redirect counter on new requests (issue2255)
Mads Kiilerich <mads@kiilerich.com>
parents: 11415
diff changeset
474 pass
2ec346160783 http digest auth: reset redirect counter on new requests (issue2255)
Mads Kiilerich <mads@kiilerich.com>
parents: 11415
diff changeset
475
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
476 def http_error_auth_reqed(self, auth_header, host, req, headers):
11457
2ec346160783 http digest auth: reset redirect counter on new requests (issue2255)
Mads Kiilerich <mads@kiilerich.com>
parents: 11415
diff changeset
477 # Reset the retry counter once for each request.
2ec346160783 http digest auth: reset redirect counter on new requests (issue2255)
Mads Kiilerich <mads@kiilerich.com>
parents: 11415
diff changeset
478 if req is not self.retried_req:
2ec346160783 http digest auth: reset redirect counter on new requests (issue2255)
Mads Kiilerich <mads@kiilerich.com>
parents: 11415
diff changeset
479 self.retried_req = req
2ec346160783 http digest auth: reset redirect counter on new requests (issue2255)
Mads Kiilerich <mads@kiilerich.com>
parents: 11415
diff changeset
480 self.retried = 0
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28861
diff changeset
481 return urlreq.httpdigestauthhandler.http_error_auth_reqed(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
482 self, auth_header, host, req, headers
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
483 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
484
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
485
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28861
diff changeset
486 class httpbasicauthhandler(urlreq.httpbasicauthhandler):
11844
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255)
Wagner Bruna <wbruna@softwareexpress.com.br>
parents: 11457
diff changeset
487 def __init__(self, *args, **kwargs):
20964
a939eeb94833 http: reuse authentication info after the first failed request (issue3567)
Stéphane Klein <contact@stephane-klein.info>
parents: 20291
diff changeset
488 self.auth = None
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28861
diff changeset
489 urlreq.httpbasicauthhandler.__init__(self, *args, **kwargs)
11844
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255)
Wagner Bruna <wbruna@softwareexpress.com.br>
parents: 11457
diff changeset
490 self.retried_req = None
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255)
Wagner Bruna <wbruna@softwareexpress.com.br>
parents: 11457
diff changeset
491
20964
a939eeb94833 http: reuse authentication info after the first failed request (issue3567)
Stéphane Klein <contact@stephane-klein.info>
parents: 20291
diff changeset
492 def http_request(self, request):
a939eeb94833 http: reuse authentication info after the first failed request (issue3567)
Stéphane Klein <contact@stephane-klein.info>
parents: 20291
diff changeset
493 if self.auth:
a939eeb94833 http: reuse authentication info after the first failed request (issue3567)
Stéphane Klein <contact@stephane-klein.info>
parents: 20291
diff changeset
494 request.add_unredirected_header(self.auth_header, self.auth)
a939eeb94833 http: reuse authentication info after the first failed request (issue3567)
Stéphane Klein <contact@stephane-klein.info>
parents: 20291
diff changeset
495
a939eeb94833 http: reuse authentication info after the first failed request (issue3567)
Stéphane Klein <contact@stephane-klein.info>
parents: 20291
diff changeset
496 return request
a939eeb94833 http: reuse authentication info after the first failed request (issue3567)
Stéphane Klein <contact@stephane-klein.info>
parents: 20291
diff changeset
497
a939eeb94833 http: reuse authentication info after the first failed request (issue3567)
Stéphane Klein <contact@stephane-klein.info>
parents: 20291
diff changeset
498 def https_request(self, request):
a939eeb94833 http: reuse authentication info after the first failed request (issue3567)
Stéphane Klein <contact@stephane-klein.info>
parents: 20291
diff changeset
499 if self.auth:
a939eeb94833 http: reuse authentication info after the first failed request (issue3567)
Stéphane Klein <contact@stephane-klein.info>
parents: 20291
diff changeset
500 request.add_unredirected_header(self.auth_header, self.auth)
a939eeb94833 http: reuse authentication info after the first failed request (issue3567)
Stéphane Klein <contact@stephane-klein.info>
parents: 20291
diff changeset
501
a939eeb94833 http: reuse authentication info after the first failed request (issue3567)
Stéphane Klein <contact@stephane-klein.info>
parents: 20291
diff changeset
502 return request
a939eeb94833 http: reuse authentication info after the first failed request (issue3567)
Stéphane Klein <contact@stephane-klein.info>
parents: 20291
diff changeset
503
11844
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255)
Wagner Bruna <wbruna@softwareexpress.com.br>
parents: 11457
diff changeset
504 def reset_retry_count(self):
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255)
Wagner Bruna <wbruna@softwareexpress.com.br>
parents: 11457
diff changeset
505 # Python 2.6.5 will call this on 401 or 407 errors and thus loop
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255)
Wagner Bruna <wbruna@softwareexpress.com.br>
parents: 11457
diff changeset
506 # forever. We disable reset_retry_count completely and reset in
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255)
Wagner Bruna <wbruna@softwareexpress.com.br>
parents: 11457
diff changeset
507 # http_error_auth_reqed instead.
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255)
Wagner Bruna <wbruna@softwareexpress.com.br>
parents: 11457
diff changeset
508 pass
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255)
Wagner Bruna <wbruna@softwareexpress.com.br>
parents: 11457
diff changeset
509
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255)
Wagner Bruna <wbruna@softwareexpress.com.br>
parents: 11457
diff changeset
510 def http_error_auth_reqed(self, auth_header, host, req, headers):
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255)
Wagner Bruna <wbruna@softwareexpress.com.br>
parents: 11457
diff changeset
511 # Reset the retry counter once for each request.
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255)
Wagner Bruna <wbruna@softwareexpress.com.br>
parents: 11457
diff changeset
512 if req is not self.retried_req:
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255)
Wagner Bruna <wbruna@softwareexpress.com.br>
parents: 11457
diff changeset
513 self.retried_req = req
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255)
Wagner Bruna <wbruna@softwareexpress.com.br>
parents: 11457
diff changeset
514 self.retried = 0
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28861
diff changeset
515 return urlreq.httpbasicauthhandler.http_error_auth_reqed(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
516 self, auth_header, host, req, headers
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
517 )
11844
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255)
Wagner Bruna <wbruna@softwareexpress.com.br>
parents: 11457
diff changeset
518
20964
a939eeb94833 http: reuse authentication info after the first failed request (issue3567)
Stéphane Klein <contact@stephane-klein.info>
parents: 20291
diff changeset
519 def retry_http_basic_auth(self, host, req, realm):
34466
1232f7fa00c3 cleanup: use urllibcompat for renamed methods on urllib request objects
Augie Fackler <augie@google.com>
parents: 34428
diff changeset
520 user, pw = self.passwd.find_user_password(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
521 realm, urllibcompat.getfullurl(req)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
522 )
20964
a939eeb94833 http: reuse authentication info after the first failed request (issue3567)
Stéphane Klein <contact@stephane-klein.info>
parents: 20291
diff changeset
523 if pw is not None:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
524 raw = b"%s:%s" % (pycompat.bytesurl(user), pycompat.bytesurl(pw))
43506
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43106
diff changeset
525 auth = 'Basic %s' % pycompat.strurl(base64.b64encode(raw).strip())
29639
6fd751fa58d3 url: avoid re-issuing incorrect password (issue3210)
Kim Randell <Kim.Randell@vicon.com>
parents: 29599
diff changeset
526 if req.get_header(self.auth_header, None) == auth:
20964
a939eeb94833 http: reuse authentication info after the first failed request (issue3567)
Stéphane Klein <contact@stephane-klein.info>
parents: 20291
diff changeset
527 return None
a939eeb94833 http: reuse authentication info after the first failed request (issue3567)
Stéphane Klein <contact@stephane-klein.info>
parents: 20291
diff changeset
528 self.auth = auth
a939eeb94833 http: reuse authentication info after the first failed request (issue3567)
Stéphane Klein <contact@stephane-klein.info>
parents: 20291
diff changeset
529 req.add_unredirected_header(self.auth_header, auth)
a939eeb94833 http: reuse authentication info after the first failed request (issue3567)
Stéphane Klein <contact@stephane-klein.info>
parents: 20291
diff changeset
530 return self.parent.open(req)
a939eeb94833 http: reuse authentication info after the first failed request (issue3567)
Stéphane Klein <contact@stephane-klein.info>
parents: 20291
diff changeset
531 else:
a939eeb94833 http: reuse authentication info after the first failed request (issue3567)
Stéphane Klein <contact@stephane-klein.info>
parents: 20291
diff changeset
532 return None
a939eeb94833 http: reuse authentication info after the first failed request (issue3567)
Stéphane Klein <contact@stephane-klein.info>
parents: 20291
diff changeset
533
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
534
31936
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
535 class cookiehandler(urlreq.basehandler):
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
536 def __init__(self, ui):
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
537 self.cookiejar = None
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
538
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
539 cookiefile = ui.config(b'auth', b'cookiefile')
31936
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
540 if not cookiefile:
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
541 return
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
542
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
543 cookiefile = util.expandpath(cookiefile)
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
544 try:
37735
126998dcfb08 url: some bytes/str cleanup where we interface with stdlib funcs
Augie Fackler <augie@google.com>
parents: 37100
diff changeset
545 cookiejar = util.cookielib.MozillaCookieJar(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
546 pycompat.fsdecode(cookiefile)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
547 )
31936
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
548 cookiejar.load()
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
549 self.cookiejar = cookiejar
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
550 except util.cookielib.LoadError as e:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
551 ui.warn(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
552 _(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
553 b'(error loading cookie file %s: %s; continuing without '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
554 b'cookies)\n'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
555 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
556 % (cookiefile, stringutil.forcebytestr(e))
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
557 )
31936
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
558
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
559 def http_request(self, request):
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
560 if self.cookiejar:
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
561 self.cookiejar.add_cookie_header(request)
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
562
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
563 return request
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
564
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
565 def https_request(self, request):
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
566 if self.cookiejar:
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
567 self.cookiejar.add_cookie_header(request)
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
568
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
569 return request
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
570
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
571
9347
d0474b184347 url: add support for custom handlers in extensions
Henrik Stuart <hg@hstuart.dk>
parents: 9122
diff changeset
572 handlerfuncs = []
d0474b184347 url: add support for custom handlers in extensions
Henrik Stuart <hg@hstuart.dk>
parents: 9122
diff changeset
573
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
574
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
575 def opener(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
576 ui,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
577 authinfo=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
578 useragent=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
579 loggingfh=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
580 loggingname=b's',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
581 loggingopts=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
582 sendaccept=True,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
583 ):
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
584 '''
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
585 construct an opener suitable for urllib2
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
586 authinfo will be added to the password manager
37011
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
587
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
588 The opener can be configured to log socket events if the various
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
589 ``logging*`` arguments are specified.
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
590
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
591 ``loggingfh`` denotes a file object to log events to.
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
592 ``loggingname`` denotes the name of the to print when logging.
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
593 ``loggingopts`` is a dict of keyword arguments to pass to the constructed
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
594 ``util.socketobserver`` instance.
37045
a708e1e4d7a8 url: support suppressing Accept header
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37011
diff changeset
595
a708e1e4d7a8 url: support suppressing Accept header
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37011
diff changeset
596 ``sendaccept`` allows controlling whether the ``Accept`` request header
a708e1e4d7a8 url: support suppressing Accept header
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37011
diff changeset
597 is sent. The header is sent by default.
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
598 '''
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
599 timeout = ui.configwith(float, b'http', b'timeout')
37011
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
600 handlers = []
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
601
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
602 if loggingfh:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
603 handlers.append(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
604 logginghttphandler(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
605 loggingfh, loggingname, loggingopts or {}, timeout=timeout
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
606 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
607 )
37011
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
608 # We don't yet support HTTPS when logging I/O. If we attempt to open
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
609 # an HTTPS URL, we'll likely fail due to unknown protocol.
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
610
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
611 else:
40043
6509fcec830c url: allow to configure timeout on http connection
Cédric Krier <ced@b2ck.com>
parents: 40034
diff changeset
612 handlers.append(httphandler(timeout=timeout))
37011
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
613 if has_https:
40043
6509fcec830c url: allow to configure timeout on http connection
Cédric Krier <ced@b2ck.com>
parents: 40034
diff changeset
614 handlers.append(httpshandler(ui, timeout=timeout))
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
615
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
616 handlers.append(proxyhandler(ui))
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
617
29378
fea71f66ebff url: remember http password database in ui object
liscju <piotr.listkiewicz@gmail.com>
parents: 29377
diff changeset
618 passmgr = passwordmgr(ui, ui.httppasswordmgrdb)
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
619 if authinfo is not None:
29379
fc777c855d66 largefiles: make cloning not ask two times about password (issue4883)
liscju <piotr.listkiewicz@gmail.com>
parents: 29378
diff changeset
620 realm, uris, user, passwd = authinfo
fc777c855d66 largefiles: make cloning not ask two times about password (issue4883)
liscju <piotr.listkiewicz@gmail.com>
parents: 29378
diff changeset
621 saveduser, savedpass = passmgr.find_stored_password(uris[0])
fc777c855d66 largefiles: make cloning not ask two times about password (issue4883)
liscju <piotr.listkiewicz@gmail.com>
parents: 29378
diff changeset
622 if user != saveduser or passwd:
fc777c855d66 largefiles: make cloning not ask two times about password (issue4883)
liscju <piotr.listkiewicz@gmail.com>
parents: 29378
diff changeset
623 passmgr.add_password(realm, uris, user, passwd)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
624 ui.debug(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
625 b'http auth: user %s, password %s\n'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
626 % (user, passwd and b'*' * len(passwd) or b'not set')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
627 )
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
628
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
629 handlers.extend(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
630 (httpbasicauthhandler(passmgr), httpdigestauthhandler(passmgr))
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
631 )
9347
d0474b184347 url: add support for custom handlers in extensions
Henrik Stuart <hg@hstuart.dk>
parents: 9122
diff changeset
632 handlers.extend([h(ui, passmgr) for h in handlerfuncs])
31936
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
633 handlers.append(cookiehandler(ui))
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28861
diff changeset
634 opener = urlreq.buildopener(*handlers)
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
635
40034
393e44324037 httppeer: report http statistics
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40031
diff changeset
636 # keepalive.py's handlers will populate these attributes if they exist.
393e44324037 httppeer: report http statistics
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40031
diff changeset
637 opener.requestscount = 0
393e44324037 httppeer: report http statistics
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40031
diff changeset
638 opener.sentbytescount = 0
393e44324037 httppeer: report http statistics
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40031
diff changeset
639 opener.receivedbytescount = 0
393e44324037 httppeer: report http statistics
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40031
diff changeset
640
29589
486de14eb394 url: add distribution and version to user-agent request header (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29455
diff changeset
641 # The user agent should should *NOT* be used by servers for e.g.
486de14eb394 url: add distribution and version to user-agent request header (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29455
diff changeset
642 # protocol detection or feature negotiation: there are other
486de14eb394 url: add distribution and version to user-agent request header (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29455
diff changeset
643 # facilities for that.
486de14eb394 url: add distribution and version to user-agent request header (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29455
diff changeset
644 #
486de14eb394 url: add distribution and version to user-agent request header (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29455
diff changeset
645 # "mercurial/proto-1.0" was the original user agent string and
486de14eb394 url: add distribution and version to user-agent request header (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29455
diff changeset
646 # exists for backwards compatibility reasons.
486de14eb394 url: add distribution and version to user-agent request header (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29455
diff changeset
647 #
486de14eb394 url: add distribution and version to user-agent request header (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29455
diff changeset
648 # The "(Mercurial %s)" string contains the distribution
486de14eb394 url: add distribution and version to user-agent request header (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29455
diff changeset
649 # name and version. Other client implementations should choose their
486de14eb394 url: add distribution and version to user-agent request header (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29455
diff changeset
650 # own distribution name. Since servers should not be using the user
486de14eb394 url: add distribution and version to user-agent request header (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29455
diff changeset
651 # agent string for anything, clients should be able to define whatever
486de14eb394 url: add distribution and version to user-agent request header (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29455
diff changeset
652 # user agent they deem appropriate.
35439
e7bb5fc4570c lfs: add git to the User-Agent header for blob transfers
Matt Harbison <matt_harbison@yahoo.com>
parents: 34694
diff changeset
653 #
e7bb5fc4570c lfs: add git to the User-Agent header for blob transfers
Matt Harbison <matt_harbison@yahoo.com>
parents: 34694
diff changeset
654 # The custom user agent is for lfs, because unfortunately some servers
e7bb5fc4570c lfs: add git to the User-Agent header for blob transfers
Matt Harbison <matt_harbison@yahoo.com>
parents: 34694
diff changeset
655 # do look at this value.
e7bb5fc4570c lfs: add git to the User-Agent header for blob transfers
Matt Harbison <matt_harbison@yahoo.com>
parents: 34694
diff changeset
656 if not useragent:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
657 agent = b'mercurial/proto-1.0 (Mercurial %s)' % util.version()
43506
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43106
diff changeset
658 opener.addheaders = [('User-agent', pycompat.sysstr(agent))]
35439
e7bb5fc4570c lfs: add git to the User-Agent header for blob transfers
Matt Harbison <matt_harbison@yahoo.com>
parents: 34694
diff changeset
659 else:
43506
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43106
diff changeset
660 opener.addheaders = [('User-agent', pycompat.sysstr(useragent))]
30763
a520aefb96f1 httppeer: advertise and support application/mercurial-0.2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30664
diff changeset
661
a520aefb96f1 httppeer: advertise and support application/mercurial-0.2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30664
diff changeset
662 # This header should only be needed by wire protocol requests. But it has
a520aefb96f1 httppeer: advertise and support application/mercurial-0.2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30664
diff changeset
663 # been sent on all requests since forever. We keep sending it for backwards
a520aefb96f1 httppeer: advertise and support application/mercurial-0.2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30664
diff changeset
664 # compatibility reasons. Modern versions of the wire protocol use
a520aefb96f1 httppeer: advertise and support application/mercurial-0.2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30664
diff changeset
665 # X-HgProto-<N> for advertising client support.
37045
a708e1e4d7a8 url: support suppressing Accept header
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37011
diff changeset
666 if sendaccept:
43506
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43106
diff changeset
667 opener.addheaders.append(('Accept', 'application/mercurial-0.1'))
37045
a708e1e4d7a8 url: support suppressing Accept header
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37011
diff changeset
668
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
669 return opener
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
670
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
671
42109
2a3c0106ded9 import: suppress accept: header
timeless <timeless@mozdev.org>
parents: 41711
diff changeset
672 def open(ui, url_, data=None, sendaccept=True):
14076
924c82157d46 url: move URL parsing functions into util to improve startup time
Brodie Rao <brodie@bitheap.org>
parents: 14071
diff changeset
673 u = util.url(url_)
13818
bf6156bab41b url: use url.url in url.open()
Brodie Rao <brodie@bitheap.org>
parents: 13817
diff changeset
674 if u.scheme:
bf6156bab41b url: use url.url in url.open()
Brodie Rao <brodie@bitheap.org>
parents: 13817
diff changeset
675 u.scheme = u.scheme.lower()
bf6156bab41b url: use url.url in url.open()
Brodie Rao <brodie@bitheap.org>
parents: 13817
diff changeset
676 url_, authinfo = u.authinfo()
bf6156bab41b url: use url.url in url.open()
Brodie Rao <brodie@bitheap.org>
parents: 13817
diff changeset
677 else:
bf6156bab41b url: use url.url in url.open()
Brodie Rao <brodie@bitheap.org>
parents: 13817
diff changeset
678 path = util.normpath(os.path.abspath(url_))
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
679 url_ = b'file://' + pycompat.bytesurl(urlreq.pathname2url(path))
7284
ac81ffac0f35 url: fix file:// URL handling
Patrick Mezard <pmezard@gmail.com>
parents: 7270
diff changeset
680 authinfo = None
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
681 return opener(ui, authinfo, sendaccept=sendaccept).open(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
682 pycompat.strurl(url_), data
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
683 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
684
40018
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
685
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
686 def wrapresponse(resp):
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
687 """Wrap a response object with common error handlers.
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
688
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
689 This ensures that any I/O from any consumer raises the appropriate
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
690 error and messaging.
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
691 """
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
692 origread = resp.read
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
693
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
694 class readerproxy(resp.__class__):
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
695 def read(self, size=None):
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
696 try:
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
697 return origread(size)
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
698 except httplib.IncompleteRead as e:
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
699 # e.expected is an integer if length known or None otherwise.
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
700 if e.expected:
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
701 got = len(e.partial)
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
702 total = e.expected + got
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
703 msg = _(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
704 b'HTTP request error (incomplete response; '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
705 b'expected %d bytes got %d)'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
706 ) % (total, got)
40018
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
707 else:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
708 msg = _(b'HTTP request error (incomplete response)')
40018
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
709
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
710 raise error.PeerTransportError(
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
711 msg,
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
712 hint=_(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
713 b'this may be an intermittent network failure; '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
714 b'if the error persists, consider contacting the '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
715 b'network or server operator'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
716 ),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
717 )
40018
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
718 except httplib.HTTPException as e:
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
719 raise error.PeerTransportError(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
720 _(b'HTTP request error (%s)') % e,
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
721 hint=_(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
722 b'this may be an intermittent network failure; '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
723 b'if the error persists, consider contacting the '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
724 b'network or server operator'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
725 ),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
726 )
40018
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
727
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
728 resp.__class__ = readerproxy