annotate mercurial/url.py @ 47387:75e1104f23a2

revlog: use dedicated code for reading sidedata We are about to introduce a new, dedicated, file to store sidedata. Before doing so, we make sidedata reading go through different code as reading data chunk. This will simplify some of the complexity of the next changesets. The reading is very simple right now and will need some improvement later to reuse some of the caching strategy we use for the data file. Differential Revision: https://phab.mercurial-scm.org/D10785
author Pierre-Yves David <pierre-yves.david@octobus.net>
date Thu, 27 May 2021 04:09:30 +0200
parents ffd3e823a7e5
children 8e5192e41e0b
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
1 # url.py - HTTP handling for mercurial
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
2 #
46819
d4ba4d51f85f contributor: change mentions of mpm to olivia
Raphaël Gomès <rgomes@octobus.net>
parents: 45942
diff changeset
3 # Copyright 2005, 2006, 2007, 2008 Olivia Mackall <olivia@selenic.com>
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
4 # Copyright 2006, 2007 Alexis S. L. Carvalho <alexis@cecm.usp.br>
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
5 # Copyright 2006 Vadim Gelfer <vadim.gelfer@gmail.com>
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
6 #
8225
46293a0c7e9f updated license to be explicit about GPL version 2
Martin Geisler <mg@lazybytes.net>
parents: 8208
diff changeset
7 # This software may be used and distributed according to the terms of the
10263
25e572394f5c Update license to GPLv2+
Matt Mackall <mpm@selenic.com>
parents: 9852
diff changeset
8 # GNU General Public License version 2 or any later version.
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
9
25990
c3efcdba08ea url: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25837
diff changeset
10 from __future__ import absolute_import
c3efcdba08ea url: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25837
diff changeset
11
c3efcdba08ea url: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25837
diff changeset
12 import base64
c3efcdba08ea url: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25837
diff changeset
13 import os
c3efcdba08ea url: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25837
diff changeset
14 import socket
37100
9e6d3465f17e url: make logginghttphandler compatible with Python 2.7.6
Yuya Nishihara <yuya@tcha.org>
parents: 37084
diff changeset
15 import sys
25990
c3efcdba08ea url: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25837
diff changeset
16
c3efcdba08ea url: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25837
diff changeset
17 from .i18n import _
43089
c59eb1560c44 py3: manually import getattr where it is needed
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43077
diff changeset
18 from .pycompat import getattr
25990
c3efcdba08ea url: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25837
diff changeset
19 from . import (
30820
6a70cf94d1b5 py3: replace pycompat.getenv with encoding.environ.get
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30763
diff changeset
20 encoding,
26587
56b2bcea2529 error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25990
diff changeset
21 error,
25990
c3efcdba08ea url: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25837
diff changeset
22 httpconnection as httpconnectionmod,
c3efcdba08ea url: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25837
diff changeset
23 keepalive,
34428
0ee9cf8d054a url: use native strings for header values
Augie Fackler <augie@google.com>
parents: 33499
diff changeset
24 pycompat,
25990
c3efcdba08ea url: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25837
diff changeset
25 sslutil,
34466
1232f7fa00c3 cleanup: use urllibcompat for renamed methods on urllib request objects
Augie Fackler <augie@google.com>
parents: 34428
diff changeset
26 urllibcompat,
25990
c3efcdba08ea url: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25837
diff changeset
27 util,
c3efcdba08ea url: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25837
diff changeset
28 )
46907
ffd3e823a7e5 urlutil: extract `url` related code from `util` into the new module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46819
diff changeset
29 from .utils import (
ffd3e823a7e5 urlutil: extract `url` related code from `util` into the new module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46819
diff changeset
30 stringutil,
ffd3e823a7e5 urlutil: extract `url` related code from `util` into the new module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46819
diff changeset
31 urlutil,
ffd3e823a7e5 urlutil: extract `url` related code from `util` into the new module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46819
diff changeset
32 )
29455
0c741fd6158a py3: conditionalize httplib import
Pulkit Goyal <7895pulkit@gmail.com>
parents: 29379
diff changeset
33
0c741fd6158a py3: conditionalize httplib import
Pulkit Goyal <7895pulkit@gmail.com>
parents: 29379
diff changeset
34 httplib = util.httplib
28861
86db5cb55d46 pycompat: switch to util.stringio for py3 compat
timeless <timeless@mozdev.org>
parents: 26806
diff changeset
35 stringio = util.stringio
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28861
diff changeset
36 urlerr = util.urlerr
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28861
diff changeset
37 urlreq = util.urlreq
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28861
diff changeset
38
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
39
34694
2976cf87a60a url: add cgi.escape equivalent for bytestrings
Augie Fackler <augie@google.com>
parents: 34466
diff changeset
40 def escape(s, quote=None):
45942
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45759
diff changeset
41 """Replace special characters "&", "<" and ">" to HTML-safe sequences.
34694
2976cf87a60a url: add cgi.escape equivalent for bytestrings
Augie Fackler <augie@google.com>
parents: 34466
diff changeset
42 If the optional flag quote is true, the quotation mark character (")
2976cf87a60a url: add cgi.escape equivalent for bytestrings
Augie Fackler <augie@google.com>
parents: 34466
diff changeset
43 is also translated.
2976cf87a60a url: add cgi.escape equivalent for bytestrings
Augie Fackler <augie@google.com>
parents: 34466
diff changeset
44
2976cf87a60a url: add cgi.escape equivalent for bytestrings
Augie Fackler <augie@google.com>
parents: 34466
diff changeset
45 This is the same as cgi.escape in Python, but always operates on
2976cf87a60a url: add cgi.escape equivalent for bytestrings
Augie Fackler <augie@google.com>
parents: 34466
diff changeset
46 bytes, whereas cgi.escape in Python 3 only works on unicodes.
45942
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45759
diff changeset
47 """
34694
2976cf87a60a url: add cgi.escape equivalent for bytestrings
Augie Fackler <augie@google.com>
parents: 34466
diff changeset
48 s = s.replace(b"&", b"&amp;")
2976cf87a60a url: add cgi.escape equivalent for bytestrings
Augie Fackler <augie@google.com>
parents: 34466
diff changeset
49 s = s.replace(b"<", b"&lt;")
2976cf87a60a url: add cgi.escape equivalent for bytestrings
Augie Fackler <augie@google.com>
parents: 34466
diff changeset
50 s = s.replace(b">", b"&gt;")
2976cf87a60a url: add cgi.escape equivalent for bytestrings
Augie Fackler <augie@google.com>
parents: 34466
diff changeset
51 if quote:
2976cf87a60a url: add cgi.escape equivalent for bytestrings
Augie Fackler <augie@google.com>
parents: 34466
diff changeset
52 s = s.replace(b'"', b"&quot;")
2976cf87a60a url: add cgi.escape equivalent for bytestrings
Augie Fackler <augie@google.com>
parents: 34466
diff changeset
53 return s
2976cf87a60a url: add cgi.escape equivalent for bytestrings
Augie Fackler <augie@google.com>
parents: 34466
diff changeset
54
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
55
29377
2c019aac6b99 url: extract password database from password manager
liscju <piotr.listkiewicz@gmail.com>
parents: 29252
diff changeset
56 class passwordmgr(object):
2c019aac6b99 url: extract password database from password manager
liscju <piotr.listkiewicz@gmail.com>
parents: 29252
diff changeset
57 def __init__(self, ui, passwddb):
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
58 self.ui = ui
29377
2c019aac6b99 url: extract password database from password manager
liscju <piotr.listkiewicz@gmail.com>
parents: 29252
diff changeset
59 self.passwddb = passwddb
2c019aac6b99 url: extract password database from password manager
liscju <piotr.listkiewicz@gmail.com>
parents: 29252
diff changeset
60
2c019aac6b99 url: extract password database from password manager
liscju <piotr.listkiewicz@gmail.com>
parents: 29252
diff changeset
61 def add_password(self, realm, uri, user, passwd):
2c019aac6b99 url: extract password database from password manager
liscju <piotr.listkiewicz@gmail.com>
parents: 29252
diff changeset
62 return self.passwddb.add_password(realm, uri, user, passwd)
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
63
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
64 def find_user_password(self, realm, authuri):
41449
bc776c31c093 url: add some defensive asserts on expected incoming types
Augie Fackler <augie@google.com>
parents: 40043
diff changeset
65 assert isinstance(realm, (type(None), str))
bc776c31c093 url: add some defensive asserts on expected incoming types
Augie Fackler <augie@google.com>
parents: 40043
diff changeset
66 assert isinstance(authuri, str)
29377
2c019aac6b99 url: extract password database from password manager
liscju <piotr.listkiewicz@gmail.com>
parents: 29252
diff changeset
67 authinfo = self.passwddb.find_user_password(realm, authuri)
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
68 user, passwd = authinfo
41450
d437d1e2a711 url: convert some variables back to bytes
Augie Fackler <augie@google.com>
parents: 41449
diff changeset
69 user, passwd = pycompat.bytesurl(user), pycompat.bytesurl(passwd)
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
70 if user and passwd:
8333
89c80c3dc584 allow http authentication information to be specified in the configuration
Sune Foldager <cryo@cyanite.org>
parents: 8225
diff changeset
71 self._writedebug(user, passwd)
41590
349c8879becd py3: ensure the HTTP password manager returns strings, not bytes
Matt Harbison <matt_harbison@yahoo.com>
parents: 41450
diff changeset
72 return (pycompat.strurl(user), pycompat.strurl(passwd))
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
73
15005
4a43e23b8c55 hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents: 14244
diff changeset
74 if not user or not passwd:
15025
0593e8f81c71 http: pass user to readauthforuri() (fix 4a43e23b8c55)
Patrick Mezard <pmezard@gmail.com>
parents: 15005
diff changeset
75 res = httpconnectionmod.readauthforuri(self.ui, authuri, user)
13372
5bced0d28a39 url: return the matched authentication group name from readauthforuri()
Steve Borho <steve@borho.org>
parents: 13371
diff changeset
76 if res:
5bced0d28a39 url: return the matched authentication group name from readauthforuri()
Steve Borho <steve@borho.org>
parents: 13371
diff changeset
77 group, auth = res
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
78 user, passwd = auth.get(b'username'), auth.get(b'password')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
79 self.ui.debug(b"using auth.%s.* for authentication\n" % group)
8333
89c80c3dc584 allow http authentication information to be specified in the configuration
Sune Foldager <cryo@cyanite.org>
parents: 8225
diff changeset
80 if not user or not passwd:
46907
ffd3e823a7e5 urlutil: extract `url` related code from `util` into the new module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46819
diff changeset
81 u = urlutil.url(pycompat.bytesurl(authuri))
20291
7d589d923b8a url: added authuri when login information is requested (issue3209)
Lucas Moscovicz <lmoscovicz@fb.com>
parents: 20174
diff changeset
82 u.query = None
8333
89c80c3dc584 allow http authentication information to be specified in the configuration
Sune Foldager <cryo@cyanite.org>
parents: 8225
diff changeset
83 if not self.ui.interactive():
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
84 raise error.Abort(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
85 _(b'http authorization required for %s')
46907
ffd3e823a7e5 urlutil: extract `url` related code from `util` into the new module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46819
diff changeset
86 % urlutil.hidepassword(bytes(u))
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
87 )
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
88
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
89 self.ui.write(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
90 _(b"http authorization required for %s\n")
46907
ffd3e823a7e5 urlutil: extract `url` related code from `util` into the new module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46819
diff changeset
91 % urlutil.hidepassword(bytes(u))
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
92 )
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
93 self.ui.write(_(b"realm: %s\n") % pycompat.bytesurl(realm))
8333
89c80c3dc584 allow http authentication information to be specified in the configuration
Sune Foldager <cryo@cyanite.org>
parents: 8225
diff changeset
94 if user:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
95 self.ui.write(_(b"user: %s\n") % user)
8333
89c80c3dc584 allow http authentication information to be specified in the configuration
Sune Foldager <cryo@cyanite.org>
parents: 8225
diff changeset
96 else:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
97 user = self.ui.prompt(_(b"user:"), default=None)
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
98
8333
89c80c3dc584 allow http authentication information to be specified in the configuration
Sune Foldager <cryo@cyanite.org>
parents: 8225
diff changeset
99 if not passwd:
89c80c3dc584 allow http authentication information to be specified in the configuration
Sune Foldager <cryo@cyanite.org>
parents: 8225
diff changeset
100 passwd = self.ui.getpass()
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
101
45759
ff48eea4a926 url: do not continue HTTP authentication with user=None (issue6425)
Yuya Nishihara <yuya@tcha.org>
parents: 44783
diff changeset
102 # As of Python 3.8, the default implementation of
ff48eea4a926 url: do not continue HTTP authentication with user=None (issue6425)
Yuya Nishihara <yuya@tcha.org>
parents: 44783
diff changeset
103 # AbstractBasicAuthHandler.retry_http_basic_auth() assumes the user
ff48eea4a926 url: do not continue HTTP authentication with user=None (issue6425)
Yuya Nishihara <yuya@tcha.org>
parents: 44783
diff changeset
104 # is set if pw is not None. This means (None, str) is not a valid
ff48eea4a926 url: do not continue HTTP authentication with user=None (issue6425)
Yuya Nishihara <yuya@tcha.org>
parents: 44783
diff changeset
105 # return type of find_user_password().
ff48eea4a926 url: do not continue HTTP authentication with user=None (issue6425)
Yuya Nishihara <yuya@tcha.org>
parents: 44783
diff changeset
106 if user is None:
ff48eea4a926 url: do not continue HTTP authentication with user=None (issue6425)
Yuya Nishihara <yuya@tcha.org>
parents: 44783
diff changeset
107 return None, None
ff48eea4a926 url: do not continue HTTP authentication with user=None (issue6425)
Yuya Nishihara <yuya@tcha.org>
parents: 44783
diff changeset
108
29377
2c019aac6b99 url: extract password database from password manager
liscju <piotr.listkiewicz@gmail.com>
parents: 29252
diff changeset
109 self.passwddb.add_password(realm, authuri, user, passwd)
8333
89c80c3dc584 allow http authentication information to be specified in the configuration
Sune Foldager <cryo@cyanite.org>
parents: 8225
diff changeset
110 self._writedebug(user, passwd)
41590
349c8879becd py3: ensure the HTTP password manager returns strings, not bytes
Matt Harbison <matt_harbison@yahoo.com>
parents: 41450
diff changeset
111 return (pycompat.strurl(user), pycompat.strurl(passwd))
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
112
8333
89c80c3dc584 allow http authentication information to be specified in the configuration
Sune Foldager <cryo@cyanite.org>
parents: 8225
diff changeset
113 def _writedebug(self, user, passwd):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
114 msg = _(b'http auth: user %s, password %s\n')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
115 self.ui.debug(msg % (user, passwd and b'*' * len(passwd) or b'not set'))
8333
89c80c3dc584 allow http authentication information to be specified in the configuration
Sune Foldager <cryo@cyanite.org>
parents: 8225
diff changeset
116
15025
0593e8f81c71 http: pass user to readauthforuri() (fix 4a43e23b8c55)
Patrick Mezard <pmezard@gmail.com>
parents: 15005
diff changeset
117 def find_stored_password(self, authuri):
29377
2c019aac6b99 url: extract password database from password manager
liscju <piotr.listkiewicz@gmail.com>
parents: 29252
diff changeset
118 return self.passwddb.find_user_password(None, authuri)
15025
0593e8f81c71 http: pass user to readauthforuri() (fix 4a43e23b8c55)
Patrick Mezard <pmezard@gmail.com>
parents: 15005
diff changeset
119
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
120
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28861
diff changeset
121 class proxyhandler(urlreq.proxyhandler):
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
122 def __init__(self, ui):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
123 proxyurl = ui.config(b"http_proxy", b"host") or encoding.environ.get(
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
124 b'http_proxy'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
125 )
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
126 # XXX proxyauthinfo = None
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
127
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
128 if proxyurl:
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
129 # proxy can be proper url or host[:port]
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
130 if not (
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
131 proxyurl.startswith(b'http:') or proxyurl.startswith(b'https:')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
132 ):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
133 proxyurl = b'http://' + proxyurl + b'/'
46907
ffd3e823a7e5 urlutil: extract `url` related code from `util` into the new module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46819
diff changeset
134 proxy = urlutil.url(proxyurl)
13820
65b89e80f892 url: use url.url in proxyhandler
Brodie Rao <brodie@bitheap.org>
parents: 13819
diff changeset
135 if not proxy.user:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
136 proxy.user = ui.config(b"http_proxy", b"user")
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
137 proxy.passwd = ui.config(b"http_proxy", b"passwd")
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
138
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
139 # see if we should use a proxy for this url
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
140 no_list = [b"localhost", b"127.0.0.1"]
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
141 no_list.extend(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
142 [p.lower() for p in ui.configlist(b"http_proxy", b"no")]
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
143 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
144 no_list.extend(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
145 [
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
146 p.strip().lower()
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
147 for p in encoding.environ.get(b"no_proxy", b'').split(b',')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
148 if p.strip()
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
149 ]
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
150 )
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
151 # "http_proxy.always" config is for running tests on localhost
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
152 if ui.configbool(b"http_proxy", b"always"):
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
153 self.no_list = []
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
154 else:
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
155 self.no_list = no_list
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
156
41710
4028897dfa05 url: always use str for proxy configuration
Gregory Szorc <gregory.szorc@gmail.com>
parents: 41708
diff changeset
157 # Keys and values need to be str because the standard library
4028897dfa05 url: always use str for proxy configuration
Gregory Szorc <gregory.szorc@gmail.com>
parents: 41708
diff changeset
158 # expects them to be.
4028897dfa05 url: always use str for proxy configuration
Gregory Szorc <gregory.szorc@gmail.com>
parents: 41708
diff changeset
159 proxyurl = str(proxy)
43506
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43106
diff changeset
160 proxies = {'http': proxyurl, 'https': proxyurl}
46907
ffd3e823a7e5 urlutil: extract `url` related code from `util` into the new module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46819
diff changeset
161 ui.debug(
ffd3e823a7e5 urlutil: extract `url` related code from `util` into the new module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46819
diff changeset
162 b'proxying through %s\n' % urlutil.hidepassword(bytes(proxy))
ffd3e823a7e5 urlutil: extract `url` related code from `util` into the new module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46819
diff changeset
163 )
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
164 else:
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
165 proxies = {}
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
166
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28861
diff changeset
167 urlreq.proxyhandler.__init__(self, proxies)
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
168 self.ui = ui
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
169
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
170 def proxy_open(self, req, proxy, type_):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
171 host = pycompat.bytesurl(urllibcompat.gethost(req)).split(b':')[0]
19535
df2155ebf502 proxy: allow wildcards in the no proxy list (issue1821)
Matt Mackall <mpm@selenic.com>
parents: 18176
diff changeset
172 for e in self.no_list:
df2155ebf502 proxy: allow wildcards in the no proxy list (issue1821)
Matt Mackall <mpm@selenic.com>
parents: 18176
diff changeset
173 if host == e:
df2155ebf502 proxy: allow wildcards in the no proxy list (issue1821)
Matt Mackall <mpm@selenic.com>
parents: 18176
diff changeset
174 return None
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
175 if e.startswith(b'*.') and host.endswith(e[2:]):
19535
df2155ebf502 proxy: allow wildcards in the no proxy list (issue1821)
Matt Mackall <mpm@selenic.com>
parents: 18176
diff changeset
176 return None
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
177 if e.startswith(b'.') and host.endswith(e[1:]):
19535
df2155ebf502 proxy: allow wildcards in the no proxy list (issue1821)
Matt Mackall <mpm@selenic.com>
parents: 18176
diff changeset
178 return None
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
179
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28861
diff changeset
180 return urlreq.proxyhandler.proxy_open(self, req, proxy, type_)
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
181
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
182
13420
051f498628f7 url: refactor _gen_sendfile
Mads Kiilerich <mads@kiilerich.com>
parents: 13419
diff changeset
183 def _gen_sendfile(orgsend):
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
184 def _sendfile(self, data):
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
185 # send a file
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents: 14204
diff changeset
186 if isinstance(data, httpconnectionmod.httpsendfile):
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
187 # if auth required, some data sent twice, so rewind here
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
188 data.seek(0)
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
189 for chunk in util.filechunkiter(data):
13420
051f498628f7 url: refactor _gen_sendfile
Mads Kiilerich <mads@kiilerich.com>
parents: 13419
diff changeset
190 orgsend(self, chunk)
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
191 else:
13420
051f498628f7 url: refactor _gen_sendfile
Mads Kiilerich <mads@kiilerich.com>
parents: 13419
diff changeset
192 orgsend(self, data)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
193
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
194 return _sendfile
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
195
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
196
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
197 has_https = util.safehasattr(urlreq, b'httpshandler')
10409
4c94a3df4b10 url: SSL server certificate verification using web.cacerts file (issue1174)
Henrik Stuart <hg@hstuart.dk>
parents: 10408
diff changeset
198
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
199
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
200 class httpconnection(keepalive.HTTPConnection):
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
201 # must be able to send big bundle as stream.
13420
051f498628f7 url: refactor _gen_sendfile
Mads Kiilerich <mads@kiilerich.com>
parents: 13419
diff changeset
202 send = _gen_sendfile(keepalive.HTTPConnection.send)
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
203
8590
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967)
Henrik Stuart <hg@hstuart.dk>
parents: 8344
diff changeset
204 def getresponse(self):
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967)
Henrik Stuart <hg@hstuart.dk>
parents: 8344
diff changeset
205 proxyres = getattr(self, 'proxyres', None)
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967)
Henrik Stuart <hg@hstuart.dk>
parents: 8344
diff changeset
206 if proxyres:
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967)
Henrik Stuart <hg@hstuart.dk>
parents: 8344
diff changeset
207 if proxyres.will_close:
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967)
Henrik Stuart <hg@hstuart.dk>
parents: 8344
diff changeset
208 self.close()
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967)
Henrik Stuart <hg@hstuart.dk>
parents: 8344
diff changeset
209 self.proxyres = None
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967)
Henrik Stuart <hg@hstuart.dk>
parents: 8344
diff changeset
210 return proxyres
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967)
Henrik Stuart <hg@hstuart.dk>
parents: 8344
diff changeset
211 return keepalive.HTTPConnection.getresponse(self)
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967)
Henrik Stuart <hg@hstuart.dk>
parents: 8344
diff changeset
212
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
213
41708
d20f1594ff4a url: always access req._tunnel_host
Gregory Szorc <gregory.szorc@gmail.com>
parents: 41590
diff changeset
214 # Large parts of this function have their origin from before Python 2.6
d20f1594ff4a url: always access req._tunnel_host
Gregory Szorc <gregory.szorc@gmail.com>
parents: 41590
diff changeset
215 # and could potentially be removed.
9852
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
216 def _generic_start_transaction(handler, h, req):
41708
d20f1594ff4a url: always access req._tunnel_host
Gregory Szorc <gregory.szorc@gmail.com>
parents: 41590
diff changeset
217 tunnel_host = req._tunnel_host
14964
376c32a5ccdc url: replace uses of hasattr with safehasattr or getattr
Augie Fackler <durin42@gmail.com>
parents: 14244
diff changeset
218 if tunnel_host:
43506
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43106
diff changeset
219 if tunnel_host[:7] not in ['http://', 'https:/']:
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43106
diff changeset
220 tunnel_host = 'https://' + tunnel_host
9852
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
221 new_tunnel = True
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
222 else:
34466
1232f7fa00c3 cleanup: use urllibcompat for renamed methods on urllib request objects
Augie Fackler <augie@google.com>
parents: 34428
diff changeset
223 tunnel_host = urllibcompat.getselector(req)
9852
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
224 new_tunnel = False
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
225
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
226 if new_tunnel or tunnel_host == urllibcompat.getfullurl(req): # has proxy
46907
ffd3e823a7e5 urlutil: extract `url` related code from `util` into the new module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46819
diff changeset
227 u = urlutil.url(pycompat.bytesurl(tunnel_host))
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
228 if new_tunnel or u.scheme == b'https': # only use CONNECT for HTTPS
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
229 h.realhostport = b':'.join([u.host, (u.port or b'443')])
9852
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
230 h.headers = req.headers.copy()
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
231 h.headers.update(handler.parent.addheaders)
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
232 return
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
233
10415
677f15da38c1 url: proxy handling, simplify and correctly deal with IPv6
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 10411
diff changeset
234 h.realhostport = None
9852
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
235 h.headers = None
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
236
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
237
9852
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
238 def _generic_proxytunnel(self):
44452
9d2b2df2c2ba cleanup: run pyupgrade on our source tree to clean up varying things
Augie Fackler <augie@google.com>
parents: 43506
diff changeset
239 proxyheaders = {
44783
a50f33f1ff24 url: fix a bytes vs str crash in processing proxy headers (issue6249)
Matt Harbison <matt_harbison@yahoo.com>
parents: 44641
diff changeset
240 pycompat.bytestr(x): pycompat.bytestr(self.headers[x])
44452
9d2b2df2c2ba cleanup: run pyupgrade on our source tree to clean up varying things
Augie Fackler <augie@google.com>
parents: 43506
diff changeset
241 for x in self.headers
9d2b2df2c2ba cleanup: run pyupgrade on our source tree to clean up varying things
Augie Fackler <augie@google.com>
parents: 43506
diff changeset
242 if x.lower().startswith('proxy-')
9d2b2df2c2ba cleanup: run pyupgrade on our source tree to clean up varying things
Augie Fackler <augie@google.com>
parents: 43506
diff changeset
243 }
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
244 self.send(b'CONNECT %s HTTP/1.0\r\n' % self.realhostport)
43106
d783f945a701 py3: finish porting iteritems() to pycompat and remove source transformer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43089
diff changeset
245 for header in pycompat.iteritems(proxyheaders):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
246 self.send(b'%s: %s\r\n' % header)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
247 self.send(b'\r\n')
9852
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
248
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
249 # majority of the following code is duplicated from
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
250 # httplib.HTTPConnection as there are no adequate places to
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
251 # override functions to provide the needed functionality
41711
544035846830 url: don't pass strict argument on Python 3
Gregory Szorc <gregory.szorc@gmail.com>
parents: 41710
diff changeset
252 # strict was removed in Python 3.4.
544035846830 url: don't pass strict argument on Python 3
Gregory Szorc <gregory.szorc@gmail.com>
parents: 41710
diff changeset
253 kwargs = {}
544035846830 url: don't pass strict argument on Python 3
Gregory Szorc <gregory.szorc@gmail.com>
parents: 41710
diff changeset
254 if not pycompat.ispy3:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
255 kwargs[b'strict'] = self.strict
41711
544035846830 url: don't pass strict argument on Python 3
Gregory Szorc <gregory.szorc@gmail.com>
parents: 41710
diff changeset
256
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
257 res = self.response_class(self.sock, method=self._method, **kwargs)
9852
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
258
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
259 while True:
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
260 version, status, reason = res._read_status()
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
261 if status != httplib.CONTINUE:
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
262 break
29729
44ea12756fef url: use `iter(callable, sentinel)` instead of while True
Augie Fackler <augie@google.com>
parents: 29662
diff changeset
263 # skip lines that are all whitespace
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
264 list(iter(lambda: res.fp.readline().strip(), b''))
9852
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
265 res.status = status
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
266 res.reason = reason.strip()
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
267
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
268 if res.status == 200:
29729
44ea12756fef url: use `iter(callable, sentinel)` instead of while True
Augie Fackler <augie@google.com>
parents: 29662
diff changeset
269 # skip lines until we find a blank line
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
270 list(iter(res.fp.readline, b'\r\n'))
9852
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
271 return True
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
272
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
273 if version == b'HTTP/1.0':
9852
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
274 res.version = 10
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
275 elif version.startswith(b'HTTP/1.'):
9852
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
276 res.version = 11
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
277 elif version == b'HTTP/0.9':
9852
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
278 res.version = 9
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
279 else:
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
280 raise httplib.UnknownProtocol(version)
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
281
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
282 if res.version == 9:
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
283 res.length = None
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
284 res.chunked = 0
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
285 res.will_close = 1
28861
86db5cb55d46 pycompat: switch to util.stringio for py3 compat
timeless <timeless@mozdev.org>
parents: 26806
diff changeset
286 res.msg = httplib.HTTPMessage(stringio())
9852
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
287 return False
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
288
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
289 res.msg = httplib.HTTPMessage(res.fp)
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
290 res.msg.fp = None
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
291
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
292 # are we using the chunked-style of transfer encoding?
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
293 trenc = res.msg.getheader(b'transfer-encoding')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
294 if trenc and trenc.lower() == b"chunked":
9852
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
295 res.chunked = 1
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
296 res.chunk_left = None
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
297 else:
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
298 res.chunked = 0
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
299
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
300 # will the connection close at the end of the response?
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
301 res.will_close = res._check_close()
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
302
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
303 # do we have a Content-Length?
17428
72803c8edaa4 avoid using abbreviations that look like spelling errors
Mads Kiilerich <mads@kiilerich.com>
parents: 17424
diff changeset
304 # NOTE: RFC 2616, section 4.4, #3 says we ignore this if
72803c8edaa4 avoid using abbreviations that look like spelling errors
Mads Kiilerich <mads@kiilerich.com>
parents: 17424
diff changeset
305 # transfer-encoding is "chunked"
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
306 length = res.msg.getheader(b'content-length')
9852
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
307 if length and not res.chunked:
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
308 try:
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
309 res.length = int(length)
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
310 except ValueError:
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
311 res.length = None
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
312 else:
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
313 if res.length < 0: # ignore nonsensical negative lengths
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
314 res.length = None
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
315 else:
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
316 res.length = None
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
317
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
318 # does the body have a fixed length? (of zero)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
319 if (
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
320 status == httplib.NO_CONTENT
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
321 or status == httplib.NOT_MODIFIED
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
322 or 100 <= status < 200
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
323 or res._method == b'HEAD' # 1xx codes
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
324 ):
9852
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
325 res.length = 0
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
326
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
327 # if the connection remains open, and we aren't using chunked, and
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
328 # a content-length was not provided, then assume that the connection
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
329 # WILL close.
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
330 if not res.will_close and not res.chunked and res.length is None:
9852
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
331 res.will_close = 1
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
332
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
333 self.proxyres = res
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
334
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
335 return False
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
336
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
337
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
338 class httphandler(keepalive.HTTPHandler):
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
339 def http_open(self, req):
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
340 return self.do_open(httpconnection, req)
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
341
8590
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967)
Henrik Stuart <hg@hstuart.dk>
parents: 8344
diff changeset
342 def _start_transaction(self, h, req):
9852
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
343 _generic_start_transaction(self, h, req)
8590
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967)
Henrik Stuart <hg@hstuart.dk>
parents: 8344
diff changeset
344 return keepalive.HTTPHandler._start_transaction(self, h, req)
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967)
Henrik Stuart <hg@hstuart.dk>
parents: 8344
diff changeset
345
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
346
37011
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
347 class logginghttpconnection(keepalive.HTTPConnection):
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
348 def __init__(self, createconn, *args, **kwargs):
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
349 keepalive.HTTPConnection.__init__(self, *args, **kwargs)
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
350 self._create_connection = createconn
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
351
37100
9e6d3465f17e url: make logginghttphandler compatible with Python 2.7.6
Yuya Nishihara <yuya@tcha.org>
parents: 37084
diff changeset
352 if sys.version_info < (2, 7, 7):
9e6d3465f17e url: make logginghttphandler compatible with Python 2.7.6
Yuya Nishihara <yuya@tcha.org>
parents: 37084
diff changeset
353 # copied from 2.7.14, since old implementations directly call
9e6d3465f17e url: make logginghttphandler compatible with Python 2.7.6
Yuya Nishihara <yuya@tcha.org>
parents: 37084
diff changeset
354 # socket.create_connection()
9e6d3465f17e url: make logginghttphandler compatible with Python 2.7.6
Yuya Nishihara <yuya@tcha.org>
parents: 37084
diff changeset
355 def connect(self):
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
356 self.sock = self._create_connection(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
357 (self.host, self.port), self.timeout, self.source_address
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
358 )
37100
9e6d3465f17e url: make logginghttphandler compatible with Python 2.7.6
Yuya Nishihara <yuya@tcha.org>
parents: 37084
diff changeset
359 if self._tunnel_host:
9e6d3465f17e url: make logginghttphandler compatible with Python 2.7.6
Yuya Nishihara <yuya@tcha.org>
parents: 37084
diff changeset
360 self._tunnel()
9e6d3465f17e url: make logginghttphandler compatible with Python 2.7.6
Yuya Nishihara <yuya@tcha.org>
parents: 37084
diff changeset
361
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
362
37011
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
363 class logginghttphandler(httphandler):
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
364 """HTTP handler that logs socket I/O."""
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
365
40043
6509fcec830c url: allow to configure timeout on http connection
Cédric Krier <ced@b2ck.com>
parents: 40034
diff changeset
366 def __init__(self, logfh, name, observeropts, timeout=None):
6509fcec830c url: allow to configure timeout on http connection
Cédric Krier <ced@b2ck.com>
parents: 40034
diff changeset
367 super(logginghttphandler, self).__init__(timeout=timeout)
37011
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
368
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
369 self._logfh = logfh
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
370 self._logname = name
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
371 self._observeropts = observeropts
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
372
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
373 # do_open() calls the passed class to instantiate an HTTPConnection. We
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
374 # pass in a callable method that creates a custom HTTPConnection instance
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
375 # whose callback to create the socket knows how to proxy the socket.
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
376 def http_open(self, req):
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
377 return self.do_open(self._makeconnection, req)
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
378
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
379 def _makeconnection(self, *args, **kwargs):
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
380 def createconnection(*args, **kwargs):
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
381 sock = socket.create_connection(*args, **kwargs)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
382 return util.makeloggingsocket(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
383 self._logfh, sock, self._logname, **self._observeropts
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
384 )
37011
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
385
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
386 return logginghttpconnection(createconnection, *args, **kwargs)
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
387
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
388
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
389 if has_https:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
390
40031
f2dffa1359c6 url: have httpsconnection inherit from our custom HTTPConnection
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40018
diff changeset
391 class httpsconnection(keepalive.HTTPConnection):
13424
08f9c587141f url: merge BetterHTTPS with httpsconnection to get some proxy https validation
Mads Kiilerich <mads@kiilerich.com>
parents: 13422
diff changeset
392 response_class = keepalive.HTTPResponse
25414
f7ccbc2776b7 https: do not inherit httplib.HTTPSConnection that creates unused SSLContext
Yuya Nishihara <yuya@tcha.org>
parents: 25207
diff changeset
393 default_port = httplib.HTTPS_PORT
13424
08f9c587141f url: merge BetterHTTPS with httpsconnection to get some proxy https validation
Mads Kiilerich <mads@kiilerich.com>
parents: 13422
diff changeset
394 # must be able to send big bundle as stream.
08f9c587141f url: merge BetterHTTPS with httpsconnection to get some proxy https validation
Mads Kiilerich <mads@kiilerich.com>
parents: 13422
diff changeset
395 send = _gen_sendfile(keepalive.safesend)
25414
f7ccbc2776b7 https: do not inherit httplib.HTTPSConnection that creates unused SSLContext
Yuya Nishihara <yuya@tcha.org>
parents: 25207
diff changeset
396 getresponse = keepalive.wrapgetresponse(httplib.HTTPConnection)
f7ccbc2776b7 https: do not inherit httplib.HTTPSConnection that creates unused SSLContext
Yuya Nishihara <yuya@tcha.org>
parents: 25207
diff changeset
397
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
398 def __init__(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
399 self,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
400 host,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
401 port=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
402 key_file=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
403 cert_file=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
404 *args,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
405 **kwargs
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
406 ):
40031
f2dffa1359c6 url: have httpsconnection inherit from our custom HTTPConnection
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40018
diff changeset
407 keepalive.HTTPConnection.__init__(self, host, port, *args, **kwargs)
25414
f7ccbc2776b7 https: do not inherit httplib.HTTPSConnection that creates unused SSLContext
Yuya Nishihara <yuya@tcha.org>
parents: 25207
diff changeset
408 self.key_file = key_file
f7ccbc2776b7 https: do not inherit httplib.HTTPSConnection that creates unused SSLContext
Yuya Nishihara <yuya@tcha.org>
parents: 25207
diff changeset
409 self.cert_file = cert_file
9726
430e59ff3437 keepalive: handle broken pipes gracefully during large POSTs
Augie Fackler <durin42@gmail.com>
parents: 9467
diff changeset
410
10409
4c94a3df4b10 url: SSL server certificate verification using web.cacerts file (issue1174)
Henrik Stuart <hg@hstuart.dk>
parents: 10408
diff changeset
411 def connect(self):
29662
b76ea1384bf2 url: drop compatibility wrapper of socket.create_connection()
Yuya Nishihara <yuya@tcha.org>
parents: 29639
diff changeset
412 self.sock = socket.create_connection((self.host, self.port))
13422
ebce5196b9db url: always create BetterHTTPS connections the same way
Mads Kiilerich <mads@kiilerich.com>
parents: 13421
diff changeset
413
13421
bd8bfa85d5a5 url: refactor BetterHTTPS.connect
Mads Kiilerich <mads@kiilerich.com>
parents: 13420
diff changeset
414 host = self.host
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
415 if self.realhostport: # use CONNECT proxy
14064
e4bfb9c337f3 remove unused imports and variables
Alexander Solovyov <alexander@solovyov.net>
parents: 13902
diff changeset
416 _generic_proxytunnel(self)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
417 host = self.realhostport.rsplit(b':', 1)[0]
25429
9d1c61715939 ssl: rename ssl_wrap_socket() to conform to our naming convention
Yuya Nishihara <yuya@tcha.org>
parents: 25415
diff changeset
418 self.sock = sslutil.wrapsocket(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
419 self.sock,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
420 self.key_file,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
421 self.cert_file,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
422 ui=self.ui,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
423 serverhostname=host,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
424 )
29227
dffe78d80a6c sslutil: convert socket validation from a class to a function (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28883
diff changeset
425 sslutil.validatesocket(self.sock)
10409
4c94a3df4b10 url: SSL server certificate verification using web.cacerts file (issue1174)
Henrik Stuart <hg@hstuart.dk>
parents: 10408
diff changeset
426
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28861
diff changeset
427 class httpshandler(keepalive.KeepAliveHandler, urlreq.httpshandler):
40043
6509fcec830c url: allow to configure timeout on http connection
Cédric Krier <ced@b2ck.com>
parents: 40034
diff changeset
428 def __init__(self, ui, timeout=None):
6509fcec830c url: allow to configure timeout on http connection
Cédric Krier <ced@b2ck.com>
parents: 40034
diff changeset
429 keepalive.KeepAliveHandler.__init__(self, timeout=timeout)
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28861
diff changeset
430 urlreq.httpshandler.__init__(self)
8847
7951f385fcb7 url: support client certificate files over HTTPS (issue643)
Henrik Stuart <hg@hstuart.dk>
parents: 8590
diff changeset
431 self.ui = ui
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
432 self.pwmgr = passwordmgr(self.ui, self.ui.httppasswordmgrdb)
8847
7951f385fcb7 url: support client certificate files over HTTPS (issue643)
Henrik Stuart <hg@hstuart.dk>
parents: 8590
diff changeset
433
9852
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
434 def _start_transaction(self, h, req):
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
435 _generic_start_transaction(self, h, req)
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
436 return keepalive.KeepAliveHandler._start_transaction(self, h, req)
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
437
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
438 def https_open(self, req):
34466
1232f7fa00c3 cleanup: use urllibcompat for renamed methods on urllib request objects
Augie Fackler <augie@google.com>
parents: 34428
diff changeset
439 # urllibcompat.getfullurl() does not contain credentials
1232f7fa00c3 cleanup: use urllibcompat for renamed methods on urllib request objects
Augie Fackler <augie@google.com>
parents: 34428
diff changeset
440 # and we may need them to match the certificates.
1232f7fa00c3 cleanup: use urllibcompat for renamed methods on urllib request objects
Augie Fackler <augie@google.com>
parents: 34428
diff changeset
441 url = urllibcompat.getfullurl(req)
15025
0593e8f81c71 http: pass user to readauthforuri() (fix 4a43e23b8c55)
Patrick Mezard <pmezard@gmail.com>
parents: 15005
diff changeset
442 user, password = self.pwmgr.find_stored_password(url)
0593e8f81c71 http: pass user to readauthforuri() (fix 4a43e23b8c55)
Patrick Mezard <pmezard@gmail.com>
parents: 15005
diff changeset
443 res = httpconnectionmod.readauthforuri(self.ui, url, user)
13372
5bced0d28a39 url: return the matched authentication group name from readauthforuri()
Steve Borho <steve@borho.org>
parents: 13371
diff changeset
444 if res:
5bced0d28a39 url: return the matched authentication group name from readauthforuri()
Steve Borho <steve@borho.org>
parents: 13371
diff changeset
445 group, auth = res
5bced0d28a39 url: return the matched authentication group name from readauthforuri()
Steve Borho <steve@borho.org>
parents: 13371
diff changeset
446 self.auth = auth
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
447 self.ui.debug(b"using auth.%s.* for authentication\n" % group)
13372
5bced0d28a39 url: return the matched authentication group name from readauthforuri()
Steve Borho <steve@borho.org>
parents: 13371
diff changeset
448 else:
5bced0d28a39 url: return the matched authentication group name from readauthforuri()
Steve Borho <steve@borho.org>
parents: 13371
diff changeset
449 self.auth = None
8847
7951f385fcb7 url: support client certificate files over HTTPS (issue643)
Henrik Stuart <hg@hstuart.dk>
parents: 8590
diff changeset
450 return self.do_open(self._makeconnection, req)
7951f385fcb7 url: support client certificate files over HTTPS (issue643)
Henrik Stuart <hg@hstuart.dk>
parents: 8590
diff changeset
451
10408
50fb1fe143ff url: httplib.HTTPSConnection already handles IPv6 and port parsing fine
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 10282
diff changeset
452 def _makeconnection(self, host, port=None, *args, **kwargs):
8847
7951f385fcb7 url: support client certificate files over HTTPS (issue643)
Henrik Stuart <hg@hstuart.dk>
parents: 8590
diff changeset
453 keyfile = None
7951f385fcb7 url: support client certificate files over HTTPS (issue643)
Henrik Stuart <hg@hstuart.dk>
parents: 8590
diff changeset
454 certfile = None
7951f385fcb7 url: support client certificate files over HTTPS (issue643)
Henrik Stuart <hg@hstuart.dk>
parents: 8590
diff changeset
455
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
456 if len(args) >= 1: # key_file
10511
6f61c480f51c url: *args argument is a tuple, not a list (found by pylint)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 10484
diff changeset
457 keyfile = args[0]
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
458 if len(args) >= 2: # cert_file
10511
6f61c480f51c url: *args argument is a tuple, not a list (found by pylint)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 10484
diff changeset
459 certfile = args[1]
6f61c480f51c url: *args argument is a tuple, not a list (found by pylint)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 10484
diff changeset
460 args = args[2:]
8847
7951f385fcb7 url: support client certificate files over HTTPS (issue643)
Henrik Stuart <hg@hstuart.dk>
parents: 8590
diff changeset
461
7951f385fcb7 url: support client certificate files over HTTPS (issue643)
Henrik Stuart <hg@hstuart.dk>
parents: 8590
diff changeset
462 # if the user has specified different key/cert files in
7951f385fcb7 url: support client certificate files over HTTPS (issue643)
Henrik Stuart <hg@hstuart.dk>
parents: 8590
diff changeset
463 # hgrc, we prefer these
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
464 if self.auth and b'key' in self.auth and b'cert' in self.auth:
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
465 keyfile = self.auth[b'key']
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
466 certfile = self.auth[b'cert']
8847
7951f385fcb7 url: support client certificate files over HTTPS (issue643)
Henrik Stuart <hg@hstuart.dk>
parents: 8590
diff changeset
467
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
468 conn = httpsconnection(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
469 host, port, keyfile, certfile, *args, **kwargs
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
470 )
10409
4c94a3df4b10 url: SSL server certificate verification using web.cacerts file (issue1174)
Henrik Stuart <hg@hstuart.dk>
parents: 10408
diff changeset
471 conn.ui = self.ui
4c94a3df4b10 url: SSL server certificate verification using web.cacerts file (issue1174)
Henrik Stuart <hg@hstuart.dk>
parents: 10408
diff changeset
472 return conn
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
473
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
474
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28861
diff changeset
475 class httpdigestauthhandler(urlreq.httpdigestauthhandler):
11457
2ec346160783 http digest auth: reset redirect counter on new requests (issue2255)
Mads Kiilerich <mads@kiilerich.com>
parents: 11415
diff changeset
476 def __init__(self, *args, **kwargs):
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28861
diff changeset
477 urlreq.httpdigestauthhandler.__init__(self, *args, **kwargs)
11457
2ec346160783 http digest auth: reset redirect counter on new requests (issue2255)
Mads Kiilerich <mads@kiilerich.com>
parents: 11415
diff changeset
478 self.retried_req = None
2ec346160783 http digest auth: reset redirect counter on new requests (issue2255)
Mads Kiilerich <mads@kiilerich.com>
parents: 11415
diff changeset
479
2ec346160783 http digest auth: reset redirect counter on new requests (issue2255)
Mads Kiilerich <mads@kiilerich.com>
parents: 11415
diff changeset
480 def reset_retry_count(self):
2ec346160783 http digest auth: reset redirect counter on new requests (issue2255)
Mads Kiilerich <mads@kiilerich.com>
parents: 11415
diff changeset
481 # Python 2.6.5 will call this on 401 or 407 errors and thus loop
2ec346160783 http digest auth: reset redirect counter on new requests (issue2255)
Mads Kiilerich <mads@kiilerich.com>
parents: 11415
diff changeset
482 # forever. We disable reset_retry_count completely and reset in
2ec346160783 http digest auth: reset redirect counter on new requests (issue2255)
Mads Kiilerich <mads@kiilerich.com>
parents: 11415
diff changeset
483 # http_error_auth_reqed instead.
2ec346160783 http digest auth: reset redirect counter on new requests (issue2255)
Mads Kiilerich <mads@kiilerich.com>
parents: 11415
diff changeset
484 pass
2ec346160783 http digest auth: reset redirect counter on new requests (issue2255)
Mads Kiilerich <mads@kiilerich.com>
parents: 11415
diff changeset
485
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
486 def http_error_auth_reqed(self, auth_header, host, req, headers):
11457
2ec346160783 http digest auth: reset redirect counter on new requests (issue2255)
Mads Kiilerich <mads@kiilerich.com>
parents: 11415
diff changeset
487 # Reset the retry counter once for each request.
2ec346160783 http digest auth: reset redirect counter on new requests (issue2255)
Mads Kiilerich <mads@kiilerich.com>
parents: 11415
diff changeset
488 if req is not self.retried_req:
2ec346160783 http digest auth: reset redirect counter on new requests (issue2255)
Mads Kiilerich <mads@kiilerich.com>
parents: 11415
diff changeset
489 self.retried_req = req
2ec346160783 http digest auth: reset redirect counter on new requests (issue2255)
Mads Kiilerich <mads@kiilerich.com>
parents: 11415
diff changeset
490 self.retried = 0
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28861
diff changeset
491 return urlreq.httpdigestauthhandler.http_error_auth_reqed(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
492 self, auth_header, host, req, headers
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
493 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
494
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
495
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28861
diff changeset
496 class httpbasicauthhandler(urlreq.httpbasicauthhandler):
11844
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255)
Wagner Bruna <wbruna@softwareexpress.com.br>
parents: 11457
diff changeset
497 def __init__(self, *args, **kwargs):
20964
a939eeb94833 http: reuse authentication info after the first failed request (issue3567)
Stéphane Klein <contact@stephane-klein.info>
parents: 20291
diff changeset
498 self.auth = None
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28861
diff changeset
499 urlreq.httpbasicauthhandler.__init__(self, *args, **kwargs)
11844
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255)
Wagner Bruna <wbruna@softwareexpress.com.br>
parents: 11457
diff changeset
500 self.retried_req = None
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255)
Wagner Bruna <wbruna@softwareexpress.com.br>
parents: 11457
diff changeset
501
20964
a939eeb94833 http: reuse authentication info after the first failed request (issue3567)
Stéphane Klein <contact@stephane-klein.info>
parents: 20291
diff changeset
502 def http_request(self, request):
a939eeb94833 http: reuse authentication info after the first failed request (issue3567)
Stéphane Klein <contact@stephane-klein.info>
parents: 20291
diff changeset
503 if self.auth:
a939eeb94833 http: reuse authentication info after the first failed request (issue3567)
Stéphane Klein <contact@stephane-klein.info>
parents: 20291
diff changeset
504 request.add_unredirected_header(self.auth_header, self.auth)
a939eeb94833 http: reuse authentication info after the first failed request (issue3567)
Stéphane Klein <contact@stephane-klein.info>
parents: 20291
diff changeset
505
a939eeb94833 http: reuse authentication info after the first failed request (issue3567)
Stéphane Klein <contact@stephane-klein.info>
parents: 20291
diff changeset
506 return request
a939eeb94833 http: reuse authentication info after the first failed request (issue3567)
Stéphane Klein <contact@stephane-klein.info>
parents: 20291
diff changeset
507
a939eeb94833 http: reuse authentication info after the first failed request (issue3567)
Stéphane Klein <contact@stephane-klein.info>
parents: 20291
diff changeset
508 def https_request(self, request):
a939eeb94833 http: reuse authentication info after the first failed request (issue3567)
Stéphane Klein <contact@stephane-klein.info>
parents: 20291
diff changeset
509 if self.auth:
a939eeb94833 http: reuse authentication info after the first failed request (issue3567)
Stéphane Klein <contact@stephane-klein.info>
parents: 20291
diff changeset
510 request.add_unredirected_header(self.auth_header, self.auth)
a939eeb94833 http: reuse authentication info after the first failed request (issue3567)
Stéphane Klein <contact@stephane-klein.info>
parents: 20291
diff changeset
511
a939eeb94833 http: reuse authentication info after the first failed request (issue3567)
Stéphane Klein <contact@stephane-klein.info>
parents: 20291
diff changeset
512 return request
a939eeb94833 http: reuse authentication info after the first failed request (issue3567)
Stéphane Klein <contact@stephane-klein.info>
parents: 20291
diff changeset
513
11844
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255)
Wagner Bruna <wbruna@softwareexpress.com.br>
parents: 11457
diff changeset
514 def reset_retry_count(self):
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255)
Wagner Bruna <wbruna@softwareexpress.com.br>
parents: 11457
diff changeset
515 # Python 2.6.5 will call this on 401 or 407 errors and thus loop
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255)
Wagner Bruna <wbruna@softwareexpress.com.br>
parents: 11457
diff changeset
516 # forever. We disable reset_retry_count completely and reset in
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255)
Wagner Bruna <wbruna@softwareexpress.com.br>
parents: 11457
diff changeset
517 # http_error_auth_reqed instead.
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255)
Wagner Bruna <wbruna@softwareexpress.com.br>
parents: 11457
diff changeset
518 pass
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255)
Wagner Bruna <wbruna@softwareexpress.com.br>
parents: 11457
diff changeset
519
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255)
Wagner Bruna <wbruna@softwareexpress.com.br>
parents: 11457
diff changeset
520 def http_error_auth_reqed(self, auth_header, host, req, headers):
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255)
Wagner Bruna <wbruna@softwareexpress.com.br>
parents: 11457
diff changeset
521 # Reset the retry counter once for each request.
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255)
Wagner Bruna <wbruna@softwareexpress.com.br>
parents: 11457
diff changeset
522 if req is not self.retried_req:
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255)
Wagner Bruna <wbruna@softwareexpress.com.br>
parents: 11457
diff changeset
523 self.retried_req = req
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255)
Wagner Bruna <wbruna@softwareexpress.com.br>
parents: 11457
diff changeset
524 self.retried = 0
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28861
diff changeset
525 return urlreq.httpbasicauthhandler.http_error_auth_reqed(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
526 self, auth_header, host, req, headers
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
527 )
11844
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255)
Wagner Bruna <wbruna@softwareexpress.com.br>
parents: 11457
diff changeset
528
20964
a939eeb94833 http: reuse authentication info after the first failed request (issue3567)
Stéphane Klein <contact@stephane-klein.info>
parents: 20291
diff changeset
529 def retry_http_basic_auth(self, host, req, realm):
34466
1232f7fa00c3 cleanup: use urllibcompat for renamed methods on urllib request objects
Augie Fackler <augie@google.com>
parents: 34428
diff changeset
530 user, pw = self.passwd.find_user_password(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
531 realm, urllibcompat.getfullurl(req)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
532 )
20964
a939eeb94833 http: reuse authentication info after the first failed request (issue3567)
Stéphane Klein <contact@stephane-klein.info>
parents: 20291
diff changeset
533 if pw is not None:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
534 raw = b"%s:%s" % (pycompat.bytesurl(user), pycompat.bytesurl(pw))
43506
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43106
diff changeset
535 auth = 'Basic %s' % pycompat.strurl(base64.b64encode(raw).strip())
29639
6fd751fa58d3 url: avoid re-issuing incorrect password (issue3210)
Kim Randell <Kim.Randell@vicon.com>
parents: 29599
diff changeset
536 if req.get_header(self.auth_header, None) == auth:
20964
a939eeb94833 http: reuse authentication info after the first failed request (issue3567)
Stéphane Klein <contact@stephane-klein.info>
parents: 20291
diff changeset
537 return None
a939eeb94833 http: reuse authentication info after the first failed request (issue3567)
Stéphane Klein <contact@stephane-klein.info>
parents: 20291
diff changeset
538 self.auth = auth
a939eeb94833 http: reuse authentication info after the first failed request (issue3567)
Stéphane Klein <contact@stephane-klein.info>
parents: 20291
diff changeset
539 req.add_unredirected_header(self.auth_header, auth)
a939eeb94833 http: reuse authentication info after the first failed request (issue3567)
Stéphane Klein <contact@stephane-klein.info>
parents: 20291
diff changeset
540 return self.parent.open(req)
a939eeb94833 http: reuse authentication info after the first failed request (issue3567)
Stéphane Klein <contact@stephane-klein.info>
parents: 20291
diff changeset
541 else:
a939eeb94833 http: reuse authentication info after the first failed request (issue3567)
Stéphane Klein <contact@stephane-klein.info>
parents: 20291
diff changeset
542 return None
a939eeb94833 http: reuse authentication info after the first failed request (issue3567)
Stéphane Klein <contact@stephane-klein.info>
parents: 20291
diff changeset
543
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
544
31936
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
545 class cookiehandler(urlreq.basehandler):
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
546 def __init__(self, ui):
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
547 self.cookiejar = None
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
548
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
549 cookiefile = ui.config(b'auth', b'cookiefile')
31936
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
550 if not cookiefile:
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
551 return
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
552
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
553 cookiefile = util.expandpath(cookiefile)
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
554 try:
37735
126998dcfb08 url: some bytes/str cleanup where we interface with stdlib funcs
Augie Fackler <augie@google.com>
parents: 37100
diff changeset
555 cookiejar = util.cookielib.MozillaCookieJar(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
556 pycompat.fsdecode(cookiefile)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
557 )
31936
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
558 cookiejar.load()
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
559 self.cookiejar = cookiejar
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
560 except util.cookielib.LoadError as e:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
561 ui.warn(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
562 _(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
563 b'(error loading cookie file %s: %s; continuing without '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
564 b'cookies)\n'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
565 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
566 % (cookiefile, stringutil.forcebytestr(e))
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
567 )
31936
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
568
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
569 def http_request(self, request):
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
570 if self.cookiejar:
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
571 self.cookiejar.add_cookie_header(request)
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
572
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
573 return request
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
574
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
575 def https_request(self, request):
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
576 if self.cookiejar:
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
577 self.cookiejar.add_cookie_header(request)
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
578
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
579 return request
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
580
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
581
9347
d0474b184347 url: add support for custom handlers in extensions
Henrik Stuart <hg@hstuart.dk>
parents: 9122
diff changeset
582 handlerfuncs = []
d0474b184347 url: add support for custom handlers in extensions
Henrik Stuart <hg@hstuart.dk>
parents: 9122
diff changeset
583
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
584
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
585 def opener(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
586 ui,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
587 authinfo=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
588 useragent=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
589 loggingfh=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
590 loggingname=b's',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
591 loggingopts=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
592 sendaccept=True,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
593 ):
45942
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45759
diff changeset
594 """
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
595 construct an opener suitable for urllib2
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
596 authinfo will be added to the password manager
37011
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
597
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
598 The opener can be configured to log socket events if the various
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
599 ``logging*`` arguments are specified.
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
600
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
601 ``loggingfh`` denotes a file object to log events to.
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
602 ``loggingname`` denotes the name of the to print when logging.
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
603 ``loggingopts`` is a dict of keyword arguments to pass to the constructed
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
604 ``util.socketobserver`` instance.
37045
a708e1e4d7a8 url: support suppressing Accept header
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37011
diff changeset
605
a708e1e4d7a8 url: support suppressing Accept header
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37011
diff changeset
606 ``sendaccept`` allows controlling whether the ``Accept`` request header
a708e1e4d7a8 url: support suppressing Accept header
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37011
diff changeset
607 is sent. The header is sent by default.
45942
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45759
diff changeset
608 """
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
609 timeout = ui.configwith(float, b'http', b'timeout')
37011
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
610 handlers = []
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
611
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
612 if loggingfh:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
613 handlers.append(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
614 logginghttphandler(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
615 loggingfh, loggingname, loggingopts or {}, timeout=timeout
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
616 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
617 )
37011
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
618 # We don't yet support HTTPS when logging I/O. If we attempt to open
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
619 # an HTTPS URL, we'll likely fail due to unknown protocol.
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
620
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
621 else:
40043
6509fcec830c url: allow to configure timeout on http connection
Cédric Krier <ced@b2ck.com>
parents: 40034
diff changeset
622 handlers.append(httphandler(timeout=timeout))
37011
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
623 if has_https:
40043
6509fcec830c url: allow to configure timeout on http connection
Cédric Krier <ced@b2ck.com>
parents: 40034
diff changeset
624 handlers.append(httpshandler(ui, timeout=timeout))
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
625
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
626 handlers.append(proxyhandler(ui))
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
627
29378
fea71f66ebff url: remember http password database in ui object
liscju <piotr.listkiewicz@gmail.com>
parents: 29377
diff changeset
628 passmgr = passwordmgr(ui, ui.httppasswordmgrdb)
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
629 if authinfo is not None:
29379
fc777c855d66 largefiles: make cloning not ask two times about password (issue4883)
liscju <piotr.listkiewicz@gmail.com>
parents: 29378
diff changeset
630 realm, uris, user, passwd = authinfo
fc777c855d66 largefiles: make cloning not ask two times about password (issue4883)
liscju <piotr.listkiewicz@gmail.com>
parents: 29378
diff changeset
631 saveduser, savedpass = passmgr.find_stored_password(uris[0])
fc777c855d66 largefiles: make cloning not ask two times about password (issue4883)
liscju <piotr.listkiewicz@gmail.com>
parents: 29378
diff changeset
632 if user != saveduser or passwd:
fc777c855d66 largefiles: make cloning not ask two times about password (issue4883)
liscju <piotr.listkiewicz@gmail.com>
parents: 29378
diff changeset
633 passmgr.add_password(realm, uris, user, passwd)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
634 ui.debug(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
635 b'http auth: user %s, password %s\n'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
636 % (user, passwd and b'*' * len(passwd) or b'not set')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
637 )
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
638
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
639 handlers.extend(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
640 (httpbasicauthhandler(passmgr), httpdigestauthhandler(passmgr))
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
641 )
9347
d0474b184347 url: add support for custom handlers in extensions
Henrik Stuart <hg@hstuart.dk>
parents: 9122
diff changeset
642 handlers.extend([h(ui, passmgr) for h in handlerfuncs])
31936
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
643 handlers.append(cookiehandler(ui))
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28861
diff changeset
644 opener = urlreq.buildopener(*handlers)
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
645
40034
393e44324037 httppeer: report http statistics
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40031
diff changeset
646 # keepalive.py's handlers will populate these attributes if they exist.
393e44324037 httppeer: report http statistics
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40031
diff changeset
647 opener.requestscount = 0
393e44324037 httppeer: report http statistics
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40031
diff changeset
648 opener.sentbytescount = 0
393e44324037 httppeer: report http statistics
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40031
diff changeset
649 opener.receivedbytescount = 0
393e44324037 httppeer: report http statistics
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40031
diff changeset
650
29589
486de14eb394 url: add distribution and version to user-agent request header (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29455
diff changeset
651 # The user agent should should *NOT* be used by servers for e.g.
486de14eb394 url: add distribution and version to user-agent request header (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29455
diff changeset
652 # protocol detection or feature negotiation: there are other
486de14eb394 url: add distribution and version to user-agent request header (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29455
diff changeset
653 # facilities for that.
486de14eb394 url: add distribution and version to user-agent request header (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29455
diff changeset
654 #
486de14eb394 url: add distribution and version to user-agent request header (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29455
diff changeset
655 # "mercurial/proto-1.0" was the original user agent string and
486de14eb394 url: add distribution and version to user-agent request header (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29455
diff changeset
656 # exists for backwards compatibility reasons.
486de14eb394 url: add distribution and version to user-agent request header (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29455
diff changeset
657 #
486de14eb394 url: add distribution and version to user-agent request header (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29455
diff changeset
658 # The "(Mercurial %s)" string contains the distribution
486de14eb394 url: add distribution and version to user-agent request header (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29455
diff changeset
659 # name and version. Other client implementations should choose their
486de14eb394 url: add distribution and version to user-agent request header (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29455
diff changeset
660 # own distribution name. Since servers should not be using the user
486de14eb394 url: add distribution and version to user-agent request header (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29455
diff changeset
661 # agent string for anything, clients should be able to define whatever
486de14eb394 url: add distribution and version to user-agent request header (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29455
diff changeset
662 # user agent they deem appropriate.
35439
e7bb5fc4570c lfs: add git to the User-Agent header for blob transfers
Matt Harbison <matt_harbison@yahoo.com>
parents: 34694
diff changeset
663 #
e7bb5fc4570c lfs: add git to the User-Agent header for blob transfers
Matt Harbison <matt_harbison@yahoo.com>
parents: 34694
diff changeset
664 # The custom user agent is for lfs, because unfortunately some servers
e7bb5fc4570c lfs: add git to the User-Agent header for blob transfers
Matt Harbison <matt_harbison@yahoo.com>
parents: 34694
diff changeset
665 # do look at this value.
e7bb5fc4570c lfs: add git to the User-Agent header for blob transfers
Matt Harbison <matt_harbison@yahoo.com>
parents: 34694
diff changeset
666 if not useragent:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
667 agent = b'mercurial/proto-1.0 (Mercurial %s)' % util.version()
43506
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43106
diff changeset
668 opener.addheaders = [('User-agent', pycompat.sysstr(agent))]
35439
e7bb5fc4570c lfs: add git to the User-Agent header for blob transfers
Matt Harbison <matt_harbison@yahoo.com>
parents: 34694
diff changeset
669 else:
43506
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43106
diff changeset
670 opener.addheaders = [('User-agent', pycompat.sysstr(useragent))]
30763
a520aefb96f1 httppeer: advertise and support application/mercurial-0.2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30664
diff changeset
671
a520aefb96f1 httppeer: advertise and support application/mercurial-0.2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30664
diff changeset
672 # This header should only be needed by wire protocol requests. But it has
a520aefb96f1 httppeer: advertise and support application/mercurial-0.2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30664
diff changeset
673 # been sent on all requests since forever. We keep sending it for backwards
a520aefb96f1 httppeer: advertise and support application/mercurial-0.2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30664
diff changeset
674 # compatibility reasons. Modern versions of the wire protocol use
a520aefb96f1 httppeer: advertise and support application/mercurial-0.2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30664
diff changeset
675 # X-HgProto-<N> for advertising client support.
37045
a708e1e4d7a8 url: support suppressing Accept header
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37011
diff changeset
676 if sendaccept:
43506
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43106
diff changeset
677 opener.addheaders.append(('Accept', 'application/mercurial-0.1'))
37045
a708e1e4d7a8 url: support suppressing Accept header
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37011
diff changeset
678
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
679 return opener
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
680
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
681
42109
2a3c0106ded9 import: suppress accept: header
timeless <timeless@mozdev.org>
parents: 41711
diff changeset
682 def open(ui, url_, data=None, sendaccept=True):
46907
ffd3e823a7e5 urlutil: extract `url` related code from `util` into the new module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46819
diff changeset
683 u = urlutil.url(url_)
13818
bf6156bab41b url: use url.url in url.open()
Brodie Rao <brodie@bitheap.org>
parents: 13817
diff changeset
684 if u.scheme:
bf6156bab41b url: use url.url in url.open()
Brodie Rao <brodie@bitheap.org>
parents: 13817
diff changeset
685 u.scheme = u.scheme.lower()
bf6156bab41b url: use url.url in url.open()
Brodie Rao <brodie@bitheap.org>
parents: 13817
diff changeset
686 url_, authinfo = u.authinfo()
bf6156bab41b url: use url.url in url.open()
Brodie Rao <brodie@bitheap.org>
parents: 13817
diff changeset
687 else:
bf6156bab41b url: use url.url in url.open()
Brodie Rao <brodie@bitheap.org>
parents: 13817
diff changeset
688 path = util.normpath(os.path.abspath(url_))
44641
e74af49aa3c9 url: pass str to pathname2url
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44452
diff changeset
689 url_ = b'file://' + pycompat.bytesurl(
e74af49aa3c9 url: pass str to pathname2url
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44452
diff changeset
690 urlreq.pathname2url(pycompat.fsdecode(path))
e74af49aa3c9 url: pass str to pathname2url
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44452
diff changeset
691 )
7284
ac81ffac0f35 url: fix file:// URL handling
Patrick Mezard <pmezard@gmail.com>
parents: 7270
diff changeset
692 authinfo = None
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
693 return opener(ui, authinfo, sendaccept=sendaccept).open(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
694 pycompat.strurl(url_), data
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
695 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
696
40018
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
697
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
698 def wrapresponse(resp):
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
699 """Wrap a response object with common error handlers.
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
700
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
701 This ensures that any I/O from any consumer raises the appropriate
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
702 error and messaging.
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
703 """
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
704 origread = resp.read
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
705
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
706 class readerproxy(resp.__class__):
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
707 def read(self, size=None):
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
708 try:
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
709 return origread(size)
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
710 except httplib.IncompleteRead as e:
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
711 # e.expected is an integer if length known or None otherwise.
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
712 if e.expected:
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
713 got = len(e.partial)
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
714 total = e.expected + got
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
715 msg = _(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
716 b'HTTP request error (incomplete response; '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
717 b'expected %d bytes got %d)'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
718 ) % (total, got)
40018
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
719 else:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
720 msg = _(b'HTTP request error (incomplete response)')
40018
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
721
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
722 raise error.PeerTransportError(
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
723 msg,
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
724 hint=_(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
725 b'this may be an intermittent network failure; '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
726 b'if the error persists, consider contacting the '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
727 b'network or server operator'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
728 ),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
729 )
40018
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
730 except httplib.HTTPException as e:
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
731 raise error.PeerTransportError(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
732 _(b'HTTP request error (%s)') % e,
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
733 hint=_(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
734 b'this may be an intermittent network failure; '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
735 b'if the error persists, consider contacting the '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
736 b'network or server operator'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
737 ),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
738 )
40018
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
739
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
740 resp.__class__ = readerproxy