annotate mercurial/url.py @ 46370:cad17d50736c

changelog: move branchinfo to changelogrevision The function parses the extra dictionary after looking up the changelogrevision. To avoid duplicated look up, it is better to provide it as property of changelogrevision instead. Keep the function for a release cycle as at least the topic extension depends on it. Differential Revision: https://phab.mercurial-scm.org/D9779
author Joerg Sonnenberger <joerg@bec.de>
date Fri, 15 Jan 2021 01:30:08 +0100
parents 89a2afe31e82
children d4ba4d51f85f
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
1 # url.py - HTTP handling for mercurial
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
2 #
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
3 # Copyright 2005, 2006, 2007, 2008 Matt Mackall <mpm@selenic.com>
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
4 # Copyright 2006, 2007 Alexis S. L. Carvalho <alexis@cecm.usp.br>
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
5 # Copyright 2006 Vadim Gelfer <vadim.gelfer@gmail.com>
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
6 #
8225
46293a0c7e9f updated license to be explicit about GPL version 2
Martin Geisler <mg@lazybytes.net>
parents: 8208
diff changeset
7 # This software may be used and distributed according to the terms of the
10263
25e572394f5c Update license to GPLv2+
Matt Mackall <mpm@selenic.com>
parents: 9852
diff changeset
8 # GNU General Public License version 2 or any later version.
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
9
25990
c3efcdba08ea url: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25837
diff changeset
10 from __future__ import absolute_import
c3efcdba08ea url: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25837
diff changeset
11
c3efcdba08ea url: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25837
diff changeset
12 import base64
c3efcdba08ea url: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25837
diff changeset
13 import os
c3efcdba08ea url: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25837
diff changeset
14 import socket
37100
9e6d3465f17e url: make logginghttphandler compatible with Python 2.7.6
Yuya Nishihara <yuya@tcha.org>
parents: 37084
diff changeset
15 import sys
25990
c3efcdba08ea url: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25837
diff changeset
16
c3efcdba08ea url: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25837
diff changeset
17 from .i18n import _
43089
c59eb1560c44 py3: manually import getattr where it is needed
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43077
diff changeset
18 from .pycompat import getattr
25990
c3efcdba08ea url: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25837
diff changeset
19 from . import (
30820
6a70cf94d1b5 py3: replace pycompat.getenv with encoding.environ.get
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30763
diff changeset
20 encoding,
26587
56b2bcea2529 error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25990
diff changeset
21 error,
25990
c3efcdba08ea url: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25837
diff changeset
22 httpconnection as httpconnectionmod,
c3efcdba08ea url: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25837
diff changeset
23 keepalive,
34428
0ee9cf8d054a url: use native strings for header values
Augie Fackler <augie@google.com>
parents: 33499
diff changeset
24 pycompat,
25990
c3efcdba08ea url: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25837
diff changeset
25 sslutil,
34466
1232f7fa00c3 cleanup: use urllibcompat for renamed methods on urllib request objects
Augie Fackler <augie@google.com>
parents: 34428
diff changeset
26 urllibcompat,
25990
c3efcdba08ea url: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25837
diff changeset
27 util,
c3efcdba08ea url: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25837
diff changeset
28 )
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
29 from .utils import stringutil
29455
0c741fd6158a py3: conditionalize httplib import
Pulkit Goyal <7895pulkit@gmail.com>
parents: 29379
diff changeset
30
0c741fd6158a py3: conditionalize httplib import
Pulkit Goyal <7895pulkit@gmail.com>
parents: 29379
diff changeset
31 httplib = util.httplib
28861
86db5cb55d46 pycompat: switch to util.stringio for py3 compat
timeless <timeless@mozdev.org>
parents: 26806
diff changeset
32 stringio = util.stringio
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28861
diff changeset
33 urlerr = util.urlerr
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28861
diff changeset
34 urlreq = util.urlreq
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28861
diff changeset
35
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
36
34694
2976cf87a60a url: add cgi.escape equivalent for bytestrings
Augie Fackler <augie@google.com>
parents: 34466
diff changeset
37 def escape(s, quote=None):
45942
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45759
diff changeset
38 """Replace special characters "&", "<" and ">" to HTML-safe sequences.
34694
2976cf87a60a url: add cgi.escape equivalent for bytestrings
Augie Fackler <augie@google.com>
parents: 34466
diff changeset
39 If the optional flag quote is true, the quotation mark character (")
2976cf87a60a url: add cgi.escape equivalent for bytestrings
Augie Fackler <augie@google.com>
parents: 34466
diff changeset
40 is also translated.
2976cf87a60a url: add cgi.escape equivalent for bytestrings
Augie Fackler <augie@google.com>
parents: 34466
diff changeset
41
2976cf87a60a url: add cgi.escape equivalent for bytestrings
Augie Fackler <augie@google.com>
parents: 34466
diff changeset
42 This is the same as cgi.escape in Python, but always operates on
2976cf87a60a url: add cgi.escape equivalent for bytestrings
Augie Fackler <augie@google.com>
parents: 34466
diff changeset
43 bytes, whereas cgi.escape in Python 3 only works on unicodes.
45942
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45759
diff changeset
44 """
34694
2976cf87a60a url: add cgi.escape equivalent for bytestrings
Augie Fackler <augie@google.com>
parents: 34466
diff changeset
45 s = s.replace(b"&", b"&amp;")
2976cf87a60a url: add cgi.escape equivalent for bytestrings
Augie Fackler <augie@google.com>
parents: 34466
diff changeset
46 s = s.replace(b"<", b"&lt;")
2976cf87a60a url: add cgi.escape equivalent for bytestrings
Augie Fackler <augie@google.com>
parents: 34466
diff changeset
47 s = s.replace(b">", b"&gt;")
2976cf87a60a url: add cgi.escape equivalent for bytestrings
Augie Fackler <augie@google.com>
parents: 34466
diff changeset
48 if quote:
2976cf87a60a url: add cgi.escape equivalent for bytestrings
Augie Fackler <augie@google.com>
parents: 34466
diff changeset
49 s = s.replace(b'"', b"&quot;")
2976cf87a60a url: add cgi.escape equivalent for bytestrings
Augie Fackler <augie@google.com>
parents: 34466
diff changeset
50 return s
2976cf87a60a url: add cgi.escape equivalent for bytestrings
Augie Fackler <augie@google.com>
parents: 34466
diff changeset
51
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
52
29377
2c019aac6b99 url: extract password database from password manager
liscju <piotr.listkiewicz@gmail.com>
parents: 29252
diff changeset
53 class passwordmgr(object):
2c019aac6b99 url: extract password database from password manager
liscju <piotr.listkiewicz@gmail.com>
parents: 29252
diff changeset
54 def __init__(self, ui, passwddb):
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
55 self.ui = ui
29377
2c019aac6b99 url: extract password database from password manager
liscju <piotr.listkiewicz@gmail.com>
parents: 29252
diff changeset
56 self.passwddb = passwddb
2c019aac6b99 url: extract password database from password manager
liscju <piotr.listkiewicz@gmail.com>
parents: 29252
diff changeset
57
2c019aac6b99 url: extract password database from password manager
liscju <piotr.listkiewicz@gmail.com>
parents: 29252
diff changeset
58 def add_password(self, realm, uri, user, passwd):
2c019aac6b99 url: extract password database from password manager
liscju <piotr.listkiewicz@gmail.com>
parents: 29252
diff changeset
59 return self.passwddb.add_password(realm, uri, user, passwd)
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
60
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
61 def find_user_password(self, realm, authuri):
41449
bc776c31c093 url: add some defensive asserts on expected incoming types
Augie Fackler <augie@google.com>
parents: 40043
diff changeset
62 assert isinstance(realm, (type(None), str))
bc776c31c093 url: add some defensive asserts on expected incoming types
Augie Fackler <augie@google.com>
parents: 40043
diff changeset
63 assert isinstance(authuri, str)
29377
2c019aac6b99 url: extract password database from password manager
liscju <piotr.listkiewicz@gmail.com>
parents: 29252
diff changeset
64 authinfo = self.passwddb.find_user_password(realm, authuri)
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
65 user, passwd = authinfo
41450
d437d1e2a711 url: convert some variables back to bytes
Augie Fackler <augie@google.com>
parents: 41449
diff changeset
66 user, passwd = pycompat.bytesurl(user), pycompat.bytesurl(passwd)
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
67 if user and passwd:
8333
89c80c3dc584 allow http authentication information to be specified in the configuration
Sune Foldager <cryo@cyanite.org>
parents: 8225
diff changeset
68 self._writedebug(user, passwd)
41590
349c8879becd py3: ensure the HTTP password manager returns strings, not bytes
Matt Harbison <matt_harbison@yahoo.com>
parents: 41450
diff changeset
69 return (pycompat.strurl(user), pycompat.strurl(passwd))
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
70
15005
4a43e23b8c55 hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents: 14244
diff changeset
71 if not user or not passwd:
15025
0593e8f81c71 http: pass user to readauthforuri() (fix 4a43e23b8c55)
Patrick Mezard <pmezard@gmail.com>
parents: 15005
diff changeset
72 res = httpconnectionmod.readauthforuri(self.ui, authuri, user)
13372
5bced0d28a39 url: return the matched authentication group name from readauthforuri()
Steve Borho <steve@borho.org>
parents: 13371
diff changeset
73 if res:
5bced0d28a39 url: return the matched authentication group name from readauthforuri()
Steve Borho <steve@borho.org>
parents: 13371
diff changeset
74 group, auth = res
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
75 user, passwd = auth.get(b'username'), auth.get(b'password')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
76 self.ui.debug(b"using auth.%s.* for authentication\n" % group)
8333
89c80c3dc584 allow http authentication information to be specified in the configuration
Sune Foldager <cryo@cyanite.org>
parents: 8225
diff changeset
77 if not user or not passwd:
36652
8381126bf43c url: more bytes/unicodes fussing in url.py around auth handling
Augie Fackler <augie@google.com>
parents: 36454
diff changeset
78 u = util.url(pycompat.bytesurl(authuri))
20291
7d589d923b8a url: added authuri when login information is requested (issue3209)
Lucas Moscovicz <lmoscovicz@fb.com>
parents: 20174
diff changeset
79 u.query = None
8333
89c80c3dc584 allow http authentication information to be specified in the configuration
Sune Foldager <cryo@cyanite.org>
parents: 8225
diff changeset
80 if not self.ui.interactive():
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
81 raise error.Abort(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
82 _(b'http authorization required for %s')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
83 % util.hidepassword(bytes(u))
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
84 )
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
85
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
86 self.ui.write(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
87 _(b"http authorization required for %s\n")
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
88 % util.hidepassword(bytes(u))
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
89 )
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
90 self.ui.write(_(b"realm: %s\n") % pycompat.bytesurl(realm))
8333
89c80c3dc584 allow http authentication information to be specified in the configuration
Sune Foldager <cryo@cyanite.org>
parents: 8225
diff changeset
91 if user:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
92 self.ui.write(_(b"user: %s\n") % user)
8333
89c80c3dc584 allow http authentication information to be specified in the configuration
Sune Foldager <cryo@cyanite.org>
parents: 8225
diff changeset
93 else:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
94 user = self.ui.prompt(_(b"user:"), default=None)
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
95
8333
89c80c3dc584 allow http authentication information to be specified in the configuration
Sune Foldager <cryo@cyanite.org>
parents: 8225
diff changeset
96 if not passwd:
89c80c3dc584 allow http authentication information to be specified in the configuration
Sune Foldager <cryo@cyanite.org>
parents: 8225
diff changeset
97 passwd = self.ui.getpass()
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
98
45759
ff48eea4a926 url: do not continue HTTP authentication with user=None (issue6425)
Yuya Nishihara <yuya@tcha.org>
parents: 44783
diff changeset
99 # As of Python 3.8, the default implementation of
ff48eea4a926 url: do not continue HTTP authentication with user=None (issue6425)
Yuya Nishihara <yuya@tcha.org>
parents: 44783
diff changeset
100 # AbstractBasicAuthHandler.retry_http_basic_auth() assumes the user
ff48eea4a926 url: do not continue HTTP authentication with user=None (issue6425)
Yuya Nishihara <yuya@tcha.org>
parents: 44783
diff changeset
101 # is set if pw is not None. This means (None, str) is not a valid
ff48eea4a926 url: do not continue HTTP authentication with user=None (issue6425)
Yuya Nishihara <yuya@tcha.org>
parents: 44783
diff changeset
102 # return type of find_user_password().
ff48eea4a926 url: do not continue HTTP authentication with user=None (issue6425)
Yuya Nishihara <yuya@tcha.org>
parents: 44783
diff changeset
103 if user is None:
ff48eea4a926 url: do not continue HTTP authentication with user=None (issue6425)
Yuya Nishihara <yuya@tcha.org>
parents: 44783
diff changeset
104 return None, None
ff48eea4a926 url: do not continue HTTP authentication with user=None (issue6425)
Yuya Nishihara <yuya@tcha.org>
parents: 44783
diff changeset
105
29377
2c019aac6b99 url: extract password database from password manager
liscju <piotr.listkiewicz@gmail.com>
parents: 29252
diff changeset
106 self.passwddb.add_password(realm, authuri, user, passwd)
8333
89c80c3dc584 allow http authentication information to be specified in the configuration
Sune Foldager <cryo@cyanite.org>
parents: 8225
diff changeset
107 self._writedebug(user, passwd)
41590
349c8879becd py3: ensure the HTTP password manager returns strings, not bytes
Matt Harbison <matt_harbison@yahoo.com>
parents: 41450
diff changeset
108 return (pycompat.strurl(user), pycompat.strurl(passwd))
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
109
8333
89c80c3dc584 allow http authentication information to be specified in the configuration
Sune Foldager <cryo@cyanite.org>
parents: 8225
diff changeset
110 def _writedebug(self, user, passwd):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
111 msg = _(b'http auth: user %s, password %s\n')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
112 self.ui.debug(msg % (user, passwd and b'*' * len(passwd) or b'not set'))
8333
89c80c3dc584 allow http authentication information to be specified in the configuration
Sune Foldager <cryo@cyanite.org>
parents: 8225
diff changeset
113
15025
0593e8f81c71 http: pass user to readauthforuri() (fix 4a43e23b8c55)
Patrick Mezard <pmezard@gmail.com>
parents: 15005
diff changeset
114 def find_stored_password(self, authuri):
29377
2c019aac6b99 url: extract password database from password manager
liscju <piotr.listkiewicz@gmail.com>
parents: 29252
diff changeset
115 return self.passwddb.find_user_password(None, authuri)
15025
0593e8f81c71 http: pass user to readauthforuri() (fix 4a43e23b8c55)
Patrick Mezard <pmezard@gmail.com>
parents: 15005
diff changeset
116
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
117
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28861
diff changeset
118 class proxyhandler(urlreq.proxyhandler):
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
119 def __init__(self, ui):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
120 proxyurl = ui.config(b"http_proxy", b"host") or encoding.environ.get(
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
121 b'http_proxy'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
122 )
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
123 # XXX proxyauthinfo = None
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
124
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
125 if proxyurl:
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
126 # proxy can be proper url or host[:port]
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
127 if not (
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
128 proxyurl.startswith(b'http:') or proxyurl.startswith(b'https:')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
129 ):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
130 proxyurl = b'http://' + proxyurl + b'/'
14076
924c82157d46 url: move URL parsing functions into util to improve startup time
Brodie Rao <brodie@bitheap.org>
parents: 14071
diff changeset
131 proxy = util.url(proxyurl)
13820
65b89e80f892 url: use url.url in proxyhandler
Brodie Rao <brodie@bitheap.org>
parents: 13819
diff changeset
132 if not proxy.user:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
133 proxy.user = ui.config(b"http_proxy", b"user")
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
134 proxy.passwd = ui.config(b"http_proxy", b"passwd")
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
135
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
136 # see if we should use a proxy for this url
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
137 no_list = [b"localhost", b"127.0.0.1"]
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
138 no_list.extend(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
139 [p.lower() for p in ui.configlist(b"http_proxy", b"no")]
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
140 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
141 no_list.extend(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
142 [
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
143 p.strip().lower()
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
144 for p in encoding.environ.get(b"no_proxy", b'').split(b',')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
145 if p.strip()
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
146 ]
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
147 )
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
148 # "http_proxy.always" config is for running tests on localhost
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
149 if ui.configbool(b"http_proxy", b"always"):
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
150 self.no_list = []
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
151 else:
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
152 self.no_list = no_list
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
153
41710
4028897dfa05 url: always use str for proxy configuration
Gregory Szorc <gregory.szorc@gmail.com>
parents: 41708
diff changeset
154 # Keys and values need to be str because the standard library
4028897dfa05 url: always use str for proxy configuration
Gregory Szorc <gregory.szorc@gmail.com>
parents: 41708
diff changeset
155 # expects them to be.
4028897dfa05 url: always use str for proxy configuration
Gregory Szorc <gregory.szorc@gmail.com>
parents: 41708
diff changeset
156 proxyurl = str(proxy)
43506
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43106
diff changeset
157 proxies = {'http': proxyurl, 'https': proxyurl}
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
158 ui.debug(b'proxying through %s\n' % util.hidepassword(bytes(proxy)))
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
159 else:
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
160 proxies = {}
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
161
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28861
diff changeset
162 urlreq.proxyhandler.__init__(self, proxies)
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
163 self.ui = ui
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
164
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
165 def proxy_open(self, req, proxy, type_):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
166 host = pycompat.bytesurl(urllibcompat.gethost(req)).split(b':')[0]
19535
df2155ebf502 proxy: allow wildcards in the no proxy list (issue1821)
Matt Mackall <mpm@selenic.com>
parents: 18176
diff changeset
167 for e in self.no_list:
df2155ebf502 proxy: allow wildcards in the no proxy list (issue1821)
Matt Mackall <mpm@selenic.com>
parents: 18176
diff changeset
168 if host == e:
df2155ebf502 proxy: allow wildcards in the no proxy list (issue1821)
Matt Mackall <mpm@selenic.com>
parents: 18176
diff changeset
169 return None
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
170 if e.startswith(b'*.') and host.endswith(e[2:]):
19535
df2155ebf502 proxy: allow wildcards in the no proxy list (issue1821)
Matt Mackall <mpm@selenic.com>
parents: 18176
diff changeset
171 return None
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
172 if e.startswith(b'.') and host.endswith(e[1:]):
19535
df2155ebf502 proxy: allow wildcards in the no proxy list (issue1821)
Matt Mackall <mpm@selenic.com>
parents: 18176
diff changeset
173 return None
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
174
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28861
diff changeset
175 return urlreq.proxyhandler.proxy_open(self, req, proxy, type_)
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
176
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
177
13420
051f498628f7 url: refactor _gen_sendfile
Mads Kiilerich <mads@kiilerich.com>
parents: 13419
diff changeset
178 def _gen_sendfile(orgsend):
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
179 def _sendfile(self, data):
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
180 # send a file
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents: 14204
diff changeset
181 if isinstance(data, httpconnectionmod.httpsendfile):
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
182 # if auth required, some data sent twice, so rewind here
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
183 data.seek(0)
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
184 for chunk in util.filechunkiter(data):
13420
051f498628f7 url: refactor _gen_sendfile
Mads Kiilerich <mads@kiilerich.com>
parents: 13419
diff changeset
185 orgsend(self, chunk)
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
186 else:
13420
051f498628f7 url: refactor _gen_sendfile
Mads Kiilerich <mads@kiilerich.com>
parents: 13419
diff changeset
187 orgsend(self, data)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
188
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
189 return _sendfile
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
190
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
191
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
192 has_https = util.safehasattr(urlreq, b'httpshandler')
10409
4c94a3df4b10 url: SSL server certificate verification using web.cacerts file (issue1174)
Henrik Stuart <hg@hstuart.dk>
parents: 10408
diff changeset
193
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
194
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
195 class httpconnection(keepalive.HTTPConnection):
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
196 # must be able to send big bundle as stream.
13420
051f498628f7 url: refactor _gen_sendfile
Mads Kiilerich <mads@kiilerich.com>
parents: 13419
diff changeset
197 send = _gen_sendfile(keepalive.HTTPConnection.send)
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
198
8590
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967)
Henrik Stuart <hg@hstuart.dk>
parents: 8344
diff changeset
199 def getresponse(self):
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967)
Henrik Stuart <hg@hstuart.dk>
parents: 8344
diff changeset
200 proxyres = getattr(self, 'proxyres', None)
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967)
Henrik Stuart <hg@hstuart.dk>
parents: 8344
diff changeset
201 if proxyres:
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967)
Henrik Stuart <hg@hstuart.dk>
parents: 8344
diff changeset
202 if proxyres.will_close:
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967)
Henrik Stuart <hg@hstuart.dk>
parents: 8344
diff changeset
203 self.close()
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967)
Henrik Stuart <hg@hstuart.dk>
parents: 8344
diff changeset
204 self.proxyres = None
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967)
Henrik Stuart <hg@hstuart.dk>
parents: 8344
diff changeset
205 return proxyres
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967)
Henrik Stuart <hg@hstuart.dk>
parents: 8344
diff changeset
206 return keepalive.HTTPConnection.getresponse(self)
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967)
Henrik Stuart <hg@hstuart.dk>
parents: 8344
diff changeset
207
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
208
41708
d20f1594ff4a url: always access req._tunnel_host
Gregory Szorc <gregory.szorc@gmail.com>
parents: 41590
diff changeset
209 # Large parts of this function have their origin from before Python 2.6
d20f1594ff4a url: always access req._tunnel_host
Gregory Szorc <gregory.szorc@gmail.com>
parents: 41590
diff changeset
210 # and could potentially be removed.
9852
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
211 def _generic_start_transaction(handler, h, req):
41708
d20f1594ff4a url: always access req._tunnel_host
Gregory Szorc <gregory.szorc@gmail.com>
parents: 41590
diff changeset
212 tunnel_host = req._tunnel_host
14964
376c32a5ccdc url: replace uses of hasattr with safehasattr or getattr
Augie Fackler <durin42@gmail.com>
parents: 14244
diff changeset
213 if tunnel_host:
43506
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43106
diff changeset
214 if tunnel_host[:7] not in ['http://', 'https:/']:
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43106
diff changeset
215 tunnel_host = 'https://' + tunnel_host
9852
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
216 new_tunnel = True
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
217 else:
34466
1232f7fa00c3 cleanup: use urllibcompat for renamed methods on urllib request objects
Augie Fackler <augie@google.com>
parents: 34428
diff changeset
218 tunnel_host = urllibcompat.getselector(req)
9852
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
219 new_tunnel = False
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
220
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
221 if new_tunnel or tunnel_host == urllibcompat.getfullurl(req): # has proxy
41710
4028897dfa05 url: always use str for proxy configuration
Gregory Szorc <gregory.szorc@gmail.com>
parents: 41708
diff changeset
222 u = util.url(pycompat.bytesurl(tunnel_host))
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
223 if new_tunnel or u.scheme == b'https': # only use CONNECT for HTTPS
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
224 h.realhostport = b':'.join([u.host, (u.port or b'443')])
9852
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
225 h.headers = req.headers.copy()
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
226 h.headers.update(handler.parent.addheaders)
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
227 return
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
228
10415
677f15da38c1 url: proxy handling, simplify and correctly deal with IPv6
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 10411
diff changeset
229 h.realhostport = None
9852
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
230 h.headers = None
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
231
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
232
9852
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
233 def _generic_proxytunnel(self):
44452
9d2b2df2c2ba cleanup: run pyupgrade on our source tree to clean up varying things
Augie Fackler <augie@google.com>
parents: 43506
diff changeset
234 proxyheaders = {
44783
a50f33f1ff24 url: fix a bytes vs str crash in processing proxy headers (issue6249)
Matt Harbison <matt_harbison@yahoo.com>
parents: 44641
diff changeset
235 pycompat.bytestr(x): pycompat.bytestr(self.headers[x])
44452
9d2b2df2c2ba cleanup: run pyupgrade on our source tree to clean up varying things
Augie Fackler <augie@google.com>
parents: 43506
diff changeset
236 for x in self.headers
9d2b2df2c2ba cleanup: run pyupgrade on our source tree to clean up varying things
Augie Fackler <augie@google.com>
parents: 43506
diff changeset
237 if x.lower().startswith('proxy-')
9d2b2df2c2ba cleanup: run pyupgrade on our source tree to clean up varying things
Augie Fackler <augie@google.com>
parents: 43506
diff changeset
238 }
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
239 self.send(b'CONNECT %s HTTP/1.0\r\n' % self.realhostport)
43106
d783f945a701 py3: finish porting iteritems() to pycompat and remove source transformer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43089
diff changeset
240 for header in pycompat.iteritems(proxyheaders):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
241 self.send(b'%s: %s\r\n' % header)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
242 self.send(b'\r\n')
9852
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
243
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
244 # majority of the following code is duplicated from
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
245 # httplib.HTTPConnection as there are no adequate places to
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
246 # override functions to provide the needed functionality
41711
544035846830 url: don't pass strict argument on Python 3
Gregory Szorc <gregory.szorc@gmail.com>
parents: 41710
diff changeset
247 # strict was removed in Python 3.4.
544035846830 url: don't pass strict argument on Python 3
Gregory Szorc <gregory.szorc@gmail.com>
parents: 41710
diff changeset
248 kwargs = {}
544035846830 url: don't pass strict argument on Python 3
Gregory Szorc <gregory.szorc@gmail.com>
parents: 41710
diff changeset
249 if not pycompat.ispy3:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
250 kwargs[b'strict'] = self.strict
41711
544035846830 url: don't pass strict argument on Python 3
Gregory Szorc <gregory.szorc@gmail.com>
parents: 41710
diff changeset
251
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
252 res = self.response_class(self.sock, method=self._method, **kwargs)
9852
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
253
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
254 while True:
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
255 version, status, reason = res._read_status()
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
256 if status != httplib.CONTINUE:
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
257 break
29729
44ea12756fef url: use `iter(callable, sentinel)` instead of while True
Augie Fackler <augie@google.com>
parents: 29662
diff changeset
258 # skip lines that are all whitespace
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
259 list(iter(lambda: res.fp.readline().strip(), b''))
9852
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
260 res.status = status
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
261 res.reason = reason.strip()
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
262
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
263 if res.status == 200:
29729
44ea12756fef url: use `iter(callable, sentinel)` instead of while True
Augie Fackler <augie@google.com>
parents: 29662
diff changeset
264 # skip lines until we find a blank line
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
265 list(iter(res.fp.readline, b'\r\n'))
9852
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
266 return True
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
267
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
268 if version == b'HTTP/1.0':
9852
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
269 res.version = 10
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
270 elif version.startswith(b'HTTP/1.'):
9852
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
271 res.version = 11
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
272 elif version == b'HTTP/0.9':
9852
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
273 res.version = 9
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
274 else:
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
275 raise httplib.UnknownProtocol(version)
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
276
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
277 if res.version == 9:
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
278 res.length = None
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
279 res.chunked = 0
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
280 res.will_close = 1
28861
86db5cb55d46 pycompat: switch to util.stringio for py3 compat
timeless <timeless@mozdev.org>
parents: 26806
diff changeset
281 res.msg = httplib.HTTPMessage(stringio())
9852
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
282 return False
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
283
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
284 res.msg = httplib.HTTPMessage(res.fp)
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
285 res.msg.fp = None
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
286
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
287 # are we using the chunked-style of transfer encoding?
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
288 trenc = res.msg.getheader(b'transfer-encoding')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
289 if trenc and trenc.lower() == b"chunked":
9852
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
290 res.chunked = 1
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
291 res.chunk_left = None
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
292 else:
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
293 res.chunked = 0
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
294
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
295 # will the connection close at the end of the response?
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
296 res.will_close = res._check_close()
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
297
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
298 # do we have a Content-Length?
17428
72803c8edaa4 avoid using abbreviations that look like spelling errors
Mads Kiilerich <mads@kiilerich.com>
parents: 17424
diff changeset
299 # NOTE: RFC 2616, section 4.4, #3 says we ignore this if
72803c8edaa4 avoid using abbreviations that look like spelling errors
Mads Kiilerich <mads@kiilerich.com>
parents: 17424
diff changeset
300 # transfer-encoding is "chunked"
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
301 length = res.msg.getheader(b'content-length')
9852
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
302 if length and not res.chunked:
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
303 try:
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
304 res.length = int(length)
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
305 except ValueError:
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
306 res.length = None
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
307 else:
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
308 if res.length < 0: # ignore nonsensical negative lengths
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
309 res.length = None
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
310 else:
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
311 res.length = None
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
312
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
313 # does the body have a fixed length? (of zero)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
314 if (
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
315 status == httplib.NO_CONTENT
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
316 or status == httplib.NOT_MODIFIED
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
317 or 100 <= status < 200
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
318 or res._method == b'HEAD' # 1xx codes
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
319 ):
9852
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
320 res.length = 0
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
321
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
322 # if the connection remains open, and we aren't using chunked, and
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
323 # a content-length was not provided, then assume that the connection
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
324 # WILL close.
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
325 if not res.will_close and not res.chunked and res.length is None:
9852
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
326 res.will_close = 1
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
327
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
328 self.proxyres = res
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
329
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
330 return False
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
331
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
332
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
333 class httphandler(keepalive.HTTPHandler):
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
334 def http_open(self, req):
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
335 return self.do_open(httpconnection, req)
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
336
8590
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967)
Henrik Stuart <hg@hstuart.dk>
parents: 8344
diff changeset
337 def _start_transaction(self, h, req):
9852
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
338 _generic_start_transaction(self, h, req)
8590
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967)
Henrik Stuart <hg@hstuart.dk>
parents: 8344
diff changeset
339 return keepalive.HTTPHandler._start_transaction(self, h, req)
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967)
Henrik Stuart <hg@hstuart.dk>
parents: 8344
diff changeset
340
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
341
37011
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
342 class logginghttpconnection(keepalive.HTTPConnection):
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
343 def __init__(self, createconn, *args, **kwargs):
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
344 keepalive.HTTPConnection.__init__(self, *args, **kwargs)
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
345 self._create_connection = createconn
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
346
37100
9e6d3465f17e url: make logginghttphandler compatible with Python 2.7.6
Yuya Nishihara <yuya@tcha.org>
parents: 37084
diff changeset
347 if sys.version_info < (2, 7, 7):
9e6d3465f17e url: make logginghttphandler compatible with Python 2.7.6
Yuya Nishihara <yuya@tcha.org>
parents: 37084
diff changeset
348 # copied from 2.7.14, since old implementations directly call
9e6d3465f17e url: make logginghttphandler compatible with Python 2.7.6
Yuya Nishihara <yuya@tcha.org>
parents: 37084
diff changeset
349 # socket.create_connection()
9e6d3465f17e url: make logginghttphandler compatible with Python 2.7.6
Yuya Nishihara <yuya@tcha.org>
parents: 37084
diff changeset
350 def connect(self):
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
351 self.sock = self._create_connection(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
352 (self.host, self.port), self.timeout, self.source_address
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
353 )
37100
9e6d3465f17e url: make logginghttphandler compatible with Python 2.7.6
Yuya Nishihara <yuya@tcha.org>
parents: 37084
diff changeset
354 if self._tunnel_host:
9e6d3465f17e url: make logginghttphandler compatible with Python 2.7.6
Yuya Nishihara <yuya@tcha.org>
parents: 37084
diff changeset
355 self._tunnel()
9e6d3465f17e url: make logginghttphandler compatible with Python 2.7.6
Yuya Nishihara <yuya@tcha.org>
parents: 37084
diff changeset
356
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
357
37011
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
358 class logginghttphandler(httphandler):
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
359 """HTTP handler that logs socket I/O."""
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
360
40043
6509fcec830c url: allow to configure timeout on http connection
Cédric Krier <ced@b2ck.com>
parents: 40034
diff changeset
361 def __init__(self, logfh, name, observeropts, timeout=None):
6509fcec830c url: allow to configure timeout on http connection
Cédric Krier <ced@b2ck.com>
parents: 40034
diff changeset
362 super(logginghttphandler, self).__init__(timeout=timeout)
37011
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
363
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
364 self._logfh = logfh
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
365 self._logname = name
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
366 self._observeropts = observeropts
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
367
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
368 # do_open() calls the passed class to instantiate an HTTPConnection. We
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
369 # pass in a callable method that creates a custom HTTPConnection instance
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
370 # whose callback to create the socket knows how to proxy the socket.
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
371 def http_open(self, req):
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
372 return self.do_open(self._makeconnection, req)
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
373
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
374 def _makeconnection(self, *args, **kwargs):
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
375 def createconnection(*args, **kwargs):
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
376 sock = socket.create_connection(*args, **kwargs)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
377 return util.makeloggingsocket(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
378 self._logfh, sock, self._logname, **self._observeropts
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
379 )
37011
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
380
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
381 return logginghttpconnection(createconnection, *args, **kwargs)
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
382
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
383
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
384 if has_https:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
385
40031
f2dffa1359c6 url: have httpsconnection inherit from our custom HTTPConnection
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40018
diff changeset
386 class httpsconnection(keepalive.HTTPConnection):
13424
08f9c587141f url: merge BetterHTTPS with httpsconnection to get some proxy https validation
Mads Kiilerich <mads@kiilerich.com>
parents: 13422
diff changeset
387 response_class = keepalive.HTTPResponse
25414
f7ccbc2776b7 https: do not inherit httplib.HTTPSConnection that creates unused SSLContext
Yuya Nishihara <yuya@tcha.org>
parents: 25207
diff changeset
388 default_port = httplib.HTTPS_PORT
13424
08f9c587141f url: merge BetterHTTPS with httpsconnection to get some proxy https validation
Mads Kiilerich <mads@kiilerich.com>
parents: 13422
diff changeset
389 # must be able to send big bundle as stream.
08f9c587141f url: merge BetterHTTPS with httpsconnection to get some proxy https validation
Mads Kiilerich <mads@kiilerich.com>
parents: 13422
diff changeset
390 send = _gen_sendfile(keepalive.safesend)
25414
f7ccbc2776b7 https: do not inherit httplib.HTTPSConnection that creates unused SSLContext
Yuya Nishihara <yuya@tcha.org>
parents: 25207
diff changeset
391 getresponse = keepalive.wrapgetresponse(httplib.HTTPConnection)
f7ccbc2776b7 https: do not inherit httplib.HTTPSConnection that creates unused SSLContext
Yuya Nishihara <yuya@tcha.org>
parents: 25207
diff changeset
392
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
393 def __init__(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
394 self,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
395 host,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
396 port=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
397 key_file=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
398 cert_file=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
399 *args,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
400 **kwargs
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
401 ):
40031
f2dffa1359c6 url: have httpsconnection inherit from our custom HTTPConnection
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40018
diff changeset
402 keepalive.HTTPConnection.__init__(self, host, port, *args, **kwargs)
25414
f7ccbc2776b7 https: do not inherit httplib.HTTPSConnection that creates unused SSLContext
Yuya Nishihara <yuya@tcha.org>
parents: 25207
diff changeset
403 self.key_file = key_file
f7ccbc2776b7 https: do not inherit httplib.HTTPSConnection that creates unused SSLContext
Yuya Nishihara <yuya@tcha.org>
parents: 25207
diff changeset
404 self.cert_file = cert_file
9726
430e59ff3437 keepalive: handle broken pipes gracefully during large POSTs
Augie Fackler <durin42@gmail.com>
parents: 9467
diff changeset
405
10409
4c94a3df4b10 url: SSL server certificate verification using web.cacerts file (issue1174)
Henrik Stuart <hg@hstuart.dk>
parents: 10408
diff changeset
406 def connect(self):
29662
b76ea1384bf2 url: drop compatibility wrapper of socket.create_connection()
Yuya Nishihara <yuya@tcha.org>
parents: 29639
diff changeset
407 self.sock = socket.create_connection((self.host, self.port))
13422
ebce5196b9db url: always create BetterHTTPS connections the same way
Mads Kiilerich <mads@kiilerich.com>
parents: 13421
diff changeset
408
13421
bd8bfa85d5a5 url: refactor BetterHTTPS.connect
Mads Kiilerich <mads@kiilerich.com>
parents: 13420
diff changeset
409 host = self.host
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
410 if self.realhostport: # use CONNECT proxy
14064
e4bfb9c337f3 remove unused imports and variables
Alexander Solovyov <alexander@solovyov.net>
parents: 13902
diff changeset
411 _generic_proxytunnel(self)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
412 host = self.realhostport.rsplit(b':', 1)[0]
25429
9d1c61715939 ssl: rename ssl_wrap_socket() to conform to our naming convention
Yuya Nishihara <yuya@tcha.org>
parents: 25415
diff changeset
413 self.sock = sslutil.wrapsocket(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
414 self.sock,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
415 self.key_file,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
416 self.cert_file,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
417 ui=self.ui,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
418 serverhostname=host,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
419 )
29227
dffe78d80a6c sslutil: convert socket validation from a class to a function (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28883
diff changeset
420 sslutil.validatesocket(self.sock)
10409
4c94a3df4b10 url: SSL server certificate verification using web.cacerts file (issue1174)
Henrik Stuart <hg@hstuart.dk>
parents: 10408
diff changeset
421
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28861
diff changeset
422 class httpshandler(keepalive.KeepAliveHandler, urlreq.httpshandler):
40043
6509fcec830c url: allow to configure timeout on http connection
Cédric Krier <ced@b2ck.com>
parents: 40034
diff changeset
423 def __init__(self, ui, timeout=None):
6509fcec830c url: allow to configure timeout on http connection
Cédric Krier <ced@b2ck.com>
parents: 40034
diff changeset
424 keepalive.KeepAliveHandler.__init__(self, timeout=timeout)
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28861
diff changeset
425 urlreq.httpshandler.__init__(self)
8847
7951f385fcb7 url: support client certificate files over HTTPS (issue643)
Henrik Stuart <hg@hstuart.dk>
parents: 8590
diff changeset
426 self.ui = ui
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
427 self.pwmgr = passwordmgr(self.ui, self.ui.httppasswordmgrdb)
8847
7951f385fcb7 url: support client certificate files over HTTPS (issue643)
Henrik Stuart <hg@hstuart.dk>
parents: 8590
diff changeset
428
9852
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
429 def _start_transaction(self, h, req):
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
430 _generic_start_transaction(self, h, req)
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
431 return keepalive.KeepAliveHandler._start_transaction(self, h, req)
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes
Henrik Stuart <hg@hstuart.dk>
parents: 9726
diff changeset
432
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
433 def https_open(self, req):
34466
1232f7fa00c3 cleanup: use urllibcompat for renamed methods on urllib request objects
Augie Fackler <augie@google.com>
parents: 34428
diff changeset
434 # urllibcompat.getfullurl() does not contain credentials
1232f7fa00c3 cleanup: use urllibcompat for renamed methods on urllib request objects
Augie Fackler <augie@google.com>
parents: 34428
diff changeset
435 # and we may need them to match the certificates.
1232f7fa00c3 cleanup: use urllibcompat for renamed methods on urllib request objects
Augie Fackler <augie@google.com>
parents: 34428
diff changeset
436 url = urllibcompat.getfullurl(req)
15025
0593e8f81c71 http: pass user to readauthforuri() (fix 4a43e23b8c55)
Patrick Mezard <pmezard@gmail.com>
parents: 15005
diff changeset
437 user, password = self.pwmgr.find_stored_password(url)
0593e8f81c71 http: pass user to readauthforuri() (fix 4a43e23b8c55)
Patrick Mezard <pmezard@gmail.com>
parents: 15005
diff changeset
438 res = httpconnectionmod.readauthforuri(self.ui, url, user)
13372
5bced0d28a39 url: return the matched authentication group name from readauthforuri()
Steve Borho <steve@borho.org>
parents: 13371
diff changeset
439 if res:
5bced0d28a39 url: return the matched authentication group name from readauthforuri()
Steve Borho <steve@borho.org>
parents: 13371
diff changeset
440 group, auth = res
5bced0d28a39 url: return the matched authentication group name from readauthforuri()
Steve Borho <steve@borho.org>
parents: 13371
diff changeset
441 self.auth = auth
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
442 self.ui.debug(b"using auth.%s.* for authentication\n" % group)
13372
5bced0d28a39 url: return the matched authentication group name from readauthforuri()
Steve Borho <steve@borho.org>
parents: 13371
diff changeset
443 else:
5bced0d28a39 url: return the matched authentication group name from readauthforuri()
Steve Borho <steve@borho.org>
parents: 13371
diff changeset
444 self.auth = None
8847
7951f385fcb7 url: support client certificate files over HTTPS (issue643)
Henrik Stuart <hg@hstuart.dk>
parents: 8590
diff changeset
445 return self.do_open(self._makeconnection, req)
7951f385fcb7 url: support client certificate files over HTTPS (issue643)
Henrik Stuart <hg@hstuart.dk>
parents: 8590
diff changeset
446
10408
50fb1fe143ff url: httplib.HTTPSConnection already handles IPv6 and port parsing fine
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 10282
diff changeset
447 def _makeconnection(self, host, port=None, *args, **kwargs):
8847
7951f385fcb7 url: support client certificate files over HTTPS (issue643)
Henrik Stuart <hg@hstuart.dk>
parents: 8590
diff changeset
448 keyfile = None
7951f385fcb7 url: support client certificate files over HTTPS (issue643)
Henrik Stuart <hg@hstuart.dk>
parents: 8590
diff changeset
449 certfile = None
7951f385fcb7 url: support client certificate files over HTTPS (issue643)
Henrik Stuart <hg@hstuart.dk>
parents: 8590
diff changeset
450
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
451 if len(args) >= 1: # key_file
10511
6f61c480f51c url: *args argument is a tuple, not a list (found by pylint)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 10484
diff changeset
452 keyfile = args[0]
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
453 if len(args) >= 2: # cert_file
10511
6f61c480f51c url: *args argument is a tuple, not a list (found by pylint)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 10484
diff changeset
454 certfile = args[1]
6f61c480f51c url: *args argument is a tuple, not a list (found by pylint)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 10484
diff changeset
455 args = args[2:]
8847
7951f385fcb7 url: support client certificate files over HTTPS (issue643)
Henrik Stuart <hg@hstuart.dk>
parents: 8590
diff changeset
456
7951f385fcb7 url: support client certificate files over HTTPS (issue643)
Henrik Stuart <hg@hstuart.dk>
parents: 8590
diff changeset
457 # if the user has specified different key/cert files in
7951f385fcb7 url: support client certificate files over HTTPS (issue643)
Henrik Stuart <hg@hstuart.dk>
parents: 8590
diff changeset
458 # hgrc, we prefer these
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
459 if self.auth and b'key' in self.auth and b'cert' in self.auth:
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
460 keyfile = self.auth[b'key']
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
461 certfile = self.auth[b'cert']
8847
7951f385fcb7 url: support client certificate files over HTTPS (issue643)
Henrik Stuart <hg@hstuart.dk>
parents: 8590
diff changeset
462
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
463 conn = httpsconnection(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
464 host, port, keyfile, certfile, *args, **kwargs
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
465 )
10409
4c94a3df4b10 url: SSL server certificate verification using web.cacerts file (issue1174)
Henrik Stuart <hg@hstuart.dk>
parents: 10408
diff changeset
466 conn.ui = self.ui
4c94a3df4b10 url: SSL server certificate verification using web.cacerts file (issue1174)
Henrik Stuart <hg@hstuart.dk>
parents: 10408
diff changeset
467 return conn
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
468
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
469
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28861
diff changeset
470 class httpdigestauthhandler(urlreq.httpdigestauthhandler):
11457
2ec346160783 http digest auth: reset redirect counter on new requests (issue2255)
Mads Kiilerich <mads@kiilerich.com>
parents: 11415
diff changeset
471 def __init__(self, *args, **kwargs):
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28861
diff changeset
472 urlreq.httpdigestauthhandler.__init__(self, *args, **kwargs)
11457
2ec346160783 http digest auth: reset redirect counter on new requests (issue2255)
Mads Kiilerich <mads@kiilerich.com>
parents: 11415
diff changeset
473 self.retried_req = None
2ec346160783 http digest auth: reset redirect counter on new requests (issue2255)
Mads Kiilerich <mads@kiilerich.com>
parents: 11415
diff changeset
474
2ec346160783 http digest auth: reset redirect counter on new requests (issue2255)
Mads Kiilerich <mads@kiilerich.com>
parents: 11415
diff changeset
475 def reset_retry_count(self):
2ec346160783 http digest auth: reset redirect counter on new requests (issue2255)
Mads Kiilerich <mads@kiilerich.com>
parents: 11415
diff changeset
476 # Python 2.6.5 will call this on 401 or 407 errors and thus loop
2ec346160783 http digest auth: reset redirect counter on new requests (issue2255)
Mads Kiilerich <mads@kiilerich.com>
parents: 11415
diff changeset
477 # forever. We disable reset_retry_count completely and reset in
2ec346160783 http digest auth: reset redirect counter on new requests (issue2255)
Mads Kiilerich <mads@kiilerich.com>
parents: 11415
diff changeset
478 # http_error_auth_reqed instead.
2ec346160783 http digest auth: reset redirect counter on new requests (issue2255)
Mads Kiilerich <mads@kiilerich.com>
parents: 11415
diff changeset
479 pass
2ec346160783 http digest auth: reset redirect counter on new requests (issue2255)
Mads Kiilerich <mads@kiilerich.com>
parents: 11415
diff changeset
480
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
481 def http_error_auth_reqed(self, auth_header, host, req, headers):
11457
2ec346160783 http digest auth: reset redirect counter on new requests (issue2255)
Mads Kiilerich <mads@kiilerich.com>
parents: 11415
diff changeset
482 # Reset the retry counter once for each request.
2ec346160783 http digest auth: reset redirect counter on new requests (issue2255)
Mads Kiilerich <mads@kiilerich.com>
parents: 11415
diff changeset
483 if req is not self.retried_req:
2ec346160783 http digest auth: reset redirect counter on new requests (issue2255)
Mads Kiilerich <mads@kiilerich.com>
parents: 11415
diff changeset
484 self.retried_req = req
2ec346160783 http digest auth: reset redirect counter on new requests (issue2255)
Mads Kiilerich <mads@kiilerich.com>
parents: 11415
diff changeset
485 self.retried = 0
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28861
diff changeset
486 return urlreq.httpdigestauthhandler.http_error_auth_reqed(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
487 self, auth_header, host, req, headers
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
488 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
489
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
490
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28861
diff changeset
491 class httpbasicauthhandler(urlreq.httpbasicauthhandler):
11844
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255)
Wagner Bruna <wbruna@softwareexpress.com.br>
parents: 11457
diff changeset
492 def __init__(self, *args, **kwargs):
20964
a939eeb94833 http: reuse authentication info after the first failed request (issue3567)
Stéphane Klein <contact@stephane-klein.info>
parents: 20291
diff changeset
493 self.auth = None
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28861
diff changeset
494 urlreq.httpbasicauthhandler.__init__(self, *args, **kwargs)
11844
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255)
Wagner Bruna <wbruna@softwareexpress.com.br>
parents: 11457
diff changeset
495 self.retried_req = None
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255)
Wagner Bruna <wbruna@softwareexpress.com.br>
parents: 11457
diff changeset
496
20964
a939eeb94833 http: reuse authentication info after the first failed request (issue3567)
Stéphane Klein <contact@stephane-klein.info>
parents: 20291
diff changeset
497 def http_request(self, request):
a939eeb94833 http: reuse authentication info after the first failed request (issue3567)
Stéphane Klein <contact@stephane-klein.info>
parents: 20291
diff changeset
498 if self.auth:
a939eeb94833 http: reuse authentication info after the first failed request (issue3567)
Stéphane Klein <contact@stephane-klein.info>
parents: 20291
diff changeset
499 request.add_unredirected_header(self.auth_header, self.auth)
a939eeb94833 http: reuse authentication info after the first failed request (issue3567)
Stéphane Klein <contact@stephane-klein.info>
parents: 20291
diff changeset
500
a939eeb94833 http: reuse authentication info after the first failed request (issue3567)
Stéphane Klein <contact@stephane-klein.info>
parents: 20291
diff changeset
501 return request
a939eeb94833 http: reuse authentication info after the first failed request (issue3567)
Stéphane Klein <contact@stephane-klein.info>
parents: 20291
diff changeset
502
a939eeb94833 http: reuse authentication info after the first failed request (issue3567)
Stéphane Klein <contact@stephane-klein.info>
parents: 20291
diff changeset
503 def https_request(self, request):
a939eeb94833 http: reuse authentication info after the first failed request (issue3567)
Stéphane Klein <contact@stephane-klein.info>
parents: 20291
diff changeset
504 if self.auth:
a939eeb94833 http: reuse authentication info after the first failed request (issue3567)
Stéphane Klein <contact@stephane-klein.info>
parents: 20291
diff changeset
505 request.add_unredirected_header(self.auth_header, self.auth)
a939eeb94833 http: reuse authentication info after the first failed request (issue3567)
Stéphane Klein <contact@stephane-klein.info>
parents: 20291
diff changeset
506
a939eeb94833 http: reuse authentication info after the first failed request (issue3567)
Stéphane Klein <contact@stephane-klein.info>
parents: 20291
diff changeset
507 return request
a939eeb94833 http: reuse authentication info after the first failed request (issue3567)
Stéphane Klein <contact@stephane-klein.info>
parents: 20291
diff changeset
508
11844
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255)
Wagner Bruna <wbruna@softwareexpress.com.br>
parents: 11457
diff changeset
509 def reset_retry_count(self):
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255)
Wagner Bruna <wbruna@softwareexpress.com.br>
parents: 11457
diff changeset
510 # Python 2.6.5 will call this on 401 or 407 errors and thus loop
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255)
Wagner Bruna <wbruna@softwareexpress.com.br>
parents: 11457
diff changeset
511 # forever. We disable reset_retry_count completely and reset in
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255)
Wagner Bruna <wbruna@softwareexpress.com.br>
parents: 11457
diff changeset
512 # http_error_auth_reqed instead.
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255)
Wagner Bruna <wbruna@softwareexpress.com.br>
parents: 11457
diff changeset
513 pass
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255)
Wagner Bruna <wbruna@softwareexpress.com.br>
parents: 11457
diff changeset
514
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255)
Wagner Bruna <wbruna@softwareexpress.com.br>
parents: 11457
diff changeset
515 def http_error_auth_reqed(self, auth_header, host, req, headers):
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255)
Wagner Bruna <wbruna@softwareexpress.com.br>
parents: 11457
diff changeset
516 # Reset the retry counter once for each request.
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255)
Wagner Bruna <wbruna@softwareexpress.com.br>
parents: 11457
diff changeset
517 if req is not self.retried_req:
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255)
Wagner Bruna <wbruna@softwareexpress.com.br>
parents: 11457
diff changeset
518 self.retried_req = req
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255)
Wagner Bruna <wbruna@softwareexpress.com.br>
parents: 11457
diff changeset
519 self.retried = 0
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28861
diff changeset
520 return urlreq.httpbasicauthhandler.http_error_auth_reqed(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
521 self, auth_header, host, req, headers
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
522 )
11844
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255)
Wagner Bruna <wbruna@softwareexpress.com.br>
parents: 11457
diff changeset
523
20964
a939eeb94833 http: reuse authentication info after the first failed request (issue3567)
Stéphane Klein <contact@stephane-klein.info>
parents: 20291
diff changeset
524 def retry_http_basic_auth(self, host, req, realm):
34466
1232f7fa00c3 cleanup: use urllibcompat for renamed methods on urllib request objects
Augie Fackler <augie@google.com>
parents: 34428
diff changeset
525 user, pw = self.passwd.find_user_password(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
526 realm, urllibcompat.getfullurl(req)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
527 )
20964
a939eeb94833 http: reuse authentication info after the first failed request (issue3567)
Stéphane Klein <contact@stephane-klein.info>
parents: 20291
diff changeset
528 if pw is not None:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
529 raw = b"%s:%s" % (pycompat.bytesurl(user), pycompat.bytesurl(pw))
43506
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43106
diff changeset
530 auth = 'Basic %s' % pycompat.strurl(base64.b64encode(raw).strip())
29639
6fd751fa58d3 url: avoid re-issuing incorrect password (issue3210)
Kim Randell <Kim.Randell@vicon.com>
parents: 29599
diff changeset
531 if req.get_header(self.auth_header, None) == auth:
20964
a939eeb94833 http: reuse authentication info after the first failed request (issue3567)
Stéphane Klein <contact@stephane-klein.info>
parents: 20291
diff changeset
532 return None
a939eeb94833 http: reuse authentication info after the first failed request (issue3567)
Stéphane Klein <contact@stephane-klein.info>
parents: 20291
diff changeset
533 self.auth = auth
a939eeb94833 http: reuse authentication info after the first failed request (issue3567)
Stéphane Klein <contact@stephane-klein.info>
parents: 20291
diff changeset
534 req.add_unredirected_header(self.auth_header, auth)
a939eeb94833 http: reuse authentication info after the first failed request (issue3567)
Stéphane Klein <contact@stephane-klein.info>
parents: 20291
diff changeset
535 return self.parent.open(req)
a939eeb94833 http: reuse authentication info after the first failed request (issue3567)
Stéphane Klein <contact@stephane-klein.info>
parents: 20291
diff changeset
536 else:
a939eeb94833 http: reuse authentication info after the first failed request (issue3567)
Stéphane Klein <contact@stephane-klein.info>
parents: 20291
diff changeset
537 return None
a939eeb94833 http: reuse authentication info after the first failed request (issue3567)
Stéphane Klein <contact@stephane-klein.info>
parents: 20291
diff changeset
538
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
539
31936
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
540 class cookiehandler(urlreq.basehandler):
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
541 def __init__(self, ui):
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
542 self.cookiejar = None
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
543
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
544 cookiefile = ui.config(b'auth', b'cookiefile')
31936
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
545 if not cookiefile:
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
546 return
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
547
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
548 cookiefile = util.expandpath(cookiefile)
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
549 try:
37735
126998dcfb08 url: some bytes/str cleanup where we interface with stdlib funcs
Augie Fackler <augie@google.com>
parents: 37100
diff changeset
550 cookiejar = util.cookielib.MozillaCookieJar(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
551 pycompat.fsdecode(cookiefile)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
552 )
31936
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
553 cookiejar.load()
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
554 self.cookiejar = cookiejar
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
555 except util.cookielib.LoadError as e:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
556 ui.warn(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
557 _(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
558 b'(error loading cookie file %s: %s; continuing without '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
559 b'cookies)\n'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
560 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
561 % (cookiefile, stringutil.forcebytestr(e))
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
562 )
31936
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
563
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
564 def http_request(self, request):
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
565 if self.cookiejar:
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
566 self.cookiejar.add_cookie_header(request)
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
567
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
568 return request
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
569
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
570 def https_request(self, request):
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
571 if self.cookiejar:
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
572 self.cookiejar.add_cookie_header(request)
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
573
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
574 return request
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
575
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
576
9347
d0474b184347 url: add support for custom handlers in extensions
Henrik Stuart <hg@hstuart.dk>
parents: 9122
diff changeset
577 handlerfuncs = []
d0474b184347 url: add support for custom handlers in extensions
Henrik Stuart <hg@hstuart.dk>
parents: 9122
diff changeset
578
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
579
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
580 def opener(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
581 ui,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
582 authinfo=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
583 useragent=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
584 loggingfh=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
585 loggingname=b's',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
586 loggingopts=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
587 sendaccept=True,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
588 ):
45942
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45759
diff changeset
589 """
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
590 construct an opener suitable for urllib2
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
591 authinfo will be added to the password manager
37011
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
592
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
593 The opener can be configured to log socket events if the various
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
594 ``logging*`` arguments are specified.
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
595
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
596 ``loggingfh`` denotes a file object to log events to.
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
597 ``loggingname`` denotes the name of the to print when logging.
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
598 ``loggingopts`` is a dict of keyword arguments to pass to the constructed
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
599 ``util.socketobserver`` instance.
37045
a708e1e4d7a8 url: support suppressing Accept header
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37011
diff changeset
600
a708e1e4d7a8 url: support suppressing Accept header
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37011
diff changeset
601 ``sendaccept`` allows controlling whether the ``Accept`` request header
a708e1e4d7a8 url: support suppressing Accept header
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37011
diff changeset
602 is sent. The header is sent by default.
45942
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45759
diff changeset
603 """
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
604 timeout = ui.configwith(float, b'http', b'timeout')
37011
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
605 handlers = []
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
606
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
607 if loggingfh:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
608 handlers.append(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
609 logginghttphandler(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
610 loggingfh, loggingname, loggingopts or {}, timeout=timeout
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
611 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
612 )
37011
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
613 # We don't yet support HTTPS when logging I/O. If we attempt to open
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
614 # an HTTPS URL, we'll likely fail due to unknown protocol.
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
615
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
616 else:
40043
6509fcec830c url: allow to configure timeout on http connection
Cédric Krier <ced@b2ck.com>
parents: 40034
diff changeset
617 handlers.append(httphandler(timeout=timeout))
37011
02221d6fb041 url: add HTTP handler that uses a proxied socket
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36652
diff changeset
618 if has_https:
40043
6509fcec830c url: allow to configure timeout on http connection
Cédric Krier <ced@b2ck.com>
parents: 40034
diff changeset
619 handlers.append(httpshandler(ui, timeout=timeout))
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
620
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
621 handlers.append(proxyhandler(ui))
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
622
29378
fea71f66ebff url: remember http password database in ui object
liscju <piotr.listkiewicz@gmail.com>
parents: 29377
diff changeset
623 passmgr = passwordmgr(ui, ui.httppasswordmgrdb)
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
624 if authinfo is not None:
29379
fc777c855d66 largefiles: make cloning not ask two times about password (issue4883)
liscju <piotr.listkiewicz@gmail.com>
parents: 29378
diff changeset
625 realm, uris, user, passwd = authinfo
fc777c855d66 largefiles: make cloning not ask two times about password (issue4883)
liscju <piotr.listkiewicz@gmail.com>
parents: 29378
diff changeset
626 saveduser, savedpass = passmgr.find_stored_password(uris[0])
fc777c855d66 largefiles: make cloning not ask two times about password (issue4883)
liscju <piotr.listkiewicz@gmail.com>
parents: 29378
diff changeset
627 if user != saveduser or passwd:
fc777c855d66 largefiles: make cloning not ask two times about password (issue4883)
liscju <piotr.listkiewicz@gmail.com>
parents: 29378
diff changeset
628 passmgr.add_password(realm, uris, user, passwd)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
629 ui.debug(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
630 b'http auth: user %s, password %s\n'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
631 % (user, passwd and b'*' * len(passwd) or b'not set')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
632 )
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
633
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
634 handlers.extend(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
635 (httpbasicauthhandler(passmgr), httpdigestauthhandler(passmgr))
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
636 )
9347
d0474b184347 url: add support for custom handlers in extensions
Henrik Stuart <hg@hstuart.dk>
parents: 9122
diff changeset
637 handlers.extend([h(ui, passmgr) for h in handlerfuncs])
31936
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30820
diff changeset
638 handlers.append(cookiehandler(ui))
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28861
diff changeset
639 opener = urlreq.buildopener(*handlers)
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
640
40034
393e44324037 httppeer: report http statistics
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40031
diff changeset
641 # keepalive.py's handlers will populate these attributes if they exist.
393e44324037 httppeer: report http statistics
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40031
diff changeset
642 opener.requestscount = 0
393e44324037 httppeer: report http statistics
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40031
diff changeset
643 opener.sentbytescount = 0
393e44324037 httppeer: report http statistics
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40031
diff changeset
644 opener.receivedbytescount = 0
393e44324037 httppeer: report http statistics
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40031
diff changeset
645
29589
486de14eb394 url: add distribution and version to user-agent request header (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29455
diff changeset
646 # The user agent should should *NOT* be used by servers for e.g.
486de14eb394 url: add distribution and version to user-agent request header (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29455
diff changeset
647 # protocol detection or feature negotiation: there are other
486de14eb394 url: add distribution and version to user-agent request header (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29455
diff changeset
648 # facilities for that.
486de14eb394 url: add distribution and version to user-agent request header (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29455
diff changeset
649 #
486de14eb394 url: add distribution and version to user-agent request header (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29455
diff changeset
650 # "mercurial/proto-1.0" was the original user agent string and
486de14eb394 url: add distribution and version to user-agent request header (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29455
diff changeset
651 # exists for backwards compatibility reasons.
486de14eb394 url: add distribution and version to user-agent request header (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29455
diff changeset
652 #
486de14eb394 url: add distribution and version to user-agent request header (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29455
diff changeset
653 # The "(Mercurial %s)" string contains the distribution
486de14eb394 url: add distribution and version to user-agent request header (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29455
diff changeset
654 # name and version. Other client implementations should choose their
486de14eb394 url: add distribution and version to user-agent request header (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29455
diff changeset
655 # own distribution name. Since servers should not be using the user
486de14eb394 url: add distribution and version to user-agent request header (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29455
diff changeset
656 # agent string for anything, clients should be able to define whatever
486de14eb394 url: add distribution and version to user-agent request header (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29455
diff changeset
657 # user agent they deem appropriate.
35439
e7bb5fc4570c lfs: add git to the User-Agent header for blob transfers
Matt Harbison <matt_harbison@yahoo.com>
parents: 34694
diff changeset
658 #
e7bb5fc4570c lfs: add git to the User-Agent header for blob transfers
Matt Harbison <matt_harbison@yahoo.com>
parents: 34694
diff changeset
659 # The custom user agent is for lfs, because unfortunately some servers
e7bb5fc4570c lfs: add git to the User-Agent header for blob transfers
Matt Harbison <matt_harbison@yahoo.com>
parents: 34694
diff changeset
660 # do look at this value.
e7bb5fc4570c lfs: add git to the User-Agent header for blob transfers
Matt Harbison <matt_harbison@yahoo.com>
parents: 34694
diff changeset
661 if not useragent:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
662 agent = b'mercurial/proto-1.0 (Mercurial %s)' % util.version()
43506
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43106
diff changeset
663 opener.addheaders = [('User-agent', pycompat.sysstr(agent))]
35439
e7bb5fc4570c lfs: add git to the User-Agent header for blob transfers
Matt Harbison <matt_harbison@yahoo.com>
parents: 34694
diff changeset
664 else:
43506
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43106
diff changeset
665 opener.addheaders = [('User-agent', pycompat.sysstr(useragent))]
30763
a520aefb96f1 httppeer: advertise and support application/mercurial-0.2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30664
diff changeset
666
a520aefb96f1 httppeer: advertise and support application/mercurial-0.2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30664
diff changeset
667 # This header should only be needed by wire protocol requests. But it has
a520aefb96f1 httppeer: advertise and support application/mercurial-0.2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30664
diff changeset
668 # been sent on all requests since forever. We keep sending it for backwards
a520aefb96f1 httppeer: advertise and support application/mercurial-0.2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30664
diff changeset
669 # compatibility reasons. Modern versions of the wire protocol use
a520aefb96f1 httppeer: advertise and support application/mercurial-0.2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30664
diff changeset
670 # X-HgProto-<N> for advertising client support.
37045
a708e1e4d7a8 url: support suppressing Accept header
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37011
diff changeset
671 if sendaccept:
43506
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43106
diff changeset
672 opener.addheaders.append(('Accept', 'application/mercurial-0.1'))
37045
a708e1e4d7a8 url: support suppressing Accept header
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37011
diff changeset
673
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
674 return opener
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
diff changeset
675
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
676
42109
2a3c0106ded9 import: suppress accept: header
timeless <timeless@mozdev.org>
parents: 41711
diff changeset
677 def open(ui, url_, data=None, sendaccept=True):
14076
924c82157d46 url: move URL parsing functions into util to improve startup time
Brodie Rao <brodie@bitheap.org>
parents: 14071
diff changeset
678 u = util.url(url_)
13818
bf6156bab41b url: use url.url in url.open()
Brodie Rao <brodie@bitheap.org>
parents: 13817
diff changeset
679 if u.scheme:
bf6156bab41b url: use url.url in url.open()
Brodie Rao <brodie@bitheap.org>
parents: 13817
diff changeset
680 u.scheme = u.scheme.lower()
bf6156bab41b url: use url.url in url.open()
Brodie Rao <brodie@bitheap.org>
parents: 13817
diff changeset
681 url_, authinfo = u.authinfo()
bf6156bab41b url: use url.url in url.open()
Brodie Rao <brodie@bitheap.org>
parents: 13817
diff changeset
682 else:
bf6156bab41b url: use url.url in url.open()
Brodie Rao <brodie@bitheap.org>
parents: 13817
diff changeset
683 path = util.normpath(os.path.abspath(url_))
44641
e74af49aa3c9 url: pass str to pathname2url
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44452
diff changeset
684 url_ = b'file://' + pycompat.bytesurl(
e74af49aa3c9 url: pass str to pathname2url
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44452
diff changeset
685 urlreq.pathname2url(pycompat.fsdecode(path))
e74af49aa3c9 url: pass str to pathname2url
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44452
diff changeset
686 )
7284
ac81ffac0f35 url: fix file:// URL handling
Patrick Mezard <pmezard@gmail.com>
parents: 7270
diff changeset
687 authinfo = None
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
688 return opener(ui, authinfo, sendaccept=sendaccept).open(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
689 pycompat.strurl(url_), data
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
690 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
691
40018
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
692
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
693 def wrapresponse(resp):
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
694 """Wrap a response object with common error handlers.
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
695
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
696 This ensures that any I/O from any consumer raises the appropriate
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
697 error and messaging.
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
698 """
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
699 origread = resp.read
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
700
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
701 class readerproxy(resp.__class__):
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
702 def read(self, size=None):
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
703 try:
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
704 return origread(size)
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
705 except httplib.IncompleteRead as e:
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
706 # e.expected is an integer if length known or None otherwise.
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
707 if e.expected:
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
708 got = len(e.partial)
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
709 total = e.expected + got
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
710 msg = _(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
711 b'HTTP request error (incomplete response; '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
712 b'expected %d bytes got %d)'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
713 ) % (total, got)
40018
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
714 else:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
715 msg = _(b'HTTP request error (incomplete response)')
40018
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
716
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
717 raise error.PeerTransportError(
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
718 msg,
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
719 hint=_(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
720 b'this may be an intermittent network failure; '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
721 b'if the error persists, consider contacting the '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
722 b'network or server operator'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
723 ),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
724 )
40018
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
725 except httplib.HTTPException as e:
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
726 raise error.PeerTransportError(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
727 _(b'HTTP request error (%s)') % e,
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
728 hint=_(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
729 b'this may be an intermittent network failure; '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
730 b'if the error persists, consider contacting the '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
731 b'network or server operator'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
732 ),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42109
diff changeset
733 )
40018
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
734
f80db6adabbe url: move _wraphttpresponse() from httpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37735
diff changeset
735 resp.__class__ = readerproxy