annotate mercurial/httpclient/socketutil.py @ 22328:fba8c1a4ce21

obsolete: extract the part of _readmarkers specific to format version 0 If we are to introduce new formats we need to be able call different functions for different formats. Creating a function for format version 0 is the first step.
author Pierre-Yves David <pierre-yves.david@fb.com>
date Mon, 25 Aug 2014 14:58:11 +0200
parents 59df9e52b5bb
children 328739ea70c3
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
14243
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
1 # Copyright 2010, Google Inc.
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
2 # All rights reserved.
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
3 #
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
4 # Redistribution and use in source and binary forms, with or without
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
5 # modification, are permitted provided that the following conditions are
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
6 # met:
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
7 #
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
8 # * Redistributions of source code must retain the above copyright
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
9 # notice, this list of conditions and the following disclaimer.
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
10 # * Redistributions in binary form must reproduce the above
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
11 # copyright notice, this list of conditions and the following disclaimer
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
12 # in the documentation and/or other materials provided with the
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
13 # distribution.
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
14 # * Neither the name of Google Inc. nor the names of its
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
15 # contributors may be used to endorse or promote products derived from
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
16 # this software without specific prior written permission.
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
17
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
18 # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
19 # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
20 # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
21 # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
22 # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
23 # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
24 # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
25 # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
26 # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
27 # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
28 # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
29 """Abstraction to simplify socket use for Python < 2.6
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
30
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
31 This will attempt to use the ssl module and the new
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
32 socket.create_connection method, but fall back to the old
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
33 methods if those are unavailable.
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
34 """
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
35 import logging
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
36 import socket
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
37
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
38 logger = logging.getLogger(__name__)
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
39
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
40 try:
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
41 import ssl
19182
fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents: 18179
diff changeset
42 # make demandimporters load the module
fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents: 18179
diff changeset
43 ssl.wrap_socket # pylint: disable=W0104
14243
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
44 have_ssl = True
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
45 except ImportError:
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
46 import httplib
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
47 import urllib2
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
48 have_ssl = getattr(urllib2, 'HTTPSHandler', False)
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
49 ssl = False
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
50
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
51
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
52 try:
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
53 create_connection = socket.create_connection
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
54 except AttributeError:
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
55 def create_connection(address):
19182
fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents: 18179
diff changeset
56 """Backport of socket.create_connection from Python 2.6."""
14243
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
57 host, port = address
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
58 msg = "getaddrinfo returns an empty list"
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
59 sock = None
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
60 for res in socket.getaddrinfo(host, port, 0,
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
61 socket.SOCK_STREAM):
19182
fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents: 18179
diff changeset
62 af, socktype, proto, unused_canonname, sa = res
14243
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
63 try:
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
64 sock = socket.socket(af, socktype, proto)
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
65 logger.info("connect: (%s, %s)", host, port)
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
66 sock.connect(sa)
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
67 except socket.error, msg:
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
68 logger.info('connect fail: %s %s', host, port)
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
69 if sock:
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
70 sock.close()
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
71 sock = None
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
72 continue
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
73 break
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
74 if not sock:
18179
f614543733b6 httpclient: apply change df9aea1def3e: remove use of two-argument raise
Augie Fackler <raf@durin42.com>
parents: 14990
diff changeset
75 raise socket.error(msg)
14243
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
76 return sock
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
77
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
78 if ssl:
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
79 wrap_socket = ssl.wrap_socket
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
80 CERT_NONE = ssl.CERT_NONE
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
81 CERT_OPTIONAL = ssl.CERT_OPTIONAL
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
82 CERT_REQUIRED = ssl.CERT_REQUIRED
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
83 else:
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
84 class FakeSocket(httplib.FakeSocket):
19182
fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents: 18179
diff changeset
85 """Socket wrapper that supports SSL."""
fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents: 18179
diff changeset
86
fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents: 18179
diff changeset
87 # Silence lint about this goofy backport class
fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents: 18179
diff changeset
88 # pylint: disable=W0232,E1101,R0903,R0913,C0111
fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents: 18179
diff changeset
89
14243
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
90 # backport the behavior from Python 2.6, which is to busy wait
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
91 # on the socket instead of anything nice. Sigh.
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
92 # See http://bugs.python.org/issue3890 for more info.
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
93 def recv(self, buflen=1024, flags=0):
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
94 """ssl-aware wrapper around socket.recv
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
95 """
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
96 if flags != 0:
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
97 raise ValueError(
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
98 "non-zero flags not allowed in calls to recv() on %s" %
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
99 self.__class__)
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
100 while True:
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
101 try:
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
102 return self._ssl.read(buflen)
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
103 except socket.sslerror, x:
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
104 if x.args[0] == socket.SSL_ERROR_WANT_READ:
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
105 continue
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
106 else:
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
107 raise x
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
108
19748
59df9e52b5bb httpclient: apply upstream revision da7579b034a4 to fix SSL problems (issue4038)
Augie Fackler <raf@durin42.com>
parents: 19489
diff changeset
109 _PROTOCOL_SSLv23 = 2
14243
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
110
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
111 CERT_NONE = 0
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
112 CERT_OPTIONAL = 1
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
113 CERT_REQUIRED = 2
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
114
19182
fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents: 18179
diff changeset
115 # Disable unused-argument because we're making a dumb wrapper
fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents: 18179
diff changeset
116 # that's like an upstream method.
fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents: 18179
diff changeset
117 #
fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents: 18179
diff changeset
118 # pylint: disable=W0613,R0913
14243
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
119 def wrap_socket(sock, keyfile=None, certfile=None,
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
120 server_side=False, cert_reqs=CERT_NONE,
19748
59df9e52b5bb httpclient: apply upstream revision da7579b034a4 to fix SSL problems (issue4038)
Augie Fackler <raf@durin42.com>
parents: 19489
diff changeset
121 ssl_version=_PROTOCOL_SSLv23, ca_certs=None,
14243
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
122 do_handshake_on_connect=True,
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
123 suppress_ragged_eofs=True):
19182
fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents: 18179
diff changeset
124 """Backport of ssl.wrap_socket from Python 2.6."""
14243
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
125 if cert_reqs != CERT_NONE and ca_certs:
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
126 raise CertificateValidationUnsupported(
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
127 'SSL certificate validation requires the ssl module'
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
128 '(included in Python 2.6 and later.)')
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
129 sslob = socket.ssl(sock)
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
130 # borrow httplib's workaround for no ssl.wrap_socket
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
131 sock = FakeSocket(sock, sslob)
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
132 return sock
19182
fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents: 18179
diff changeset
133 # pylint: enable=W0613,R0913
14243
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
134
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
135
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
136 class CertificateValidationUnsupported(Exception):
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
137 """Exception raised when cert validation is requested but unavailable."""
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
138 # no-check-code