annotate mercurial/httpclient/socketutil.py @ 26061:be8a4e0800d8

reachableroots: use baseset lazy sorting smartset sorting is lazy (so faster in some case) and better (informs that the set is sorted allowing some optimisation). So we rely on it directly. Some test output are updated because we now have more information (ordering).
author Pierre-Yves David <pierre-yves.david@fb.com>
date Thu, 20 Aug 2015 17:23:21 -0700
parents 328739ea70c3
children 1ad9da968a2e
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
14243
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
1 # Copyright 2010, Google Inc.
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
2 # All rights reserved.
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
3 #
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
4 # Redistribution and use in source and binary forms, with or without
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
5 # modification, are permitted provided that the following conditions are
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
6 # met:
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
7 #
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
8 # * Redistributions of source code must retain the above copyright
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
9 # notice, this list of conditions and the following disclaimer.
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
10 # * Redistributions in binary form must reproduce the above
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
11 # copyright notice, this list of conditions and the following disclaimer
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
12 # in the documentation and/or other materials provided with the
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
13 # distribution.
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
14 # * Neither the name of Google Inc. nor the names of its
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
15 # contributors may be used to endorse or promote products derived from
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
16 # this software without specific prior written permission.
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
17
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
18 # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
19 # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
20 # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
21 # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
22 # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
23 # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
24 # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
25 # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
26 # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
27 # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
28 # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
29 """Abstraction to simplify socket use for Python < 2.6
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
30
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
31 This will attempt to use the ssl module and the new
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
32 socket.create_connection method, but fall back to the old
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
33 methods if those are unavailable.
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
34 """
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
35 import logging
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
36 import socket
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
37
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
38 logger = logging.getLogger(__name__)
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
39
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
40 try:
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
41 import ssl
19182
fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents: 18179
diff changeset
42 # make demandimporters load the module
fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents: 18179
diff changeset
43 ssl.wrap_socket # pylint: disable=W0104
14243
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
44 have_ssl = True
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
45 except ImportError:
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
46 import httplib
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
47 import urllib2
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
48 have_ssl = getattr(urllib2, 'HTTPSHandler', False)
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
49 ssl = False
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
50
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
51
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
52 try:
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
53 create_connection = socket.create_connection
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
54 except AttributeError:
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
55 def create_connection(address):
19182
fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents: 18179
diff changeset
56 """Backport of socket.create_connection from Python 2.6."""
14243
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
57 host, port = address
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
58 msg = "getaddrinfo returns an empty list"
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
59 sock = None
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
60 for res in socket.getaddrinfo(host, port, 0,
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
61 socket.SOCK_STREAM):
19182
fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents: 18179
diff changeset
62 af, socktype, proto, unused_canonname, sa = res
14243
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
63 try:
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
64 sock = socket.socket(af, socktype, proto)
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
65 logger.info("connect: (%s, %s)", host, port)
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
66 sock.connect(sa)
25660
328739ea70c3 global: mass rewrite to use modern exception syntax
Gregory Szorc <gregory.szorc@gmail.com>
parents: 19748
diff changeset
67 except socket.error as msg:
14243
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
68 logger.info('connect fail: %s %s', host, port)
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
69 if sock:
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
70 sock.close()
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
71 sock = None
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
72 continue
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
73 break
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
74 if not sock:
18179
f614543733b6 httpclient: apply change df9aea1def3e: remove use of two-argument raise
Augie Fackler <raf@durin42.com>
parents: 14990
diff changeset
75 raise socket.error(msg)
14243
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
76 return sock
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
77
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
78 if ssl:
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
79 wrap_socket = ssl.wrap_socket
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
80 CERT_NONE = ssl.CERT_NONE
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
81 CERT_OPTIONAL = ssl.CERT_OPTIONAL
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
82 CERT_REQUIRED = ssl.CERT_REQUIRED
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
83 else:
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
84 class FakeSocket(httplib.FakeSocket):
19182
fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents: 18179
diff changeset
85 """Socket wrapper that supports SSL."""
fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents: 18179
diff changeset
86
fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents: 18179
diff changeset
87 # Silence lint about this goofy backport class
fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents: 18179
diff changeset
88 # pylint: disable=W0232,E1101,R0903,R0913,C0111
fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents: 18179
diff changeset
89
14243
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
90 # backport the behavior from Python 2.6, which is to busy wait
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
91 # on the socket instead of anything nice. Sigh.
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
92 # See http://bugs.python.org/issue3890 for more info.
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
93 def recv(self, buflen=1024, flags=0):
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
94 """ssl-aware wrapper around socket.recv
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
95 """
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
96 if flags != 0:
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
97 raise ValueError(
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
98 "non-zero flags not allowed in calls to recv() on %s" %
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
99 self.__class__)
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
100 while True:
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
101 try:
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
102 return self._ssl.read(buflen)
25660
328739ea70c3 global: mass rewrite to use modern exception syntax
Gregory Szorc <gregory.szorc@gmail.com>
parents: 19748
diff changeset
103 except socket.sslerror as x:
14243
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
104 if x.args[0] == socket.SSL_ERROR_WANT_READ:
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
105 continue
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
106 else:
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
107 raise x
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
108
19748
59df9e52b5bb httpclient: apply upstream revision da7579b034a4 to fix SSL problems (issue4038)
Augie Fackler <raf@durin42.com>
parents: 19489
diff changeset
109 _PROTOCOL_SSLv23 = 2
14243
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
110
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
111 CERT_NONE = 0
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
112 CERT_OPTIONAL = 1
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
113 CERT_REQUIRED = 2
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
114
19182
fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents: 18179
diff changeset
115 # Disable unused-argument because we're making a dumb wrapper
fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents: 18179
diff changeset
116 # that's like an upstream method.
fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents: 18179
diff changeset
117 #
fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents: 18179
diff changeset
118 # pylint: disable=W0613,R0913
14243
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
119 def wrap_socket(sock, keyfile=None, certfile=None,
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
120 server_side=False, cert_reqs=CERT_NONE,
19748
59df9e52b5bb httpclient: apply upstream revision da7579b034a4 to fix SSL problems (issue4038)
Augie Fackler <raf@durin42.com>
parents: 19489
diff changeset
121 ssl_version=_PROTOCOL_SSLv23, ca_certs=None,
14243
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
122 do_handshake_on_connect=True,
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
123 suppress_ragged_eofs=True):
19182
fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents: 18179
diff changeset
124 """Backport of ssl.wrap_socket from Python 2.6."""
14243
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
125 if cert_reqs != CERT_NONE and ca_certs:
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
126 raise CertificateValidationUnsupported(
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
127 'SSL certificate validation requires the ssl module'
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
128 '(included in Python 2.6 and later.)')
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
129 sslob = socket.ssl(sock)
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
130 # borrow httplib's workaround for no ssl.wrap_socket
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
131 sock = FakeSocket(sock, sslob)
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
132 return sock
19182
fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents: 18179
diff changeset
133 # pylint: enable=W0613,R0913
14243
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
134
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
135
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
136 class CertificateValidationUnsupported(Exception):
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
137 """Exception raised when cert validation is requested but unavailable."""
861f28212398 Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
138 # no-check-code