Mercurial > hg
annotate mercurial/httpclient/__init__.py @ 28238:b57d45ec33b2
test: update test-bundle2-format.t comment
Now that bundle2 is used by default for all exchanges, this comment is obviously
out of date. Having deep testing of the API and expected behavior of the format
and its processing is still valuable, so the comment is updated.
author | Pierre-Yves David <pierre-yves.david@fb.com> |
---|---|
date | Thu, 25 Feb 2016 10:34:31 +0100 |
parents | 1ad9da968a2e |
children | 86db5cb55d46 |
rev | line source |
---|---|
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
1 # Copyright 2010, Google Inc. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
2 # All rights reserved. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
3 # |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
4 # Redistribution and use in source and binary forms, with or without |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
5 # modification, are permitted provided that the following conditions are |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
6 # met: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
7 # |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
8 # * Redistributions of source code must retain the above copyright |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
9 # notice, this list of conditions and the following disclaimer. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
10 # * Redistributions in binary form must reproduce the above |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
11 # copyright notice, this list of conditions and the following disclaimer |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
12 # in the documentation and/or other materials provided with the |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
13 # distribution. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
14 # * Neither the name of Google Inc. nor the names of its |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
15 # contributors may be used to endorse or promote products derived from |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
16 # this software without specific prior written permission. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
17 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
18 # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
19 # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
20 # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
21 # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
22 # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
23 # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
24 # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
25 # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
26 # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
27 # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
28 # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
29 """Improved HTTP/1.1 client library |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
30 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
31 This library contains an HTTPConnection which is similar to the one in |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
32 httplib, but has several additional features: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
33 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
34 * supports keepalives natively |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
35 * uses select() to block for incoming data |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
36 * notices when the server responds early to a request |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
37 * implements ssl inline instead of in a different class |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
38 """ |
27601
1ad9da968a2e
httpclient: update to 938f2107d6e2 of httpplus
Augie Fackler <augie@google.com>
parents:
25660
diff
changeset
|
39 from __future__ import absolute_import |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
40 |
19182
fae47ecaa952
httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents:
19038
diff
changeset
|
41 # Many functions in this file have too many arguments. |
fae47ecaa952
httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents:
19038
diff
changeset
|
42 # pylint: disable=R0913 |
fae47ecaa952
httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents:
19038
diff
changeset
|
43 |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
44 import cStringIO |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
45 import errno |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
46 import httplib |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
47 import logging |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
48 import rfc822 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
49 import select |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
50 import socket |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
51 |
27601
1ad9da968a2e
httpclient: update to 938f2107d6e2 of httpplus
Augie Fackler <augie@google.com>
parents:
25660
diff
changeset
|
52 from . import ( |
1ad9da968a2e
httpclient: update to 938f2107d6e2 of httpplus
Augie Fackler <augie@google.com>
parents:
25660
diff
changeset
|
53 _readers, |
1ad9da968a2e
httpclient: update to 938f2107d6e2 of httpplus
Augie Fackler <augie@google.com>
parents:
25660
diff
changeset
|
54 socketutil, |
1ad9da968a2e
httpclient: update to 938f2107d6e2 of httpplus
Augie Fackler <augie@google.com>
parents:
25660
diff
changeset
|
55 ) |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
56 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
57 logger = logging.getLogger(__name__) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
58 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
59 __all__ = ['HTTPConnection', 'HTTPResponse'] |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
60 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
61 HTTP_VER_1_0 = 'HTTP/1.0' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
62 HTTP_VER_1_1 = 'HTTP/1.1' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
63 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
64 OUTGOING_BUFFER_SIZE = 1 << 15 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
65 INCOMING_BUFFER_SIZE = 1 << 20 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
66 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
67 HDR_ACCEPT_ENCODING = 'accept-encoding' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
68 HDR_CONNECTION_CTRL = 'connection' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
69 HDR_CONTENT_LENGTH = 'content-length' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
70 HDR_XFER_ENCODING = 'transfer-encoding' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
71 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
72 XFER_ENCODING_CHUNKED = 'chunked' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
73 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
74 CONNECTION_CLOSE = 'close' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
75 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
76 EOL = '\r\n' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
77 _END_HEADERS = EOL * 2 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
78 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
79 # Based on some searching around, 1 second seems like a reasonable |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
80 # default here. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
81 TIMEOUT_ASSUME_CONTINUE = 1 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
82 TIMEOUT_DEFAULT = None |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
83 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
84 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
85 class HTTPResponse(object): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
86 """Response from an HTTP server. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
87 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
88 The response will continue to load as available. If you need the |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
89 complete response before continuing, check the .complete() method. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
90 """ |
16643
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
91 def __init__(self, sock, timeout, method): |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
92 self.sock = sock |
16643
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
93 self.method = method |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
94 self.raw_response = '' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
95 self._headers_len = 0 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
96 self.headers = None |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
97 self.will_close = False |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
98 self.status_line = '' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
99 self.status = None |
16643
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
100 self.continued = False |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
101 self.http_version = None |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
102 self.reason = None |
16643
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
103 self._reader = None |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
104 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
105 self._read_location = 0 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
106 self._eol = EOL |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
107 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
108 self._timeout = timeout |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
109 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
110 @property |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
111 def _end_headers(self): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
112 return self._eol * 2 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
113 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
114 def complete(self): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
115 """Returns true if this response is completely loaded. |
14376
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
116 |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
117 Note that if this is a connection where complete means the |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
118 socket is closed, this will nearly always return False, even |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
119 in cases where all the data has actually been loaded. |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
120 """ |
16643
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
121 if self._reader: |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
122 return self._reader.done() |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
123 |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
124 def _close(self): |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
125 if self._reader is not None: |
19182
fae47ecaa952
httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents:
19038
diff
changeset
|
126 # We're a friend of the reader class here. |
fae47ecaa952
httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents:
19038
diff
changeset
|
127 # pylint: disable=W0212 |
16643
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
128 self._reader._close() |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
129 |
27601
1ad9da968a2e
httpclient: update to 938f2107d6e2 of httpplus
Augie Fackler <augie@google.com>
parents:
25660
diff
changeset
|
130 def getheader(self, header, default=None): |
1ad9da968a2e
httpclient: update to 938f2107d6e2 of httpplus
Augie Fackler <augie@google.com>
parents:
25660
diff
changeset
|
131 return self.headers.getheader(header, default=default) |
1ad9da968a2e
httpclient: update to 938f2107d6e2 of httpplus
Augie Fackler <augie@google.com>
parents:
25660
diff
changeset
|
132 |
1ad9da968a2e
httpclient: update to 938f2107d6e2 of httpplus
Augie Fackler <augie@google.com>
parents:
25660
diff
changeset
|
133 def getheaders(self): |
1ad9da968a2e
httpclient: update to 938f2107d6e2 of httpplus
Augie Fackler <augie@google.com>
parents:
25660
diff
changeset
|
134 return self.headers.items() |
1ad9da968a2e
httpclient: update to 938f2107d6e2 of httpplus
Augie Fackler <augie@google.com>
parents:
25660
diff
changeset
|
135 |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
136 def readline(self): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
137 """Read a single line from the response body. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
138 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
139 This may block until either a line ending is found or the |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
140 response is complete. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
141 """ |
19038 | 142 blocks = [] |
143 while True: | |
144 self._reader.readto('\n', blocks) | |
145 | |
146 if blocks and blocks[-1][-1] == '\n' or self.complete(): | |
147 break | |
148 | |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
149 self._select() |
19038 | 150 |
151 return ''.join(blocks) | |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
152 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
153 def read(self, length=None): |
19182
fae47ecaa952
httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents:
19038
diff
changeset
|
154 """Read data from the response body.""" |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
155 # if length is None, unbounded read |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
156 while (not self.complete() # never select on a finished read |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
157 and (not length # unbounded, so we wait for complete() |
16643
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
158 or length > self._reader.available_data)): |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
159 self._select() |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
160 if not length: |
16643
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
161 length = self._reader.available_data |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
162 r = self._reader.read(length) |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
163 if self.complete() and self.will_close: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
164 self.sock.close() |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
165 return r |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
166 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
167 def _select(self): |
19182
fae47ecaa952
httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents:
19038
diff
changeset
|
168 r, unused_write, unused_err = select.select( |
fae47ecaa952
httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents:
19038
diff
changeset
|
169 [self.sock], [], [], self._timeout) |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
170 if not r: |
16643
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
171 # socket was not readable. If the response is not |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
172 # complete, raise a timeout. |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
173 if not self.complete(): |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
174 logger.info('timed out with timeout of %s', self._timeout) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
175 raise HTTPTimeoutException('timeout reading data') |
14341
5c3de67e7402
httpclient: import revision b8c3511a8cae from py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14293
diff
changeset
|
176 try: |
5c3de67e7402
httpclient: import revision b8c3511a8cae from py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14293
diff
changeset
|
177 data = self.sock.recv(INCOMING_BUFFER_SIZE) |
25660
328739ea70c3
global: mass rewrite to use modern exception syntax
Gregory Szorc <gregory.szorc@gmail.com>
parents:
24306
diff
changeset
|
178 except socket.sslerror as e: |
14341
5c3de67e7402
httpclient: import revision b8c3511a8cae from py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14293
diff
changeset
|
179 if e.args[0] != socket.SSL_ERROR_WANT_READ: |
5c3de67e7402
httpclient: import revision b8c3511a8cae from py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14293
diff
changeset
|
180 raise |
17428
72803c8edaa4
avoid using abbreviations that look like spelling errors
Mads Kiilerich <mads@kiilerich.com>
parents:
17424
diff
changeset
|
181 logger.debug('SSL_ERROR_WANT_READ in _select, should retry later') |
14341
5c3de67e7402
httpclient: import revision b8c3511a8cae from py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14293
diff
changeset
|
182 return True |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
183 logger.debug('response read %d data during _select', len(data)) |
16643
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
184 # If the socket was readable and no data was read, that means |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
185 # the socket was closed. Inform the reader (if any) so it can |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
186 # raise an exception if this is an invalid situation. |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
187 if not data: |
16643
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
188 if self._reader: |
19182
fae47ecaa952
httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents:
19038
diff
changeset
|
189 # We're a friend of the reader class here. |
fae47ecaa952
httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents:
19038
diff
changeset
|
190 # pylint: disable=W0212 |
16643
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
191 self._reader._close() |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
192 return False |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
193 else: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
194 self._load_response(data) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
195 return True |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
196 |
19182
fae47ecaa952
httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents:
19038
diff
changeset
|
197 # This method gets replaced by _load later, which confuses pylint. |
fae47ecaa952
httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents:
19038
diff
changeset
|
198 def _load_response(self, data): # pylint: disable=E0202 |
16643
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
199 # Being here implies we're not at the end of the headers yet, |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
200 # since at the end of this method if headers were completely |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
201 # loaded we replace this method with the load() method of the |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
202 # reader we created. |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
203 self.raw_response += data |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
204 # This is a bogus server with bad line endings |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
205 if self._eol not in self.raw_response: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
206 for bad_eol in ('\n', '\r'): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
207 if (bad_eol in self.raw_response |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
208 # verify that bad_eol is not the end of the incoming data |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
209 # as this could be a response line that just got |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
210 # split between \r and \n. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
211 and (self.raw_response.index(bad_eol) < |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
212 (len(self.raw_response) - 1))): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
213 logger.info('bogus line endings detected, ' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
214 'using %r for EOL', bad_eol) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
215 self._eol = bad_eol |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
216 break |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
217 # exit early if not at end of headers |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
218 if self._end_headers not in self.raw_response or self.headers: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
219 return |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
220 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
221 # handle 100-continue response |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
222 hdrs, body = self.raw_response.split(self._end_headers, 1) |
19182
fae47ecaa952
httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents:
19038
diff
changeset
|
223 unused_http_ver, status = hdrs.split(' ', 1) |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
224 if status.startswith('100'): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
225 self.raw_response = body |
16643
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
226 self.continued = True |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
227 logger.debug('continue seen, setting body to %r', body) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
228 return |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
229 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
230 # arriving here means we should parse response headers |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
231 # as all headers have arrived completely |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
232 hdrs, body = self.raw_response.split(self._end_headers, 1) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
233 del self.raw_response |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
234 if self._eol in hdrs: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
235 self.status_line, hdrs = hdrs.split(self._eol, 1) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
236 else: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
237 self.status_line = hdrs |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
238 hdrs = '' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
239 # TODO HTTP < 1.0 support |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
240 (self.http_version, self.status, |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
241 self.reason) = self.status_line.split(' ', 2) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
242 self.status = int(self.status) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
243 if self._eol != EOL: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
244 hdrs = hdrs.replace(self._eol, '\r\n') |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
245 headers = rfc822.Message(cStringIO.StringIO(hdrs)) |
16643
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
246 content_len = None |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
247 if HDR_CONTENT_LENGTH in headers: |
16643
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
248 content_len = int(headers[HDR_CONTENT_LENGTH]) |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
249 if self.http_version == HTTP_VER_1_0: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
250 self.will_close = True |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
251 elif HDR_CONNECTION_CTRL in headers: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
252 self.will_close = ( |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
253 headers[HDR_CONNECTION_CTRL].lower() == CONNECTION_CLOSE) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
254 if (HDR_XFER_ENCODING in headers |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
255 and headers[HDR_XFER_ENCODING].lower() == XFER_ENCODING_CHUNKED): |
16643
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
256 self._reader = _readers.ChunkedReader(self._eol) |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
257 logger.debug('using a chunked reader') |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
258 else: |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
259 # HEAD responses are forbidden from returning a body, and |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
260 # it's implausible for a CONNECT response to use |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
261 # close-is-end logic for an OK response. |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
262 if (self.method == 'HEAD' or |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
263 (self.method == 'CONNECT' and content_len is None)): |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
264 content_len = 0 |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
265 if content_len is not None: |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
266 logger.debug('using a content-length reader with length %d', |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
267 content_len) |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
268 self._reader = _readers.ContentLengthReader(content_len) |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
269 else: |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
270 # Response body had no length specified and is not |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
271 # chunked, so the end of the body will only be |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
272 # identifiable by the termination of the socket by the |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
273 # server. My interpretation of the spec means that we |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
274 # are correct in hitting this case if |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
275 # transfer-encoding, content-length, and |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
276 # connection-control were left unspecified. |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
277 self._reader = _readers.CloseIsEndReader() |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
278 logger.debug('using a close-is-end reader') |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
279 self.will_close = True |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
280 |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
281 if body: |
19182
fae47ecaa952
httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents:
19038
diff
changeset
|
282 # We're a friend of the reader class here. |
fae47ecaa952
httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents:
19038
diff
changeset
|
283 # pylint: disable=W0212 |
16643
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
284 self._reader._load(body) |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
285 logger.debug('headers complete') |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
286 self.headers = headers |
19182
fae47ecaa952
httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents:
19038
diff
changeset
|
287 # We're a friend of the reader class here. |
fae47ecaa952
httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents:
19038
diff
changeset
|
288 # pylint: disable=W0212 |
16643
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
289 self._load_response = self._reader._load |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
290 |
27601
1ad9da968a2e
httpclient: update to 938f2107d6e2 of httpplus
Augie Fackler <augie@google.com>
parents:
25660
diff
changeset
|
291 def _foldheaders(headers): |
1ad9da968a2e
httpclient: update to 938f2107d6e2 of httpplus
Augie Fackler <augie@google.com>
parents:
25660
diff
changeset
|
292 """Given some headers, rework them so we can safely overwrite values. |
1ad9da968a2e
httpclient: update to 938f2107d6e2 of httpplus
Augie Fackler <augie@google.com>
parents:
25660
diff
changeset
|
293 |
1ad9da968a2e
httpclient: update to 938f2107d6e2 of httpplus
Augie Fackler <augie@google.com>
parents:
25660
diff
changeset
|
294 >>> _foldheaders({'Accept-Encoding': 'wat'}) |
1ad9da968a2e
httpclient: update to 938f2107d6e2 of httpplus
Augie Fackler <augie@google.com>
parents:
25660
diff
changeset
|
295 {'accept-encoding': ('Accept-Encoding', 'wat')} |
1ad9da968a2e
httpclient: update to 938f2107d6e2 of httpplus
Augie Fackler <augie@google.com>
parents:
25660
diff
changeset
|
296 """ |
1ad9da968a2e
httpclient: update to 938f2107d6e2 of httpplus
Augie Fackler <augie@google.com>
parents:
25660
diff
changeset
|
297 return dict((k.lower(), (k, v)) for k, v in headers.iteritems()) |
1ad9da968a2e
httpclient: update to 938f2107d6e2 of httpplus
Augie Fackler <augie@google.com>
parents:
25660
diff
changeset
|
298 |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
299 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
300 class HTTPConnection(object): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
301 """Connection to a single http server. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
302 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
303 Supports 100-continue and keepalives natively. Uses select() for |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
304 non-blocking socket operations. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
305 """ |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
306 http_version = HTTP_VER_1_1 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
307 response_class = HTTPResponse |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
308 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
309 def __init__(self, host, port=None, use_ssl=None, ssl_validator=None, |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
310 timeout=TIMEOUT_DEFAULT, |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
311 continue_timeout=TIMEOUT_ASSUME_CONTINUE, |
27601
1ad9da968a2e
httpclient: update to 938f2107d6e2 of httpplus
Augie Fackler <augie@google.com>
parents:
25660
diff
changeset
|
312 proxy_hostport=None, proxy_headers=None, |
1ad9da968a2e
httpclient: update to 938f2107d6e2 of httpplus
Augie Fackler <augie@google.com>
parents:
25660
diff
changeset
|
313 ssl_wrap_socket=None, **ssl_opts): |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
314 """Create a new HTTPConnection. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
315 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
316 Args: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
317 host: The host to which we'll connect. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
318 port: Optional. The port over which we'll connect. Default 80 for |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
319 non-ssl, 443 for ssl. |
17424
e7cfe3587ea4
fix trivial spelling errors
Mads Kiilerich <mads@kiilerich.com>
parents:
16774
diff
changeset
|
320 use_ssl: Optional. Whether to use ssl. Defaults to False if port is |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
321 not 443, true if port is 443. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
322 ssl_validator: a function(socket) to validate the ssl cert |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
323 timeout: Optional. Connection timeout, default is TIMEOUT_DEFAULT. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
324 continue_timeout: Optional. Timeout for waiting on an expected |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
325 "100 Continue" response. Default is TIMEOUT_ASSUME_CONTINUE. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
326 proxy_hostport: Optional. Tuple of (host, port) to use as an http |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
327 proxy for the connection. Default is to not use a proxy. |
27601
1ad9da968a2e
httpclient: update to 938f2107d6e2 of httpplus
Augie Fackler <augie@google.com>
parents:
25660
diff
changeset
|
328 proxy_headers: Optional dict of header keys and values to send to |
1ad9da968a2e
httpclient: update to 938f2107d6e2 of httpplus
Augie Fackler <augie@google.com>
parents:
25660
diff
changeset
|
329 a proxy when using CONNECT. For compatibility with |
1ad9da968a2e
httpclient: update to 938f2107d6e2 of httpplus
Augie Fackler <augie@google.com>
parents:
25660
diff
changeset
|
330 httplib, the Proxy-Authorization header may be |
1ad9da968a2e
httpclient: update to 938f2107d6e2 of httpplus
Augie Fackler <augie@google.com>
parents:
25660
diff
changeset
|
331 specified in headers for request(), which will clobber |
1ad9da968a2e
httpclient: update to 938f2107d6e2 of httpplus
Augie Fackler <augie@google.com>
parents:
25660
diff
changeset
|
332 any such header specified here if specified. Providing |
1ad9da968a2e
httpclient: update to 938f2107d6e2 of httpplus
Augie Fackler <augie@google.com>
parents:
25660
diff
changeset
|
333 this option and not proxy_hostport will raise an |
1ad9da968a2e
httpclient: update to 938f2107d6e2 of httpplus
Augie Fackler <augie@google.com>
parents:
25660
diff
changeset
|
334 ValueError. |
19807
c48df403caae
httpclient: import 4bb625347d4a to provide SSL wrapper injection
Augie Fackler <raf@durin42.com>
parents:
19620
diff
changeset
|
335 ssl_wrap_socket: Optional function to use for wrapping |
c48df403caae
httpclient: import 4bb625347d4a to provide SSL wrapper injection
Augie Fackler <raf@durin42.com>
parents:
19620
diff
changeset
|
336 sockets. If unspecified, the one from the ssl module will |
c48df403caae
httpclient: import 4bb625347d4a to provide SSL wrapper injection
Augie Fackler <raf@durin42.com>
parents:
19620
diff
changeset
|
337 be used if available, or something that's compatible with |
c48df403caae
httpclient: import 4bb625347d4a to provide SSL wrapper injection
Augie Fackler <raf@durin42.com>
parents:
19620
diff
changeset
|
338 it if on a Python older than 2.6. |
c48df403caae
httpclient: import 4bb625347d4a to provide SSL wrapper injection
Augie Fackler <raf@durin42.com>
parents:
19620
diff
changeset
|
339 |
c48df403caae
httpclient: import 4bb625347d4a to provide SSL wrapper injection
Augie Fackler <raf@durin42.com>
parents:
19620
diff
changeset
|
340 Any extra keyword arguments to this function will be provided |
c48df403caae
httpclient: import 4bb625347d4a to provide SSL wrapper injection
Augie Fackler <raf@durin42.com>
parents:
19620
diff
changeset
|
341 to the ssl_wrap_socket method. If no ssl |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
342 """ |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
343 if port is None and host.count(':') == 1 or ']:' in host: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
344 host, port = host.rsplit(':', 1) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
345 port = int(port) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
346 if '[' in host: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
347 host = host[1:-1] |
19807
c48df403caae
httpclient: import 4bb625347d4a to provide SSL wrapper injection
Augie Fackler <raf@durin42.com>
parents:
19620
diff
changeset
|
348 if ssl_wrap_socket is not None: |
c48df403caae
httpclient: import 4bb625347d4a to provide SSL wrapper injection
Augie Fackler <raf@durin42.com>
parents:
19620
diff
changeset
|
349 self._ssl_wrap_socket = ssl_wrap_socket |
c48df403caae
httpclient: import 4bb625347d4a to provide SSL wrapper injection
Augie Fackler <raf@durin42.com>
parents:
19620
diff
changeset
|
350 else: |
c48df403caae
httpclient: import 4bb625347d4a to provide SSL wrapper injection
Augie Fackler <raf@durin42.com>
parents:
19620
diff
changeset
|
351 self._ssl_wrap_socket = socketutil.wrap_socket |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
352 if use_ssl is None and port is None: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
353 use_ssl = False |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
354 port = 80 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
355 elif use_ssl is None: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
356 use_ssl = (port == 443) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
357 elif port is None: |
27601
1ad9da968a2e
httpclient: update to 938f2107d6e2 of httpplus
Augie Fackler <augie@google.com>
parents:
25660
diff
changeset
|
358 port = (use_ssl and 443 or 80) |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
359 self.port = port |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
360 if use_ssl and not socketutil.have_ssl: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
361 raise Exception('ssl requested but unavailable on this Python') |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
362 self.ssl = use_ssl |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
363 self.ssl_opts = ssl_opts |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
364 self._ssl_validator = ssl_validator |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
365 self.host = host |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
366 self.sock = None |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
367 self._current_response = None |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
368 self._current_response_taken = False |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
369 if proxy_hostport is None: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
370 self._proxy_host = self._proxy_port = None |
27601
1ad9da968a2e
httpclient: update to 938f2107d6e2 of httpplus
Augie Fackler <augie@google.com>
parents:
25660
diff
changeset
|
371 if proxy_headers: |
1ad9da968a2e
httpclient: update to 938f2107d6e2 of httpplus
Augie Fackler <augie@google.com>
parents:
25660
diff
changeset
|
372 raise ValueError( |
1ad9da968a2e
httpclient: update to 938f2107d6e2 of httpplus
Augie Fackler <augie@google.com>
parents:
25660
diff
changeset
|
373 'proxy_headers may not be specified unless ' |
1ad9da968a2e
httpclient: update to 938f2107d6e2 of httpplus
Augie Fackler <augie@google.com>
parents:
25660
diff
changeset
|
374 'proxy_hostport is also specified.') |
1ad9da968a2e
httpclient: update to 938f2107d6e2 of httpplus
Augie Fackler <augie@google.com>
parents:
25660
diff
changeset
|
375 else: |
1ad9da968a2e
httpclient: update to 938f2107d6e2 of httpplus
Augie Fackler <augie@google.com>
parents:
25660
diff
changeset
|
376 self._proxy_headers = {} |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
377 else: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
378 self._proxy_host, self._proxy_port = proxy_hostport |
27601
1ad9da968a2e
httpclient: update to 938f2107d6e2 of httpplus
Augie Fackler <augie@google.com>
parents:
25660
diff
changeset
|
379 self._proxy_headers = _foldheaders(proxy_headers or {}) |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
380 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
381 self.timeout = timeout |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
382 self.continue_timeout = continue_timeout |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
383 |
27601
1ad9da968a2e
httpclient: update to 938f2107d6e2 of httpplus
Augie Fackler <augie@google.com>
parents:
25660
diff
changeset
|
384 def _connect(self, proxy_headers): |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
385 """Connect to the host and port specified in __init__.""" |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
386 if self.sock: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
387 return |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
388 if self._proxy_host is not None: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
389 logger.info('Connecting to http proxy %s:%s', |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
390 self._proxy_host, self._proxy_port) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
391 sock = socketutil.create_connection((self._proxy_host, |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
392 self._proxy_port)) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
393 if self.ssl: |
19182
fae47ecaa952
httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents:
19038
diff
changeset
|
394 data = self._buildheaders('CONNECT', '%s:%d' % (self.host, |
fae47ecaa952
httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents:
19038
diff
changeset
|
395 self.port), |
27601
1ad9da968a2e
httpclient: update to 938f2107d6e2 of httpplus
Augie Fackler <augie@google.com>
parents:
25660
diff
changeset
|
396 proxy_headers, HTTP_VER_1_0) |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
397 sock.send(data) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
398 sock.setblocking(0) |
16643
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
399 r = self.response_class(sock, self.timeout, 'CONNECT') |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
400 timeout_exc = HTTPTimeoutException( |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
401 'Timed out waiting for CONNECT response from proxy') |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
402 while not r.complete(): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
403 try: |
19182
fae47ecaa952
httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents:
19038
diff
changeset
|
404 # We're a friend of the response class, so let |
fae47ecaa952
httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents:
19038
diff
changeset
|
405 # us use the private attribute. |
fae47ecaa952
httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents:
19038
diff
changeset
|
406 # pylint: disable=W0212 |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
407 if not r._select(): |
16643
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
408 if not r.complete(): |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
409 raise timeout_exc |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
410 except HTTPTimeoutException: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
411 # This raise/except pattern looks goofy, but |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
412 # _select can raise the timeout as well as the |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
413 # loop body. I wish it wasn't this convoluted, |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
414 # but I don't have a better solution |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
415 # immediately handy. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
416 raise timeout_exc |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
417 if r.status != 200: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
418 raise HTTPProxyConnectFailedException( |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
419 'Proxy connection failed: %d %s' % (r.status, |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
420 r.read())) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
421 logger.info('CONNECT (for SSL) to %s:%s via proxy succeeded.', |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
422 self.host, self.port) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
423 else: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
424 sock = socketutil.create_connection((self.host, self.port)) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
425 if self.ssl: |
16774
69af967b6d6f
httpclient: update to c5abd358e543 of httpplus
Augie Fackler <raf@durin42.com>
parents:
16643
diff
changeset
|
426 # This is the default, but in the case of proxied SSL |
69af967b6d6f
httpclient: update to c5abd358e543 of httpplus
Augie Fackler <raf@durin42.com>
parents:
16643
diff
changeset
|
427 # requests the proxy logic above will have cleared |
17424
e7cfe3587ea4
fix trivial spelling errors
Mads Kiilerich <mads@kiilerich.com>
parents:
16774
diff
changeset
|
428 # blocking mode, so re-enable it just to be safe. |
16774
69af967b6d6f
httpclient: update to c5abd358e543 of httpplus
Augie Fackler <raf@durin42.com>
parents:
16643
diff
changeset
|
429 sock.setblocking(1) |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
430 logger.debug('wrapping socket for ssl with options %r', |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
431 self.ssl_opts) |
19807
c48df403caae
httpclient: import 4bb625347d4a to provide SSL wrapper injection
Augie Fackler <raf@durin42.com>
parents:
19620
diff
changeset
|
432 sock = self._ssl_wrap_socket(sock, **self.ssl_opts) |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
433 if self._ssl_validator: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
434 self._ssl_validator(sock) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
435 sock.setblocking(0) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
436 self.sock = sock |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
437 |
19182
fae47ecaa952
httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents:
19038
diff
changeset
|
438 def _buildheaders(self, method, path, headers, http_ver): |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
439 if self.ssl and self.port == 443 or self.port == 80: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
440 # default port for protocol, so leave it out |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
441 hdrhost = self.host |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
442 else: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
443 # include nonstandard port in header |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
444 if ':' in self.host: # must be IPv6 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
445 hdrhost = '[%s]:%d' % (self.host, self.port) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
446 else: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
447 hdrhost = '%s:%d' % (self.host, self.port) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
448 if self._proxy_host and not self.ssl: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
449 # When talking to a regular http proxy we must send the |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
450 # full URI, but in all other cases we must not (although |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
451 # technically RFC 2616 says servers must accept our |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
452 # request if we screw up, experimentally few do that |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
453 # correctly.) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
454 assert path[0] == '/', 'path must start with a /' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
455 path = 'http://%s%s' % (hdrhost, path) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
456 outgoing = ['%s %s %s%s' % (method, path, http_ver, EOL)] |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
457 headers['host'] = ('Host', hdrhost) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
458 headers[HDR_ACCEPT_ENCODING] = (HDR_ACCEPT_ENCODING, 'identity') |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
459 for hdr, val in headers.itervalues(): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
460 outgoing.append('%s: %s%s' % (hdr, val, EOL)) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
461 outgoing.append(EOL) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
462 return ''.join(outgoing) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
463 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
464 def close(self): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
465 """Close the connection to the server. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
466 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
467 This is a no-op if the connection is already closed. The |
17536 | 468 connection may automatically close if requested by the server |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
469 or required by the nature of a response. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
470 """ |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
471 if self.sock is None: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
472 return |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
473 self.sock.close() |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
474 self.sock = None |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
475 logger.info('closed connection to %s on %s', self.host, self.port) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
476 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
477 def busy(self): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
478 """Returns True if this connection object is currently in use. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
479 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
480 If a response is still pending, this will return True, even if |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
481 the request has finished sending. In the future, |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
482 HTTPConnection may transparently juggle multiple connections |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
483 to the server, in which case this will be useful to detect if |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
484 any of those connections is ready for use. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
485 """ |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
486 cr = self._current_response |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
487 if cr is not None: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
488 if self._current_response_taken: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
489 if cr.will_close: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
490 self.sock = None |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
491 self._current_response = None |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
492 return False |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
493 elif cr.complete(): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
494 self._current_response = None |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
495 return False |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
496 return True |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
497 return False |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
498 |
27601
1ad9da968a2e
httpclient: update to 938f2107d6e2 of httpplus
Augie Fackler <augie@google.com>
parents:
25660
diff
changeset
|
499 def _reconnect(self, where, pheaders): |
19182
fae47ecaa952
httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents:
19038
diff
changeset
|
500 logger.info('reconnecting during %s', where) |
fae47ecaa952
httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents:
19038
diff
changeset
|
501 self.close() |
27601
1ad9da968a2e
httpclient: update to 938f2107d6e2 of httpplus
Augie Fackler <augie@google.com>
parents:
25660
diff
changeset
|
502 self._connect(pheaders) |
19182
fae47ecaa952
httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents:
19038
diff
changeset
|
503 |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
504 def request(self, method, path, body=None, headers={}, |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
505 expect_continue=False): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
506 """Send a request to the server. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
507 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
508 For increased flexibility, this does not return the response |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
509 object. Future versions of HTTPConnection that juggle multiple |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
510 sockets will be able to send (for example) 5 requests all at |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
511 once, and then let the requests arrive as data is |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
512 available. Use the `getresponse()` method to retrieve the |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
513 response. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
514 """ |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
515 if self.busy(): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
516 raise httplib.CannotSendRequest( |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
517 'Can not send another request before ' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
518 'current response is read!') |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
519 self._current_response_taken = False |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
520 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
521 logger.info('sending %s request for %s to %s on port %s', |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
522 method, path, self.host, self.port) |
27601
1ad9da968a2e
httpclient: update to 938f2107d6e2 of httpplus
Augie Fackler <augie@google.com>
parents:
25660
diff
changeset
|
523 hdrs = _foldheaders(headers) |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
524 if hdrs.get('expect', ('', ''))[1].lower() == '100-continue': |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
525 expect_continue = True |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
526 elif expect_continue: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
527 hdrs['expect'] = ('Expect', '100-Continue') |
27601
1ad9da968a2e
httpclient: update to 938f2107d6e2 of httpplus
Augie Fackler <augie@google.com>
parents:
25660
diff
changeset
|
528 # httplib compatibility: if the user specified a |
1ad9da968a2e
httpclient: update to 938f2107d6e2 of httpplus
Augie Fackler <augie@google.com>
parents:
25660
diff
changeset
|
529 # proxy-authorization header, that's actually intended for a |
1ad9da968a2e
httpclient: update to 938f2107d6e2 of httpplus
Augie Fackler <augie@google.com>
parents:
25660
diff
changeset
|
530 # proxy CONNECT action, not the real request, but only if |
1ad9da968a2e
httpclient: update to 938f2107d6e2 of httpplus
Augie Fackler <augie@google.com>
parents:
25660
diff
changeset
|
531 # we're going to use a proxy. |
1ad9da968a2e
httpclient: update to 938f2107d6e2 of httpplus
Augie Fackler <augie@google.com>
parents:
25660
diff
changeset
|
532 pheaders = dict(self._proxy_headers) |
1ad9da968a2e
httpclient: update to 938f2107d6e2 of httpplus
Augie Fackler <augie@google.com>
parents:
25660
diff
changeset
|
533 if self._proxy_host and self.ssl: |
1ad9da968a2e
httpclient: update to 938f2107d6e2 of httpplus
Augie Fackler <augie@google.com>
parents:
25660
diff
changeset
|
534 pa = hdrs.pop('proxy-authorization', None) |
1ad9da968a2e
httpclient: update to 938f2107d6e2 of httpplus
Augie Fackler <augie@google.com>
parents:
25660
diff
changeset
|
535 if pa is not None: |
1ad9da968a2e
httpclient: update to 938f2107d6e2 of httpplus
Augie Fackler <augie@google.com>
parents:
25660
diff
changeset
|
536 pheaders['proxy-authorization'] = pa |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
537 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
538 chunked = False |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
539 if body and HDR_CONTENT_LENGTH not in hdrs: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
540 if getattr(body, '__len__', False): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
541 hdrs[HDR_CONTENT_LENGTH] = (HDR_CONTENT_LENGTH, len(body)) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
542 elif getattr(body, 'read', False): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
543 hdrs[HDR_XFER_ENCODING] = (HDR_XFER_ENCODING, |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
544 XFER_ENCODING_CHUNKED) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
545 chunked = True |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
546 else: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
547 raise BadRequestData('body has no __len__() nor read()') |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
548 |
19620
d4a0055af149
httpclient: import 0d1b0a8bc549 to fix bug involving late-arriving RST after a response
Augie Fackler <raf@durin42.com>
parents:
19489
diff
changeset
|
549 # If we're reusing the underlying socket, there are some |
d4a0055af149
httpclient: import 0d1b0a8bc549 to fix bug involving late-arriving RST after a response
Augie Fackler <raf@durin42.com>
parents:
19489
diff
changeset
|
550 # conditions where we'll want to retry, so make a note of the |
d4a0055af149
httpclient: import 0d1b0a8bc549 to fix bug involving late-arriving RST after a response
Augie Fackler <raf@durin42.com>
parents:
19489
diff
changeset
|
551 # state of self.sock |
d4a0055af149
httpclient: import 0d1b0a8bc549 to fix bug involving late-arriving RST after a response
Augie Fackler <raf@durin42.com>
parents:
19489
diff
changeset
|
552 fresh_socket = self.sock is None |
27601
1ad9da968a2e
httpclient: update to 938f2107d6e2 of httpplus
Augie Fackler <augie@google.com>
parents:
25660
diff
changeset
|
553 self._connect(pheaders) |
19182
fae47ecaa952
httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents:
19038
diff
changeset
|
554 outgoing_headers = self._buildheaders( |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
555 method, path, hdrs, self.http_version) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
556 response = None |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
557 first = True |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
558 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
559 while ((outgoing_headers or body) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
560 and not (response and response.complete())): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
561 select_timeout = self.timeout |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
562 out = outgoing_headers or body |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
563 blocking_on_continue = False |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
564 if expect_continue and not outgoing_headers and not ( |
16643
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
565 response and (response.headers or response.continued)): |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
566 logger.info( |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
567 'waiting up to %s seconds for' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
568 ' continue response from server', |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
569 self.continue_timeout) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
570 select_timeout = self.continue_timeout |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
571 blocking_on_continue = True |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
572 out = False |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
573 if out: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
574 w = [self.sock] |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
575 else: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
576 w = [] |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
577 r, w, x = select.select([self.sock], w, [], select_timeout) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
578 # if we were expecting a 100 continue and it's been long |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
579 # enough, just go ahead and assume it's ok. This is the |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
580 # recommended behavior from the RFC. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
581 if r == w == x == []: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
582 if blocking_on_continue: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
583 expect_continue = False |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
584 logger.info('no response to continue expectation from ' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
585 'server, optimistically sending request body') |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
586 else: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
587 raise HTTPTimeoutException('timeout sending data') |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
588 was_first = first |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
589 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
590 # incoming data |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
591 if r: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
592 try: |
14341
5c3de67e7402
httpclient: import revision b8c3511a8cae from py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14293
diff
changeset
|
593 try: |
5c3de67e7402
httpclient: import revision b8c3511a8cae from py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14293
diff
changeset
|
594 data = r[0].recv(INCOMING_BUFFER_SIZE) |
25660
328739ea70c3
global: mass rewrite to use modern exception syntax
Gregory Szorc <gregory.szorc@gmail.com>
parents:
24306
diff
changeset
|
595 except socket.sslerror as e: |
14341
5c3de67e7402
httpclient: import revision b8c3511a8cae from py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14293
diff
changeset
|
596 if e.args[0] != socket.SSL_ERROR_WANT_READ: |
5c3de67e7402
httpclient: import revision b8c3511a8cae from py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14293
diff
changeset
|
597 raise |
19182
fae47ecaa952
httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents:
19038
diff
changeset
|
598 logger.debug('SSL_ERROR_WANT_READ while sending ' |
fae47ecaa952
httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents:
19038
diff
changeset
|
599 'data, retrying...') |
14341
5c3de67e7402
httpclient: import revision b8c3511a8cae from py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14293
diff
changeset
|
600 continue |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
601 if not data: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
602 logger.info('socket appears closed in read') |
14376
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
603 self.sock = None |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
604 self._current_response = None |
16643
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
605 if response is not None: |
19182
fae47ecaa952
httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents:
19038
diff
changeset
|
606 # We're a friend of the response class, so let |
fae47ecaa952
httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents:
19038
diff
changeset
|
607 # us use the private attribute. |
fae47ecaa952
httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents:
19038
diff
changeset
|
608 # pylint: disable=W0212 |
16643
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
609 response._close() |
14376
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
610 # This if/elif ladder is a bit subtle, |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
611 # comments in each branch should help. |
16643
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
612 if response is not None and response.complete(): |
14376
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
613 # Server responded completely and then |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
614 # closed the socket. We should just shut |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
615 # things down and let the caller get their |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
616 # response. |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
617 logger.info('Got an early response, ' |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
618 'aborting remaining request.') |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
619 break |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
620 elif was_first and response is None: |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
621 # Most likely a keepalive that got killed |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
622 # on the server's end. Commonly happens |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
623 # after getting a really large response |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
624 # from the server. |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
625 logger.info( |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
626 'Connection appeared closed in read on first' |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
627 ' request loop iteration, will retry.') |
27601
1ad9da968a2e
httpclient: update to 938f2107d6e2 of httpplus
Augie Fackler <augie@google.com>
parents:
25660
diff
changeset
|
628 self._reconnect('read', pheaders) |
14376
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
629 continue |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
630 else: |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
631 # We didn't just send the first data hunk, |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
632 # and either have a partial response or no |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
633 # response at all. There's really nothing |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
634 # meaningful we can do here. |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
635 raise HTTPStateError( |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
636 'Connection appears closed after ' |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
637 'some request data was written, but the ' |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
638 'response was missing or incomplete!') |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
639 logger.debug('read %d bytes in request()', len(data)) |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
640 if response is None: |
19182
fae47ecaa952
httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents:
19038
diff
changeset
|
641 response = self.response_class( |
fae47ecaa952
httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents:
19038
diff
changeset
|
642 r[0], self.timeout, method) |
fae47ecaa952
httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents:
19038
diff
changeset
|
643 # We're a friend of the response class, so let us |
fae47ecaa952
httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents:
19038
diff
changeset
|
644 # use the private attribute. |
fae47ecaa952
httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents:
19038
diff
changeset
|
645 # pylint: disable=W0212 |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
646 response._load_response(data) |
14376
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
647 # Jump to the next select() call so we load more |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
648 # data if the server is still sending us content. |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
649 continue |
25660
328739ea70c3
global: mass rewrite to use modern exception syntax
Gregory Szorc <gregory.szorc@gmail.com>
parents:
24306
diff
changeset
|
650 except socket.error as e: |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
651 if e[0] != errno.EPIPE and not was_first: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
652 raise |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
653 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
654 # outgoing data |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
655 if w and out: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
656 try: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
657 if getattr(out, 'read', False): |
19182
fae47ecaa952
httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents:
19038
diff
changeset
|
658 # pylint guesses the type of out incorrectly here |
fae47ecaa952
httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents:
19038
diff
changeset
|
659 # pylint: disable=E1103 |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
660 data = out.read(OUTGOING_BUFFER_SIZE) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
661 if not data: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
662 continue |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
663 if len(data) < OUTGOING_BUFFER_SIZE: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
664 if chunked: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
665 body = '0' + EOL + EOL |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
666 else: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
667 body = None |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
668 if chunked: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
669 out = hex(len(data))[2:] + EOL + data + EOL |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
670 else: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
671 out = data |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
672 amt = w[0].send(out) |
25660
328739ea70c3
global: mass rewrite to use modern exception syntax
Gregory Szorc <gregory.szorc@gmail.com>
parents:
24306
diff
changeset
|
673 except socket.error as e: |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
674 if e[0] == socket.SSL_ERROR_WANT_WRITE and self.ssl: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
675 # This means that SSL hasn't flushed its buffer into |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
676 # the socket yet. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
677 # TODO: find a way to block on ssl flushing its buffer |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
678 # similar to selecting on a raw socket. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
679 continue |
19489
42fcb2f7787d
httpclient: update to revision 9517a2b56fe9 of httpplus (issue3905)
Augie Fackler <raf@durin42.com>
parents:
19182
diff
changeset
|
680 if e[0] == errno.EWOULDBLOCK or e[0] == errno.EAGAIN: |
42fcb2f7787d
httpclient: update to revision 9517a2b56fe9 of httpplus (issue3905)
Augie Fackler <raf@durin42.com>
parents:
19182
diff
changeset
|
681 continue |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
682 elif (e[0] not in (errno.ECONNRESET, errno.EPIPE) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
683 and not first): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
684 raise |
27601
1ad9da968a2e
httpclient: update to 938f2107d6e2 of httpplus
Augie Fackler <augie@google.com>
parents:
25660
diff
changeset
|
685 self._reconnect('write', pheaders) |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
686 amt = self.sock.send(out) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
687 logger.debug('sent %d', amt) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
688 first = False |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
689 if out is body: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
690 body = out[amt:] |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
691 else: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
692 outgoing_headers = out[amt:] |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
693 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
694 # close if the server response said to or responded before eating |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
695 # the whole request |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
696 if response is None: |
16643
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
697 response = self.response_class(self.sock, self.timeout, method) |
19620
d4a0055af149
httpclient: import 0d1b0a8bc549 to fix bug involving late-arriving RST after a response
Augie Fackler <raf@durin42.com>
parents:
19489
diff
changeset
|
698 if not fresh_socket: |
d4a0055af149
httpclient: import 0d1b0a8bc549 to fix bug involving late-arriving RST after a response
Augie Fackler <raf@durin42.com>
parents:
19489
diff
changeset
|
699 if not response._select(): |
d4a0055af149
httpclient: import 0d1b0a8bc549 to fix bug involving late-arriving RST after a response
Augie Fackler <raf@durin42.com>
parents:
19489
diff
changeset
|
700 # This means the response failed to get any response |
d4a0055af149
httpclient: import 0d1b0a8bc549 to fix bug involving late-arriving RST after a response
Augie Fackler <raf@durin42.com>
parents:
19489
diff
changeset
|
701 # data at all, and in all probability the socket was |
d4a0055af149
httpclient: import 0d1b0a8bc549 to fix bug involving late-arriving RST after a response
Augie Fackler <raf@durin42.com>
parents:
19489
diff
changeset
|
702 # closed before the server even saw our request. Try |
d4a0055af149
httpclient: import 0d1b0a8bc549 to fix bug involving late-arriving RST after a response
Augie Fackler <raf@durin42.com>
parents:
19489
diff
changeset
|
703 # the request again on a fresh socket. |
27601
1ad9da968a2e
httpclient: update to 938f2107d6e2 of httpplus
Augie Fackler <augie@google.com>
parents:
25660
diff
changeset
|
704 logger.debug('response._select() failed during request().' |
1ad9da968a2e
httpclient: update to 938f2107d6e2 of httpplus
Augie Fackler <augie@google.com>
parents:
25660
diff
changeset
|
705 ' Assuming request needs to be retried.') |
19620
d4a0055af149
httpclient: import 0d1b0a8bc549 to fix bug involving late-arriving RST after a response
Augie Fackler <raf@durin42.com>
parents:
19489
diff
changeset
|
706 self.sock = None |
d4a0055af149
httpclient: import 0d1b0a8bc549 to fix bug involving late-arriving RST after a response
Augie Fackler <raf@durin42.com>
parents:
19489
diff
changeset
|
707 # Call this method explicitly to re-try the |
d4a0055af149
httpclient: import 0d1b0a8bc549 to fix bug involving late-arriving RST after a response
Augie Fackler <raf@durin42.com>
parents:
19489
diff
changeset
|
708 # request. We don't use self.request() because |
d4a0055af149
httpclient: import 0d1b0a8bc549 to fix bug involving late-arriving RST after a response
Augie Fackler <raf@durin42.com>
parents:
19489
diff
changeset
|
709 # some tools (notably Mercurial) expect to be able |
d4a0055af149
httpclient: import 0d1b0a8bc549 to fix bug involving late-arriving RST after a response
Augie Fackler <raf@durin42.com>
parents:
19489
diff
changeset
|
710 # to subclass and redefine request(), and they |
d4a0055af149
httpclient: import 0d1b0a8bc549 to fix bug involving late-arriving RST after a response
Augie Fackler <raf@durin42.com>
parents:
19489
diff
changeset
|
711 # don't have the same argspec as we do. |
d4a0055af149
httpclient: import 0d1b0a8bc549 to fix bug involving late-arriving RST after a response
Augie Fackler <raf@durin42.com>
parents:
19489
diff
changeset
|
712 # |
d4a0055af149
httpclient: import 0d1b0a8bc549 to fix bug involving late-arriving RST after a response
Augie Fackler <raf@durin42.com>
parents:
19489
diff
changeset
|
713 # TODO restructure sending of requests to avoid |
d4a0055af149
httpclient: import 0d1b0a8bc549 to fix bug involving late-arriving RST after a response
Augie Fackler <raf@durin42.com>
parents:
19489
diff
changeset
|
714 # this recursion |
d4a0055af149
httpclient: import 0d1b0a8bc549 to fix bug involving late-arriving RST after a response
Augie Fackler <raf@durin42.com>
parents:
19489
diff
changeset
|
715 return HTTPConnection.request( |
d4a0055af149
httpclient: import 0d1b0a8bc549 to fix bug involving late-arriving RST after a response
Augie Fackler <raf@durin42.com>
parents:
19489
diff
changeset
|
716 self, method, path, body=body, headers=headers, |
d4a0055af149
httpclient: import 0d1b0a8bc549 to fix bug involving late-arriving RST after a response
Augie Fackler <raf@durin42.com>
parents:
19489
diff
changeset
|
717 expect_continue=expect_continue) |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
718 data_left = bool(outgoing_headers or body) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
719 if data_left: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
720 logger.info('stopped sending request early, ' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
721 'will close the socket to be safe.') |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
722 response.will_close = True |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
723 if response.will_close: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
724 # The socket will be closed by the response, so we disown |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
725 # the socket |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
726 self.sock = None |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
727 self._current_response = response |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
728 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
729 def getresponse(self): |
19182
fae47ecaa952
httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents:
19038
diff
changeset
|
730 """Returns the response to the most recent request.""" |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
731 if self._current_response is None: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
732 raise httplib.ResponseNotReady() |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
733 r = self._current_response |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
734 while r.headers is None: |
19182
fae47ecaa952
httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents:
19038
diff
changeset
|
735 # We're a friend of the response class, so let us use the |
fae47ecaa952
httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents:
19038
diff
changeset
|
736 # private attribute. |
fae47ecaa952
httpclient: upgrade to fe8c09e4db64 of httpplus
Augie Fackler <raf@durin42.com>
parents:
19038
diff
changeset
|
737 # pylint: disable=W0212 |
16643
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
738 if not r._select() and not r.complete(): |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
739 raise _readers.HTTPRemoteClosedError() |
14293
9adbb5ef0964
httpclient: import f4c380237fd5 to fix keepalive not working
Augie Fackler <durin42@gmail.com>
parents:
14243
diff
changeset
|
740 if r.will_close: |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
741 self.sock = None |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
742 self._current_response = None |
14293
9adbb5ef0964
httpclient: import f4c380237fd5 to fix keepalive not working
Augie Fackler <durin42@gmail.com>
parents:
14243
diff
changeset
|
743 elif r.complete(): |
9adbb5ef0964
httpclient: import f4c380237fd5 to fix keepalive not working
Augie Fackler <durin42@gmail.com>
parents:
14243
diff
changeset
|
744 self._current_response = None |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
745 else: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
746 self._current_response_taken = True |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
747 return r |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
748 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
749 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
750 class HTTPTimeoutException(httplib.HTTPException): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
751 """A timeout occurred while waiting on the server.""" |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
752 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
753 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
754 class BadRequestData(httplib.HTTPException): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
755 """Request body object has neither __len__ nor read.""" |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
756 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
757 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
758 class HTTPProxyConnectFailedException(httplib.HTTPException): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
759 """Connecting to the HTTP proxy failed.""" |
14376
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
760 |
15218
c81dce8a7bb6
httpclient: update to 07d8c356f4d1 of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14376
diff
changeset
|
761 |
14376
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
762 class HTTPStateError(httplib.HTTPException): |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
763 """Invalid internal state encountered.""" |
15218
c81dce8a7bb6
httpclient: update to 07d8c356f4d1 of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14376
diff
changeset
|
764 |
16643
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
765 # Forward this exception type from _readers since it needs to be part |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
766 # of the public API. |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
767 HTTPRemoteClosedError = _readers.HTTPRemoteClosedError |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
768 # no-check-code |