Mercurial > hg
annotate mercurial/httpclient/__init__.py @ 14341:5c3de67e7402
httpclient: import revision b8c3511a8cae from py-nonblocking-http
Fixes issues with SSL_ERROR_WANT_READ incorrectly breaking the
response read.
author | Augie Fackler <durin42@gmail.com> |
---|---|
date | Thu, 12 May 2011 10:48:31 -0500 |
parents | 9adbb5ef0964 |
children | a75e0f4ba0ab |
rev | line source |
---|---|
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
1 # Copyright 2010, Google Inc. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
2 # All rights reserved. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
3 # |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
4 # Redistribution and use in source and binary forms, with or without |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
5 # modification, are permitted provided that the following conditions are |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
6 # met: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
7 # |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
8 # * Redistributions of source code must retain the above copyright |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
9 # notice, this list of conditions and the following disclaimer. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
10 # * Redistributions in binary form must reproduce the above |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
11 # copyright notice, this list of conditions and the following disclaimer |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
12 # in the documentation and/or other materials provided with the |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
13 # distribution. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
14 # * Neither the name of Google Inc. nor the names of its |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
15 # contributors may be used to endorse or promote products derived from |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
16 # this software without specific prior written permission. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
17 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
18 # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
19 # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
20 # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
21 # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
22 # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
23 # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
24 # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
25 # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
26 # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
27 # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
28 # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
29 """Improved HTTP/1.1 client library |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
30 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
31 This library contains an HTTPConnection which is similar to the one in |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
32 httplib, but has several additional features: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
33 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
34 * supports keepalives natively |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
35 * uses select() to block for incoming data |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
36 * notices when the server responds early to a request |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
37 * implements ssl inline instead of in a different class |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
38 """ |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
39 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
40 import cStringIO |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
41 import errno |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
42 import httplib |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
43 import logging |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
44 import rfc822 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
45 import select |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
46 import socket |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
47 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
48 import socketutil |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
49 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
50 logger = logging.getLogger(__name__) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
51 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
52 __all__ = ['HTTPConnection', 'HTTPResponse'] |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
53 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
54 HTTP_VER_1_0 = 'HTTP/1.0' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
55 HTTP_VER_1_1 = 'HTTP/1.1' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
56 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
57 _LEN_CLOSE_IS_END = -1 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
58 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
59 OUTGOING_BUFFER_SIZE = 1 << 15 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
60 INCOMING_BUFFER_SIZE = 1 << 20 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
61 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
62 HDR_ACCEPT_ENCODING = 'accept-encoding' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
63 HDR_CONNECTION_CTRL = 'connection' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
64 HDR_CONTENT_LENGTH = 'content-length' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
65 HDR_XFER_ENCODING = 'transfer-encoding' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
66 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
67 XFER_ENCODING_CHUNKED = 'chunked' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
68 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
69 CONNECTION_CLOSE = 'close' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
70 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
71 EOL = '\r\n' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
72 _END_HEADERS = EOL * 2 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
73 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
74 # Based on some searching around, 1 second seems like a reasonable |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
75 # default here. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
76 TIMEOUT_ASSUME_CONTINUE = 1 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
77 TIMEOUT_DEFAULT = None |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
78 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
79 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
80 class HTTPResponse(object): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
81 """Response from an HTTP server. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
82 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
83 The response will continue to load as available. If you need the |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
84 complete response before continuing, check the .complete() method. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
85 """ |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
86 def __init__(self, sock, timeout): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
87 self.sock = sock |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
88 self.raw_response = '' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
89 self._body = None |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
90 self._headers_len = 0 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
91 self._content_len = 0 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
92 self.headers = None |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
93 self.will_close = False |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
94 self.status_line = '' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
95 self.status = None |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
96 self.http_version = None |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
97 self.reason = None |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
98 self._chunked = False |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
99 self._chunked_done = False |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
100 self._chunked_until_next = 0 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
101 self._chunked_skip_bytes = 0 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
102 self._chunked_preloaded_block = None |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
103 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
104 self._read_location = 0 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
105 self._eol = EOL |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
106 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
107 self._timeout = timeout |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
108 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
109 @property |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
110 def _end_headers(self): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
111 return self._eol * 2 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
112 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
113 def complete(self): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
114 """Returns true if this response is completely loaded. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
115 """ |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
116 if self._chunked: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
117 return self._chunked_done |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
118 if self._content_len == _LEN_CLOSE_IS_END: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
119 return False |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
120 return self._body is not None and len(self._body) >= self._content_len |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
121 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
122 def readline(self): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
123 """Read a single line from the response body. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
124 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
125 This may block until either a line ending is found or the |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
126 response is complete. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
127 """ |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
128 eol = self._body.find('\n', self._read_location) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
129 while eol == -1 and not self.complete(): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
130 self._select() |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
131 eol = self._body.find('\n', self._read_location) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
132 if eol != -1: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
133 eol += 1 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
134 else: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
135 eol = len(self._body) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
136 data = self._body[self._read_location:eol] |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
137 self._read_location = eol |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
138 return data |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
139 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
140 def read(self, length=None): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
141 # if length is None, unbounded read |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
142 while (not self.complete() # never select on a finished read |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
143 and (not length # unbounded, so we wait for complete() |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
144 or (self._read_location + length) > len(self._body))): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
145 self._select() |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
146 if not length: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
147 length = len(self._body) - self._read_location |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
148 elif len(self._body) < (self._read_location + length): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
149 length = len(self._body) - self._read_location |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
150 r = self._body[self._read_location:self._read_location + length] |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
151 self._read_location += len(r) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
152 if self.complete() and self.will_close: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
153 self.sock.close() |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
154 return r |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
155 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
156 def _select(self): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
157 r, _, _ = select.select([self.sock], [], [], self._timeout) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
158 if not r: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
159 # socket was not readable. If the response is not complete |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
160 # and we're not a _LEN_CLOSE_IS_END response, raise a timeout. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
161 # If we are a _LEN_CLOSE_IS_END response and we have no data, |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
162 # raise a timeout. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
163 if not (self.complete() or |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
164 (self._content_len == _LEN_CLOSE_IS_END and self._body)): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
165 logger.info('timed out with timeout of %s', self._timeout) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
166 raise HTTPTimeoutException('timeout reading data') |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
167 logger.info('cl: %r body: %r', self._content_len, self._body) |
14341
5c3de67e7402
httpclient: import revision b8c3511a8cae from py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14293
diff
changeset
|
168 try: |
5c3de67e7402
httpclient: import revision b8c3511a8cae from py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14293
diff
changeset
|
169 data = self.sock.recv(INCOMING_BUFFER_SIZE) |
5c3de67e7402
httpclient: import revision b8c3511a8cae from py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14293
diff
changeset
|
170 except socket.sslerror, e: |
5c3de67e7402
httpclient: import revision b8c3511a8cae from py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14293
diff
changeset
|
171 if e.args[0] != socket.SSL_ERROR_WANT_READ: |
5c3de67e7402
httpclient: import revision b8c3511a8cae from py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14293
diff
changeset
|
172 raise |
5c3de67e7402
httpclient: import revision b8c3511a8cae from py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14293
diff
changeset
|
173 logger.debug('SSL_WANT_READ in _select, should retry later') |
5c3de67e7402
httpclient: import revision b8c3511a8cae from py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14293
diff
changeset
|
174 return True |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
175 logger.debug('response read %d data during _select', len(data)) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
176 if not data: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
177 if not self.headers: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
178 self._load_response(self._end_headers) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
179 self._content_len = 0 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
180 elif self._content_len == _LEN_CLOSE_IS_END: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
181 self._content_len = len(self._body) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
182 return False |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
183 else: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
184 self._load_response(data) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
185 return True |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
186 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
187 def _chunked_parsedata(self, data): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
188 if self._chunked_preloaded_block: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
189 data = self._chunked_preloaded_block + data |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
190 self._chunked_preloaded_block = None |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
191 while data: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
192 logger.debug('looping with %d data remaining', len(data)) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
193 # Slice out anything we should skip |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
194 if self._chunked_skip_bytes: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
195 if len(data) <= self._chunked_skip_bytes: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
196 self._chunked_skip_bytes -= len(data) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
197 data = '' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
198 break |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
199 else: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
200 data = data[self._chunked_skip_bytes:] |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
201 self._chunked_skip_bytes = 0 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
202 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
203 # determine how much is until the next chunk |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
204 if self._chunked_until_next: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
205 amt = self._chunked_until_next |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
206 logger.debug('reading remaining %d of existing chunk', amt) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
207 self._chunked_until_next = 0 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
208 body = data |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
209 else: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
210 try: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
211 amt, body = data.split(self._eol, 1) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
212 except ValueError: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
213 self._chunked_preloaded_block = data |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
214 logger.debug('saving %r as a preloaded block for chunked', |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
215 self._chunked_preloaded_block) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
216 return |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
217 amt = int(amt, base=16) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
218 logger.debug('reading chunk of length %d', amt) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
219 if amt == 0: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
220 self._chunked_done = True |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
221 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
222 # read through end of what we have or the chunk |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
223 self._body += body[:amt] |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
224 if len(body) >= amt: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
225 data = body[amt:] |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
226 self._chunked_skip_bytes = len(self._eol) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
227 else: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
228 self._chunked_until_next = amt - len(body) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
229 self._chunked_skip_bytes = 0 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
230 data = '' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
231 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
232 def _load_response(self, data): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
233 if self._chunked: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
234 self._chunked_parsedata(data) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
235 return |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
236 elif self._body is not None: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
237 self._body += data |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
238 return |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
239 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
240 # We haven't seen end of headers yet |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
241 self.raw_response += data |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
242 # This is a bogus server with bad line endings |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
243 if self._eol not in self.raw_response: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
244 for bad_eol in ('\n', '\r'): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
245 if (bad_eol in self.raw_response |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
246 # verify that bad_eol is not the end of the incoming data |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
247 # as this could be a response line that just got |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
248 # split between \r and \n. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
249 and (self.raw_response.index(bad_eol) < |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
250 (len(self.raw_response) - 1))): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
251 logger.info('bogus line endings detected, ' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
252 'using %r for EOL', bad_eol) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
253 self._eol = bad_eol |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
254 break |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
255 # exit early if not at end of headers |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
256 if self._end_headers not in self.raw_response or self.headers: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
257 return |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
258 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
259 # handle 100-continue response |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
260 hdrs, body = self.raw_response.split(self._end_headers, 1) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
261 http_ver, status = hdrs.split(' ', 1) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
262 if status.startswith('100'): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
263 self.raw_response = body |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
264 logger.debug('continue seen, setting body to %r', body) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
265 return |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
266 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
267 # arriving here means we should parse response headers |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
268 # as all headers have arrived completely |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
269 hdrs, body = self.raw_response.split(self._end_headers, 1) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
270 del self.raw_response |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
271 if self._eol in hdrs: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
272 self.status_line, hdrs = hdrs.split(self._eol, 1) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
273 else: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
274 self.status_line = hdrs |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
275 hdrs = '' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
276 # TODO HTTP < 1.0 support |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
277 (self.http_version, self.status, |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
278 self.reason) = self.status_line.split(' ', 2) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
279 self.status = int(self.status) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
280 if self._eol != EOL: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
281 hdrs = hdrs.replace(self._eol, '\r\n') |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
282 headers = rfc822.Message(cStringIO.StringIO(hdrs)) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
283 if HDR_CONTENT_LENGTH in headers: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
284 self._content_len = int(headers[HDR_CONTENT_LENGTH]) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
285 if self.http_version == HTTP_VER_1_0: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
286 self.will_close = True |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
287 elif HDR_CONNECTION_CTRL in headers: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
288 self.will_close = ( |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
289 headers[HDR_CONNECTION_CTRL].lower() == CONNECTION_CLOSE) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
290 if self._content_len == 0: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
291 self._content_len = _LEN_CLOSE_IS_END |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
292 if (HDR_XFER_ENCODING in headers |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
293 and headers[HDR_XFER_ENCODING].lower() == XFER_ENCODING_CHUNKED): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
294 self._body = '' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
295 self._chunked_parsedata(body) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
296 self._chunked = True |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
297 if self._body is None: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
298 self._body = body |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
299 self.headers = headers |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
300 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
301 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
302 class HTTPConnection(object): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
303 """Connection to a single http server. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
304 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
305 Supports 100-continue and keepalives natively. Uses select() for |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
306 non-blocking socket operations. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
307 """ |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
308 http_version = HTTP_VER_1_1 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
309 response_class = HTTPResponse |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
310 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
311 def __init__(self, host, port=None, use_ssl=None, ssl_validator=None, |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
312 timeout=TIMEOUT_DEFAULT, |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
313 continue_timeout=TIMEOUT_ASSUME_CONTINUE, |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
314 proxy_hostport=None, **ssl_opts): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
315 """Create a new HTTPConnection. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
316 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
317 Args: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
318 host: The host to which we'll connect. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
319 port: Optional. The port over which we'll connect. Default 80 for |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
320 non-ssl, 443 for ssl. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
321 use_ssl: Optional. Wether to use ssl. Defaults to False if port is |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
322 not 443, true if port is 443. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
323 ssl_validator: a function(socket) to validate the ssl cert |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
324 timeout: Optional. Connection timeout, default is TIMEOUT_DEFAULT. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
325 continue_timeout: Optional. Timeout for waiting on an expected |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
326 "100 Continue" response. Default is TIMEOUT_ASSUME_CONTINUE. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
327 proxy_hostport: Optional. Tuple of (host, port) to use as an http |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
328 proxy for the connection. Default is to not use a proxy. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
329 """ |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
330 if port is None and host.count(':') == 1 or ']:' in host: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
331 host, port = host.rsplit(':', 1) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
332 port = int(port) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
333 if '[' in host: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
334 host = host[1:-1] |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
335 if use_ssl is None and port is None: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
336 use_ssl = False |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
337 port = 80 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
338 elif use_ssl is None: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
339 use_ssl = (port == 443) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
340 elif port is None: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
341 port = (use_ssl and 443 or 80) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
342 self.port = port |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
343 if use_ssl and not socketutil.have_ssl: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
344 raise Exception('ssl requested but unavailable on this Python') |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
345 self.ssl = use_ssl |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
346 self.ssl_opts = ssl_opts |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
347 self._ssl_validator = ssl_validator |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
348 self.host = host |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
349 self.sock = None |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
350 self._current_response = None |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
351 self._current_response_taken = False |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
352 if proxy_hostport is None: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
353 self._proxy_host = self._proxy_port = None |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
354 else: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
355 self._proxy_host, self._proxy_port = proxy_hostport |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
356 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
357 self.timeout = timeout |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
358 self.continue_timeout = continue_timeout |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
359 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
360 def _connect(self): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
361 """Connect to the host and port specified in __init__.""" |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
362 if self.sock: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
363 return |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
364 if self._proxy_host is not None: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
365 logger.info('Connecting to http proxy %s:%s', |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
366 self._proxy_host, self._proxy_port) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
367 sock = socketutil.create_connection((self._proxy_host, |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
368 self._proxy_port)) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
369 if self.ssl: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
370 # TODO proxy header support |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
371 data = self.buildheaders('CONNECT', '%s:%d' % (self.host, |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
372 self.port), |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
373 {}, HTTP_VER_1_0) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
374 sock.send(data) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
375 sock.setblocking(0) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
376 r = self.response_class(sock, self.timeout) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
377 timeout_exc = HTTPTimeoutException( |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
378 'Timed out waiting for CONNECT response from proxy') |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
379 while not r.complete(): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
380 try: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
381 if not r._select(): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
382 raise timeout_exc |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
383 except HTTPTimeoutException: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
384 # This raise/except pattern looks goofy, but |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
385 # _select can raise the timeout as well as the |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
386 # loop body. I wish it wasn't this convoluted, |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
387 # but I don't have a better solution |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
388 # immediately handy. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
389 raise timeout_exc |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
390 if r.status != 200: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
391 raise HTTPProxyConnectFailedException( |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
392 'Proxy connection failed: %d %s' % (r.status, |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
393 r.read())) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
394 logger.info('CONNECT (for SSL) to %s:%s via proxy succeeded.', |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
395 self.host, self.port) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
396 else: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
397 sock = socketutil.create_connection((self.host, self.port)) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
398 if self.ssl: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
399 logger.debug('wrapping socket for ssl with options %r', |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
400 self.ssl_opts) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
401 sock = socketutil.wrap_socket(sock, **self.ssl_opts) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
402 if self._ssl_validator: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
403 self._ssl_validator(sock) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
404 sock.setblocking(0) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
405 self.sock = sock |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
406 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
407 def buildheaders(self, method, path, headers, http_ver): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
408 if self.ssl and self.port == 443 or self.port == 80: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
409 # default port for protocol, so leave it out |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
410 hdrhost = self.host |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
411 else: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
412 # include nonstandard port in header |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
413 if ':' in self.host: # must be IPv6 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
414 hdrhost = '[%s]:%d' % (self.host, self.port) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
415 else: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
416 hdrhost = '%s:%d' % (self.host, self.port) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
417 if self._proxy_host and not self.ssl: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
418 # When talking to a regular http proxy we must send the |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
419 # full URI, but in all other cases we must not (although |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
420 # technically RFC 2616 says servers must accept our |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
421 # request if we screw up, experimentally few do that |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
422 # correctly.) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
423 assert path[0] == '/', 'path must start with a /' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
424 path = 'http://%s%s' % (hdrhost, path) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
425 outgoing = ['%s %s %s%s' % (method, path, http_ver, EOL)] |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
426 headers['host'] = ('Host', hdrhost) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
427 headers[HDR_ACCEPT_ENCODING] = (HDR_ACCEPT_ENCODING, 'identity') |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
428 for hdr, val in headers.itervalues(): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
429 outgoing.append('%s: %s%s' % (hdr, val, EOL)) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
430 outgoing.append(EOL) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
431 return ''.join(outgoing) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
432 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
433 def close(self): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
434 """Close the connection to the server. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
435 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
436 This is a no-op if the connection is already closed. The |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
437 connection may automatically close if requessted by the server |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
438 or required by the nature of a response. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
439 """ |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
440 if self.sock is None: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
441 return |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
442 self.sock.close() |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
443 self.sock = None |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
444 logger.info('closed connection to %s on %s', self.host, self.port) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
445 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
446 def busy(self): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
447 """Returns True if this connection object is currently in use. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
448 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
449 If a response is still pending, this will return True, even if |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
450 the request has finished sending. In the future, |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
451 HTTPConnection may transparently juggle multiple connections |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
452 to the server, in which case this will be useful to detect if |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
453 any of those connections is ready for use. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
454 """ |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
455 cr = self._current_response |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
456 if cr is not None: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
457 if self._current_response_taken: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
458 if cr.will_close: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
459 self.sock = None |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
460 self._current_response = None |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
461 return False |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
462 elif cr.complete(): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
463 self._current_response = None |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
464 return False |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
465 return True |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
466 return False |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
467 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
468 def request(self, method, path, body=None, headers={}, |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
469 expect_continue=False): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
470 """Send a request to the server. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
471 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
472 For increased flexibility, this does not return the response |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
473 object. Future versions of HTTPConnection that juggle multiple |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
474 sockets will be able to send (for example) 5 requests all at |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
475 once, and then let the requests arrive as data is |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
476 available. Use the `getresponse()` method to retrieve the |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
477 response. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
478 """ |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
479 if self.busy(): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
480 raise httplib.CannotSendRequest( |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
481 'Can not send another request before ' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
482 'current response is read!') |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
483 self._current_response_taken = False |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
484 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
485 logger.info('sending %s request for %s to %s on port %s', |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
486 method, path, self.host, self.port) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
487 hdrs = dict((k.lower(), (k, v)) for k, v in headers.iteritems()) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
488 if hdrs.get('expect', ('', ''))[1].lower() == '100-continue': |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
489 expect_continue = True |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
490 elif expect_continue: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
491 hdrs['expect'] = ('Expect', '100-Continue') |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
492 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
493 chunked = False |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
494 if body and HDR_CONTENT_LENGTH not in hdrs: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
495 if getattr(body, '__len__', False): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
496 hdrs[HDR_CONTENT_LENGTH] = (HDR_CONTENT_LENGTH, len(body)) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
497 elif getattr(body, 'read', False): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
498 hdrs[HDR_XFER_ENCODING] = (HDR_XFER_ENCODING, |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
499 XFER_ENCODING_CHUNKED) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
500 chunked = True |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
501 else: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
502 raise BadRequestData('body has no __len__() nor read()') |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
503 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
504 self._connect() |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
505 outgoing_headers = self.buildheaders( |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
506 method, path, hdrs, self.http_version) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
507 response = None |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
508 first = True |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
509 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
510 def reconnect(where): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
511 logger.info('reconnecting during %s', where) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
512 self.close() |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
513 self._connect() |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
514 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
515 while ((outgoing_headers or body) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
516 and not (response and response.complete())): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
517 select_timeout = self.timeout |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
518 out = outgoing_headers or body |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
519 blocking_on_continue = False |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
520 if expect_continue and not outgoing_headers and not ( |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
521 response and response.headers): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
522 logger.info( |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
523 'waiting up to %s seconds for' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
524 ' continue response from server', |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
525 self.continue_timeout) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
526 select_timeout = self.continue_timeout |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
527 blocking_on_continue = True |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
528 out = False |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
529 if out: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
530 w = [self.sock] |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
531 else: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
532 w = [] |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
533 r, w, x = select.select([self.sock], w, [], select_timeout) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
534 # if we were expecting a 100 continue and it's been long |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
535 # enough, just go ahead and assume it's ok. This is the |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
536 # recommended behavior from the RFC. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
537 if r == w == x == []: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
538 if blocking_on_continue: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
539 expect_continue = False |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
540 logger.info('no response to continue expectation from ' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
541 'server, optimistically sending request body') |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
542 else: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
543 raise HTTPTimeoutException('timeout sending data') |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
544 # TODO exceptional conditions with select? (what are those be?) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
545 # TODO if the response is loading, must we finish sending at all? |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
546 # |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
547 # Certainly not if it's going to close the connection and/or |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
548 # the response is already done...I think. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
549 was_first = first |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
550 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
551 # incoming data |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
552 if r: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
553 try: |
14341
5c3de67e7402
httpclient: import revision b8c3511a8cae from py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14293
diff
changeset
|
554 try: |
5c3de67e7402
httpclient: import revision b8c3511a8cae from py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14293
diff
changeset
|
555 data = r[0].recv(INCOMING_BUFFER_SIZE) |
5c3de67e7402
httpclient: import revision b8c3511a8cae from py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14293
diff
changeset
|
556 except socket.sslerror, e: |
5c3de67e7402
httpclient: import revision b8c3511a8cae from py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14293
diff
changeset
|
557 if e.args[0] != socket.SSL_ERROR_WANT_READ: |
5c3de67e7402
httpclient: import revision b8c3511a8cae from py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14293
diff
changeset
|
558 raise |
5c3de67e7402
httpclient: import revision b8c3511a8cae from py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14293
diff
changeset
|
559 logger.debug( |
5c3de67e7402
httpclient: import revision b8c3511a8cae from py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14293
diff
changeset
|
560 'SSL_WANT_READ while sending data, retrying...') |
5c3de67e7402
httpclient: import revision b8c3511a8cae from py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14293
diff
changeset
|
561 continue |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
562 if not data: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
563 logger.info('socket appears closed in read') |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
564 outgoing_headers = body = None |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
565 break |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
566 if response is None: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
567 response = self.response_class(r[0], self.timeout) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
568 response._load_response(data) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
569 if (response._content_len == _LEN_CLOSE_IS_END |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
570 and len(data) == 0): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
571 response._content_len = len(response._body) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
572 if response.complete(): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
573 w = [] |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
574 response.will_close = True |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
575 except socket.error, e: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
576 if e[0] != errno.EPIPE and not was_first: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
577 raise |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
578 if (response._content_len |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
579 and response._content_len != _LEN_CLOSE_IS_END): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
580 outgoing_headers = sent_data + outgoing_headers |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
581 reconnect('read') |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
582 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
583 # outgoing data |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
584 if w and out: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
585 try: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
586 if getattr(out, 'read', False): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
587 data = out.read(OUTGOING_BUFFER_SIZE) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
588 if not data: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
589 continue |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
590 if len(data) < OUTGOING_BUFFER_SIZE: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
591 if chunked: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
592 body = '0' + EOL + EOL |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
593 else: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
594 body = None |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
595 if chunked: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
596 out = hex(len(data))[2:] + EOL + data + EOL |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
597 else: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
598 out = data |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
599 amt = w[0].send(out) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
600 except socket.error, e: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
601 if e[0] == socket.SSL_ERROR_WANT_WRITE and self.ssl: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
602 # This means that SSL hasn't flushed its buffer into |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
603 # the socket yet. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
604 # TODO: find a way to block on ssl flushing its buffer |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
605 # similar to selecting on a raw socket. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
606 continue |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
607 elif (e[0] not in (errno.ECONNRESET, errno.EPIPE) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
608 and not first): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
609 raise |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
610 reconnect('write') |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
611 amt = self.sock.send(out) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
612 logger.debug('sent %d', amt) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
613 first = False |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
614 # stash data we think we sent in case the socket breaks |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
615 # when we read from it |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
616 if was_first: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
617 sent_data = out[:amt] |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
618 if out is body: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
619 body = out[amt:] |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
620 else: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
621 outgoing_headers = out[amt:] |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
622 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
623 # close if the server response said to or responded before eating |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
624 # the whole request |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
625 if response is None: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
626 response = self.response_class(self.sock, self.timeout) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
627 complete = response.complete() |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
628 data_left = bool(outgoing_headers or body) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
629 if data_left: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
630 logger.info('stopped sending request early, ' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
631 'will close the socket to be safe.') |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
632 response.will_close = True |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
633 if response.will_close: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
634 # The socket will be closed by the response, so we disown |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
635 # the socket |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
636 self.sock = None |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
637 self._current_response = response |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
638 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
639 def getresponse(self): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
640 if self._current_response is None: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
641 raise httplib.ResponseNotReady() |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
642 r = self._current_response |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
643 while r.headers is None: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
644 r._select() |
14293
9adbb5ef0964
httpclient: import f4c380237fd5 to fix keepalive not working
Augie Fackler <durin42@gmail.com>
parents:
14243
diff
changeset
|
645 if r.will_close: |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
646 self.sock = None |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
647 self._current_response = None |
14293
9adbb5ef0964
httpclient: import f4c380237fd5 to fix keepalive not working
Augie Fackler <durin42@gmail.com>
parents:
14243
diff
changeset
|
648 elif r.complete(): |
9adbb5ef0964
httpclient: import f4c380237fd5 to fix keepalive not working
Augie Fackler <durin42@gmail.com>
parents:
14243
diff
changeset
|
649 self._current_response = None |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
650 else: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
651 self._current_response_taken = True |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
652 return r |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
653 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
654 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
655 class HTTPTimeoutException(httplib.HTTPException): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
656 """A timeout occurred while waiting on the server.""" |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
657 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
658 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
659 class BadRequestData(httplib.HTTPException): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
660 """Request body object has neither __len__ nor read.""" |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
661 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
662 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
663 class HTTPProxyConnectFailedException(httplib.HTTPException): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
664 """Connecting to the HTTP proxy failed.""" |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
665 # no-check-code |