Mercurial > hg
annotate mercurial/httpclient/__init__.py @ 14376:a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
author | Augie Fackler <durin42@gmail.com> |
---|---|
date | Tue, 17 May 2011 10:28:03 -0500 |
parents | 5c3de67e7402 |
children | c81dce8a7bb6 |
rev | line source |
---|---|
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
1 # Copyright 2010, Google Inc. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
2 # All rights reserved. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
3 # |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
4 # Redistribution and use in source and binary forms, with or without |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
5 # modification, are permitted provided that the following conditions are |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
6 # met: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
7 # |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
8 # * Redistributions of source code must retain the above copyright |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
9 # notice, this list of conditions and the following disclaimer. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
10 # * Redistributions in binary form must reproduce the above |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
11 # copyright notice, this list of conditions and the following disclaimer |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
12 # in the documentation and/or other materials provided with the |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
13 # distribution. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
14 # * Neither the name of Google Inc. nor the names of its |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
15 # contributors may be used to endorse or promote products derived from |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
16 # this software without specific prior written permission. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
17 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
18 # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
19 # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
20 # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
21 # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
22 # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
23 # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
24 # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
25 # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
26 # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
27 # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
28 # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
29 """Improved HTTP/1.1 client library |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
30 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
31 This library contains an HTTPConnection which is similar to the one in |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
32 httplib, but has several additional features: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
33 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
34 * supports keepalives natively |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
35 * uses select() to block for incoming data |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
36 * notices when the server responds early to a request |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
37 * implements ssl inline instead of in a different class |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
38 """ |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
39 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
40 import cStringIO |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
41 import errno |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
42 import httplib |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
43 import logging |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
44 import rfc822 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
45 import select |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
46 import socket |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
47 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
48 import socketutil |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
49 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
50 logger = logging.getLogger(__name__) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
51 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
52 __all__ = ['HTTPConnection', 'HTTPResponse'] |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
53 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
54 HTTP_VER_1_0 = 'HTTP/1.0' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
55 HTTP_VER_1_1 = 'HTTP/1.1' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
56 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
57 _LEN_CLOSE_IS_END = -1 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
58 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
59 OUTGOING_BUFFER_SIZE = 1 << 15 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
60 INCOMING_BUFFER_SIZE = 1 << 20 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
61 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
62 HDR_ACCEPT_ENCODING = 'accept-encoding' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
63 HDR_CONNECTION_CTRL = 'connection' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
64 HDR_CONTENT_LENGTH = 'content-length' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
65 HDR_XFER_ENCODING = 'transfer-encoding' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
66 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
67 XFER_ENCODING_CHUNKED = 'chunked' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
68 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
69 CONNECTION_CLOSE = 'close' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
70 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
71 EOL = '\r\n' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
72 _END_HEADERS = EOL * 2 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
73 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
74 # Based on some searching around, 1 second seems like a reasonable |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
75 # default here. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
76 TIMEOUT_ASSUME_CONTINUE = 1 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
77 TIMEOUT_DEFAULT = None |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
78 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
79 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
80 class HTTPResponse(object): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
81 """Response from an HTTP server. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
82 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
83 The response will continue to load as available. If you need the |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
84 complete response before continuing, check the .complete() method. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
85 """ |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
86 def __init__(self, sock, timeout): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
87 self.sock = sock |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
88 self.raw_response = '' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
89 self._body = None |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
90 self._headers_len = 0 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
91 self._content_len = 0 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
92 self.headers = None |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
93 self.will_close = False |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
94 self.status_line = '' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
95 self.status = None |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
96 self.http_version = None |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
97 self.reason = None |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
98 self._chunked = False |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
99 self._chunked_done = False |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
100 self._chunked_until_next = 0 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
101 self._chunked_skip_bytes = 0 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
102 self._chunked_preloaded_block = None |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
103 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
104 self._read_location = 0 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
105 self._eol = EOL |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
106 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
107 self._timeout = timeout |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
108 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
109 @property |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
110 def _end_headers(self): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
111 return self._eol * 2 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
112 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
113 def complete(self): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
114 """Returns true if this response is completely loaded. |
14376
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
115 |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
116 Note that if this is a connection where complete means the |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
117 socket is closed, this will nearly always return False, even |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
118 in cases where all the data has actually been loaded. |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
119 """ |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
120 if self._chunked: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
121 return self._chunked_done |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
122 if self._content_len == _LEN_CLOSE_IS_END: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
123 return False |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
124 return self._body is not None and len(self._body) >= self._content_len |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
125 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
126 def readline(self): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
127 """Read a single line from the response body. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
128 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
129 This may block until either a line ending is found or the |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
130 response is complete. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
131 """ |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
132 eol = self._body.find('\n', self._read_location) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
133 while eol == -1 and not self.complete(): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
134 self._select() |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
135 eol = self._body.find('\n', self._read_location) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
136 if eol != -1: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
137 eol += 1 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
138 else: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
139 eol = len(self._body) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
140 data = self._body[self._read_location:eol] |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
141 self._read_location = eol |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
142 return data |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
143 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
144 def read(self, length=None): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
145 # if length is None, unbounded read |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
146 while (not self.complete() # never select on a finished read |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
147 and (not length # unbounded, so we wait for complete() |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
148 or (self._read_location + length) > len(self._body))): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
149 self._select() |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
150 if not length: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
151 length = len(self._body) - self._read_location |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
152 elif len(self._body) < (self._read_location + length): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
153 length = len(self._body) - self._read_location |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
154 r = self._body[self._read_location:self._read_location + length] |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
155 self._read_location += len(r) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
156 if self.complete() and self.will_close: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
157 self.sock.close() |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
158 return r |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
159 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
160 def _select(self): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
161 r, _, _ = select.select([self.sock], [], [], self._timeout) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
162 if not r: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
163 # socket was not readable. If the response is not complete |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
164 # and we're not a _LEN_CLOSE_IS_END response, raise a timeout. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
165 # If we are a _LEN_CLOSE_IS_END response and we have no data, |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
166 # raise a timeout. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
167 if not (self.complete() or |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
168 (self._content_len == _LEN_CLOSE_IS_END and self._body)): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
169 logger.info('timed out with timeout of %s', self._timeout) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
170 raise HTTPTimeoutException('timeout reading data') |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
171 logger.info('cl: %r body: %r', self._content_len, self._body) |
14341
5c3de67e7402
httpclient: import revision b8c3511a8cae from py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14293
diff
changeset
|
172 try: |
5c3de67e7402
httpclient: import revision b8c3511a8cae from py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14293
diff
changeset
|
173 data = self.sock.recv(INCOMING_BUFFER_SIZE) |
5c3de67e7402
httpclient: import revision b8c3511a8cae from py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14293
diff
changeset
|
174 except socket.sslerror, e: |
5c3de67e7402
httpclient: import revision b8c3511a8cae from py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14293
diff
changeset
|
175 if e.args[0] != socket.SSL_ERROR_WANT_READ: |
5c3de67e7402
httpclient: import revision b8c3511a8cae from py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14293
diff
changeset
|
176 raise |
5c3de67e7402
httpclient: import revision b8c3511a8cae from py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14293
diff
changeset
|
177 logger.debug('SSL_WANT_READ in _select, should retry later') |
5c3de67e7402
httpclient: import revision b8c3511a8cae from py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14293
diff
changeset
|
178 return True |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
179 logger.debug('response read %d data during _select', len(data)) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
180 if not data: |
14376
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
181 if self.headers and self._content_len == _LEN_CLOSE_IS_END: |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
182 self._content_len = len(self._body) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
183 return False |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
184 else: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
185 self._load_response(data) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
186 return True |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
187 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
188 def _chunked_parsedata(self, data): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
189 if self._chunked_preloaded_block: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
190 data = self._chunked_preloaded_block + data |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
191 self._chunked_preloaded_block = None |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
192 while data: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
193 logger.debug('looping with %d data remaining', len(data)) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
194 # Slice out anything we should skip |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
195 if self._chunked_skip_bytes: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
196 if len(data) <= self._chunked_skip_bytes: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
197 self._chunked_skip_bytes -= len(data) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
198 data = '' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
199 break |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
200 else: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
201 data = data[self._chunked_skip_bytes:] |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
202 self._chunked_skip_bytes = 0 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
203 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
204 # determine how much is until the next chunk |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
205 if self._chunked_until_next: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
206 amt = self._chunked_until_next |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
207 logger.debug('reading remaining %d of existing chunk', amt) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
208 self._chunked_until_next = 0 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
209 body = data |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
210 else: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
211 try: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
212 amt, body = data.split(self._eol, 1) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
213 except ValueError: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
214 self._chunked_preloaded_block = data |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
215 logger.debug('saving %r as a preloaded block for chunked', |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
216 self._chunked_preloaded_block) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
217 return |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
218 amt = int(amt, base=16) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
219 logger.debug('reading chunk of length %d', amt) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
220 if amt == 0: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
221 self._chunked_done = True |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
222 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
223 # read through end of what we have or the chunk |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
224 self._body += body[:amt] |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
225 if len(body) >= amt: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
226 data = body[amt:] |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
227 self._chunked_skip_bytes = len(self._eol) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
228 else: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
229 self._chunked_until_next = amt - len(body) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
230 self._chunked_skip_bytes = 0 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
231 data = '' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
232 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
233 def _load_response(self, data): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
234 if self._chunked: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
235 self._chunked_parsedata(data) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
236 return |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
237 elif self._body is not None: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
238 self._body += data |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
239 return |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
240 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
241 # We haven't seen end of headers yet |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
242 self.raw_response += data |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
243 # This is a bogus server with bad line endings |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
244 if self._eol not in self.raw_response: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
245 for bad_eol in ('\n', '\r'): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
246 if (bad_eol in self.raw_response |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
247 # verify that bad_eol is not the end of the incoming data |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
248 # as this could be a response line that just got |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
249 # split between \r and \n. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
250 and (self.raw_response.index(bad_eol) < |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
251 (len(self.raw_response) - 1))): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
252 logger.info('bogus line endings detected, ' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
253 'using %r for EOL', bad_eol) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
254 self._eol = bad_eol |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
255 break |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
256 # exit early if not at end of headers |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
257 if self._end_headers not in self.raw_response or self.headers: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
258 return |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
259 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
260 # handle 100-continue response |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
261 hdrs, body = self.raw_response.split(self._end_headers, 1) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
262 http_ver, status = hdrs.split(' ', 1) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
263 if status.startswith('100'): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
264 self.raw_response = body |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
265 logger.debug('continue seen, setting body to %r', body) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
266 return |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
267 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
268 # arriving here means we should parse response headers |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
269 # as all headers have arrived completely |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
270 hdrs, body = self.raw_response.split(self._end_headers, 1) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
271 del self.raw_response |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
272 if self._eol in hdrs: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
273 self.status_line, hdrs = hdrs.split(self._eol, 1) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
274 else: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
275 self.status_line = hdrs |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
276 hdrs = '' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
277 # TODO HTTP < 1.0 support |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
278 (self.http_version, self.status, |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
279 self.reason) = self.status_line.split(' ', 2) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
280 self.status = int(self.status) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
281 if self._eol != EOL: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
282 hdrs = hdrs.replace(self._eol, '\r\n') |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
283 headers = rfc822.Message(cStringIO.StringIO(hdrs)) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
284 if HDR_CONTENT_LENGTH in headers: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
285 self._content_len = int(headers[HDR_CONTENT_LENGTH]) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
286 if self.http_version == HTTP_VER_1_0: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
287 self.will_close = True |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
288 elif HDR_CONNECTION_CTRL in headers: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
289 self.will_close = ( |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
290 headers[HDR_CONNECTION_CTRL].lower() == CONNECTION_CLOSE) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
291 if self._content_len == 0: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
292 self._content_len = _LEN_CLOSE_IS_END |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
293 if (HDR_XFER_ENCODING in headers |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
294 and headers[HDR_XFER_ENCODING].lower() == XFER_ENCODING_CHUNKED): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
295 self._body = '' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
296 self._chunked_parsedata(body) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
297 self._chunked = True |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
298 if self._body is None: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
299 self._body = body |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
300 self.headers = headers |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
301 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
302 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
303 class HTTPConnection(object): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
304 """Connection to a single http server. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
305 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
306 Supports 100-continue and keepalives natively. Uses select() for |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
307 non-blocking socket operations. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
308 """ |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
309 http_version = HTTP_VER_1_1 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
310 response_class = HTTPResponse |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
311 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
312 def __init__(self, host, port=None, use_ssl=None, ssl_validator=None, |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
313 timeout=TIMEOUT_DEFAULT, |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
314 continue_timeout=TIMEOUT_ASSUME_CONTINUE, |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
315 proxy_hostport=None, **ssl_opts): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
316 """Create a new HTTPConnection. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
317 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
318 Args: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
319 host: The host to which we'll connect. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
320 port: Optional. The port over which we'll connect. Default 80 for |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
321 non-ssl, 443 for ssl. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
322 use_ssl: Optional. Wether to use ssl. Defaults to False if port is |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
323 not 443, true if port is 443. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
324 ssl_validator: a function(socket) to validate the ssl cert |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
325 timeout: Optional. Connection timeout, default is TIMEOUT_DEFAULT. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
326 continue_timeout: Optional. Timeout for waiting on an expected |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
327 "100 Continue" response. Default is TIMEOUT_ASSUME_CONTINUE. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
328 proxy_hostport: Optional. Tuple of (host, port) to use as an http |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
329 proxy for the connection. Default is to not use a proxy. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
330 """ |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
331 if port is None and host.count(':') == 1 or ']:' in host: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
332 host, port = host.rsplit(':', 1) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
333 port = int(port) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
334 if '[' in host: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
335 host = host[1:-1] |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
336 if use_ssl is None and port is None: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
337 use_ssl = False |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
338 port = 80 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
339 elif use_ssl is None: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
340 use_ssl = (port == 443) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
341 elif port is None: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
342 port = (use_ssl and 443 or 80) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
343 self.port = port |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
344 if use_ssl and not socketutil.have_ssl: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
345 raise Exception('ssl requested but unavailable on this Python') |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
346 self.ssl = use_ssl |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
347 self.ssl_opts = ssl_opts |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
348 self._ssl_validator = ssl_validator |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
349 self.host = host |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
350 self.sock = None |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
351 self._current_response = None |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
352 self._current_response_taken = False |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
353 if proxy_hostport is None: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
354 self._proxy_host = self._proxy_port = None |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
355 else: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
356 self._proxy_host, self._proxy_port = proxy_hostport |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
357 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
358 self.timeout = timeout |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
359 self.continue_timeout = continue_timeout |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
360 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
361 def _connect(self): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
362 """Connect to the host and port specified in __init__.""" |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
363 if self.sock: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
364 return |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
365 if self._proxy_host is not None: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
366 logger.info('Connecting to http proxy %s:%s', |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
367 self._proxy_host, self._proxy_port) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
368 sock = socketutil.create_connection((self._proxy_host, |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
369 self._proxy_port)) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
370 if self.ssl: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
371 # TODO proxy header support |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
372 data = self.buildheaders('CONNECT', '%s:%d' % (self.host, |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
373 self.port), |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
374 {}, HTTP_VER_1_0) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
375 sock.send(data) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
376 sock.setblocking(0) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
377 r = self.response_class(sock, self.timeout) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
378 timeout_exc = HTTPTimeoutException( |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
379 'Timed out waiting for CONNECT response from proxy') |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
380 while not r.complete(): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
381 try: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
382 if not r._select(): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
383 raise timeout_exc |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
384 except HTTPTimeoutException: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
385 # This raise/except pattern looks goofy, but |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
386 # _select can raise the timeout as well as the |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
387 # loop body. I wish it wasn't this convoluted, |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
388 # but I don't have a better solution |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
389 # immediately handy. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
390 raise timeout_exc |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
391 if r.status != 200: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
392 raise HTTPProxyConnectFailedException( |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
393 'Proxy connection failed: %d %s' % (r.status, |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
394 r.read())) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
395 logger.info('CONNECT (for SSL) to %s:%s via proxy succeeded.', |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
396 self.host, self.port) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
397 else: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
398 sock = socketutil.create_connection((self.host, self.port)) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
399 if self.ssl: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
400 logger.debug('wrapping socket for ssl with options %r', |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
401 self.ssl_opts) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
402 sock = socketutil.wrap_socket(sock, **self.ssl_opts) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
403 if self._ssl_validator: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
404 self._ssl_validator(sock) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
405 sock.setblocking(0) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
406 self.sock = sock |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
407 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
408 def buildheaders(self, method, path, headers, http_ver): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
409 if self.ssl and self.port == 443 or self.port == 80: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
410 # default port for protocol, so leave it out |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
411 hdrhost = self.host |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
412 else: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
413 # include nonstandard port in header |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
414 if ':' in self.host: # must be IPv6 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
415 hdrhost = '[%s]:%d' % (self.host, self.port) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
416 else: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
417 hdrhost = '%s:%d' % (self.host, self.port) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
418 if self._proxy_host and not self.ssl: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
419 # When talking to a regular http proxy we must send the |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
420 # full URI, but in all other cases we must not (although |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
421 # technically RFC 2616 says servers must accept our |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
422 # request if we screw up, experimentally few do that |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
423 # correctly.) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
424 assert path[0] == '/', 'path must start with a /' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
425 path = 'http://%s%s' % (hdrhost, path) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
426 outgoing = ['%s %s %s%s' % (method, path, http_ver, EOL)] |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
427 headers['host'] = ('Host', hdrhost) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
428 headers[HDR_ACCEPT_ENCODING] = (HDR_ACCEPT_ENCODING, 'identity') |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
429 for hdr, val in headers.itervalues(): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
430 outgoing.append('%s: %s%s' % (hdr, val, EOL)) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
431 outgoing.append(EOL) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
432 return ''.join(outgoing) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
433 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
434 def close(self): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
435 """Close the connection to the server. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
436 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
437 This is a no-op if the connection is already closed. The |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
438 connection may automatically close if requessted by the server |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
439 or required by the nature of a response. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
440 """ |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
441 if self.sock is None: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
442 return |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
443 self.sock.close() |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
444 self.sock = None |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
445 logger.info('closed connection to %s on %s', self.host, self.port) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
446 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
447 def busy(self): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
448 """Returns True if this connection object is currently in use. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
449 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
450 If a response is still pending, this will return True, even if |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
451 the request has finished sending. In the future, |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
452 HTTPConnection may transparently juggle multiple connections |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
453 to the server, in which case this will be useful to detect if |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
454 any of those connections is ready for use. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
455 """ |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
456 cr = self._current_response |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
457 if cr is not None: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
458 if self._current_response_taken: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
459 if cr.will_close: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
460 self.sock = None |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
461 self._current_response = None |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
462 return False |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
463 elif cr.complete(): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
464 self._current_response = None |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
465 return False |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
466 return True |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
467 return False |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
468 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
469 def request(self, method, path, body=None, headers={}, |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
470 expect_continue=False): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
471 """Send a request to the server. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
472 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
473 For increased flexibility, this does not return the response |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
474 object. Future versions of HTTPConnection that juggle multiple |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
475 sockets will be able to send (for example) 5 requests all at |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
476 once, and then let the requests arrive as data is |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
477 available. Use the `getresponse()` method to retrieve the |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
478 response. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
479 """ |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
480 if self.busy(): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
481 raise httplib.CannotSendRequest( |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
482 'Can not send another request before ' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
483 'current response is read!') |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
484 self._current_response_taken = False |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
485 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
486 logger.info('sending %s request for %s to %s on port %s', |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
487 method, path, self.host, self.port) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
488 hdrs = dict((k.lower(), (k, v)) for k, v in headers.iteritems()) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
489 if hdrs.get('expect', ('', ''))[1].lower() == '100-continue': |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
490 expect_continue = True |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
491 elif expect_continue: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
492 hdrs['expect'] = ('Expect', '100-Continue') |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
493 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
494 chunked = False |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
495 if body and HDR_CONTENT_LENGTH not in hdrs: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
496 if getattr(body, '__len__', False): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
497 hdrs[HDR_CONTENT_LENGTH] = (HDR_CONTENT_LENGTH, len(body)) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
498 elif getattr(body, 'read', False): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
499 hdrs[HDR_XFER_ENCODING] = (HDR_XFER_ENCODING, |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
500 XFER_ENCODING_CHUNKED) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
501 chunked = True |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
502 else: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
503 raise BadRequestData('body has no __len__() nor read()') |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
504 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
505 self._connect() |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
506 outgoing_headers = self.buildheaders( |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
507 method, path, hdrs, self.http_version) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
508 response = None |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
509 first = True |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
510 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
511 def reconnect(where): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
512 logger.info('reconnecting during %s', where) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
513 self.close() |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
514 self._connect() |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
515 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
516 while ((outgoing_headers or body) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
517 and not (response and response.complete())): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
518 select_timeout = self.timeout |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
519 out = outgoing_headers or body |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
520 blocking_on_continue = False |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
521 if expect_continue and not outgoing_headers and not ( |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
522 response and response.headers): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
523 logger.info( |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
524 'waiting up to %s seconds for' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
525 ' continue response from server', |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
526 self.continue_timeout) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
527 select_timeout = self.continue_timeout |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
528 blocking_on_continue = True |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
529 out = False |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
530 if out: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
531 w = [self.sock] |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
532 else: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
533 w = [] |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
534 r, w, x = select.select([self.sock], w, [], select_timeout) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
535 # if we were expecting a 100 continue and it's been long |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
536 # enough, just go ahead and assume it's ok. This is the |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
537 # recommended behavior from the RFC. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
538 if r == w == x == []: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
539 if blocking_on_continue: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
540 expect_continue = False |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
541 logger.info('no response to continue expectation from ' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
542 'server, optimistically sending request body') |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
543 else: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
544 raise HTTPTimeoutException('timeout sending data') |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
545 # TODO exceptional conditions with select? (what are those be?) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
546 # TODO if the response is loading, must we finish sending at all? |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
547 # |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
548 # Certainly not if it's going to close the connection and/or |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
549 # the response is already done...I think. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
550 was_first = first |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
551 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
552 # incoming data |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
553 if r: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
554 try: |
14341
5c3de67e7402
httpclient: import revision b8c3511a8cae from py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14293
diff
changeset
|
555 try: |
5c3de67e7402
httpclient: import revision b8c3511a8cae from py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14293
diff
changeset
|
556 data = r[0].recv(INCOMING_BUFFER_SIZE) |
5c3de67e7402
httpclient: import revision b8c3511a8cae from py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14293
diff
changeset
|
557 except socket.sslerror, e: |
5c3de67e7402
httpclient: import revision b8c3511a8cae from py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14293
diff
changeset
|
558 if e.args[0] != socket.SSL_ERROR_WANT_READ: |
5c3de67e7402
httpclient: import revision b8c3511a8cae from py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14293
diff
changeset
|
559 raise |
5c3de67e7402
httpclient: import revision b8c3511a8cae from py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14293
diff
changeset
|
560 logger.debug( |
5c3de67e7402
httpclient: import revision b8c3511a8cae from py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14293
diff
changeset
|
561 'SSL_WANT_READ while sending data, retrying...') |
5c3de67e7402
httpclient: import revision b8c3511a8cae from py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14293
diff
changeset
|
562 continue |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
563 if not data: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
564 logger.info('socket appears closed in read') |
14376
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
565 self.sock = None |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
566 self._current_response = None |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
567 # This if/elif ladder is a bit subtle, |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
568 # comments in each branch should help. |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
569 if response is not None and ( |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
570 response.complete() or |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
571 response._content_len == _LEN_CLOSE_IS_END): |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
572 # Server responded completely and then |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
573 # closed the socket. We should just shut |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
574 # things down and let the caller get their |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
575 # response. |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
576 logger.info('Got an early response, ' |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
577 'aborting remaining request.') |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
578 break |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
579 elif was_first and response is None: |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
580 # Most likely a keepalive that got killed |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
581 # on the server's end. Commonly happens |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
582 # after getting a really large response |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
583 # from the server. |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
584 logger.info( |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
585 'Connection appeared closed in read on first' |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
586 ' request loop iteration, will retry.') |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
587 reconnect('read') |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
588 continue |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
589 else: |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
590 # We didn't just send the first data hunk, |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
591 # and either have a partial response or no |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
592 # response at all. There's really nothing |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
593 # meaningful we can do here. |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
594 raise HTTPStateError( |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
595 'Connection appears closed after ' |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
596 'some request data was written, but the ' |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
597 'response was missing or incomplete!') |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
598 logger.debug('read %d bytes in request()', len(data)) |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
599 if response is None: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
600 response = self.response_class(r[0], self.timeout) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
601 response._load_response(data) |
14376
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
602 # Jump to the next select() call so we load more |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
603 # data if the server is still sending us content. |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
604 continue |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
605 except socket.error, e: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
606 if e[0] != errno.EPIPE and not was_first: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
607 raise |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
608 if (response._content_len |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
609 and response._content_len != _LEN_CLOSE_IS_END): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
610 outgoing_headers = sent_data + outgoing_headers |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
611 reconnect('read') |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
612 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
613 # outgoing data |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
614 if w and out: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
615 try: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
616 if getattr(out, 'read', False): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
617 data = out.read(OUTGOING_BUFFER_SIZE) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
618 if not data: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
619 continue |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
620 if len(data) < OUTGOING_BUFFER_SIZE: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
621 if chunked: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
622 body = '0' + EOL + EOL |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
623 else: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
624 body = None |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
625 if chunked: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
626 out = hex(len(data))[2:] + EOL + data + EOL |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
627 else: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
628 out = data |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
629 amt = w[0].send(out) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
630 except socket.error, e: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
631 if e[0] == socket.SSL_ERROR_WANT_WRITE and self.ssl: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
632 # This means that SSL hasn't flushed its buffer into |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
633 # the socket yet. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
634 # TODO: find a way to block on ssl flushing its buffer |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
635 # similar to selecting on a raw socket. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
636 continue |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
637 elif (e[0] not in (errno.ECONNRESET, errno.EPIPE) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
638 and not first): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
639 raise |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
640 reconnect('write') |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
641 amt = self.sock.send(out) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
642 logger.debug('sent %d', amt) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
643 first = False |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
644 # stash data we think we sent in case the socket breaks |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
645 # when we read from it |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
646 if was_first: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
647 sent_data = out[:amt] |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
648 if out is body: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
649 body = out[amt:] |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
650 else: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
651 outgoing_headers = out[amt:] |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
652 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
653 # close if the server response said to or responded before eating |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
654 # the whole request |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
655 if response is None: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
656 response = self.response_class(self.sock, self.timeout) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
657 complete = response.complete() |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
658 data_left = bool(outgoing_headers or body) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
659 if data_left: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
660 logger.info('stopped sending request early, ' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
661 'will close the socket to be safe.') |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
662 response.will_close = True |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
663 if response.will_close: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
664 # The socket will be closed by the response, so we disown |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
665 # the socket |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
666 self.sock = None |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
667 self._current_response = response |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
668 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
669 def getresponse(self): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
670 if self._current_response is None: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
671 raise httplib.ResponseNotReady() |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
672 r = self._current_response |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
673 while r.headers is None: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
674 r._select() |
14293
9adbb5ef0964
httpclient: import f4c380237fd5 to fix keepalive not working
Augie Fackler <durin42@gmail.com>
parents:
14243
diff
changeset
|
675 if r.will_close: |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
676 self.sock = None |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
677 self._current_response = None |
14293
9adbb5ef0964
httpclient: import f4c380237fd5 to fix keepalive not working
Augie Fackler <durin42@gmail.com>
parents:
14243
diff
changeset
|
678 elif r.complete(): |
9adbb5ef0964
httpclient: import f4c380237fd5 to fix keepalive not working
Augie Fackler <durin42@gmail.com>
parents:
14243
diff
changeset
|
679 self._current_response = None |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
680 else: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
681 self._current_response_taken = True |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
682 return r |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
683 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
684 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
685 class HTTPTimeoutException(httplib.HTTPException): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
686 """A timeout occurred while waiting on the server.""" |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
687 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
688 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
689 class BadRequestData(httplib.HTTPException): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
690 """Request body object has neither __len__ nor read.""" |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
691 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
692 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
693 class HTTPProxyConnectFailedException(httplib.HTTPException): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
694 """Connecting to the HTTP proxy failed.""" |
14376
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
695 |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
696 class HTTPStateError(httplib.HTTPException): |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
697 """Invalid internal state encountered.""" |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
698 # no-check-code |