Mercurial > hg
annotate mercurial/httpclient/__init__.py @ 16643:24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
author | Augie Fackler <raf@durin42.com> |
---|---|
date | Fri, 04 May 2012 16:00:33 -0500 |
parents | c81dce8a7bb6 |
children | 69af967b6d6f |
rev | line source |
---|---|
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
1 # Copyright 2010, Google Inc. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
2 # All rights reserved. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
3 # |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
4 # Redistribution and use in source and binary forms, with or without |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
5 # modification, are permitted provided that the following conditions are |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
6 # met: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
7 # |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
8 # * Redistributions of source code must retain the above copyright |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
9 # notice, this list of conditions and the following disclaimer. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
10 # * Redistributions in binary form must reproduce the above |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
11 # copyright notice, this list of conditions and the following disclaimer |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
12 # in the documentation and/or other materials provided with the |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
13 # distribution. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
14 # * Neither the name of Google Inc. nor the names of its |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
15 # contributors may be used to endorse or promote products derived from |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
16 # this software without specific prior written permission. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
17 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
18 # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
19 # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
20 # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
21 # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
22 # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
23 # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
24 # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
25 # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
26 # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
27 # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
28 # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
29 """Improved HTTP/1.1 client library |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
30 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
31 This library contains an HTTPConnection which is similar to the one in |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
32 httplib, but has several additional features: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
33 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
34 * supports keepalives natively |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
35 * uses select() to block for incoming data |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
36 * notices when the server responds early to a request |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
37 * implements ssl inline instead of in a different class |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
38 """ |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
39 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
40 import cStringIO |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
41 import errno |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
42 import httplib |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
43 import logging |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
44 import rfc822 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
45 import select |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
46 import socket |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
47 |
16643
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
48 import _readers |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
49 import socketutil |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
50 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
51 logger = logging.getLogger(__name__) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
52 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
53 __all__ = ['HTTPConnection', 'HTTPResponse'] |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
54 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
55 HTTP_VER_1_0 = 'HTTP/1.0' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
56 HTTP_VER_1_1 = 'HTTP/1.1' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
57 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
58 OUTGOING_BUFFER_SIZE = 1 << 15 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
59 INCOMING_BUFFER_SIZE = 1 << 20 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
60 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
61 HDR_ACCEPT_ENCODING = 'accept-encoding' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
62 HDR_CONNECTION_CTRL = 'connection' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
63 HDR_CONTENT_LENGTH = 'content-length' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
64 HDR_XFER_ENCODING = 'transfer-encoding' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
65 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
66 XFER_ENCODING_CHUNKED = 'chunked' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
67 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
68 CONNECTION_CLOSE = 'close' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
69 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
70 EOL = '\r\n' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
71 _END_HEADERS = EOL * 2 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
72 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
73 # Based on some searching around, 1 second seems like a reasonable |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
74 # default here. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
75 TIMEOUT_ASSUME_CONTINUE = 1 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
76 TIMEOUT_DEFAULT = None |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
77 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
78 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
79 class HTTPResponse(object): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
80 """Response from an HTTP server. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
81 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
82 The response will continue to load as available. If you need the |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
83 complete response before continuing, check the .complete() method. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
84 """ |
16643
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
85 def __init__(self, sock, timeout, method): |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
86 self.sock = sock |
16643
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
87 self.method = method |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
88 self.raw_response = '' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
89 self._headers_len = 0 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
90 self.headers = None |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
91 self.will_close = False |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
92 self.status_line = '' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
93 self.status = None |
16643
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
94 self.continued = False |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
95 self.http_version = None |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
96 self.reason = None |
16643
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
97 self._reader = None |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
98 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
99 self._read_location = 0 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
100 self._eol = EOL |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
101 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
102 self._timeout = timeout |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
103 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
104 @property |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
105 def _end_headers(self): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
106 return self._eol * 2 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
107 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
108 def complete(self): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
109 """Returns true if this response is completely loaded. |
14376
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
110 |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
111 Note that if this is a connection where complete means the |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
112 socket is closed, this will nearly always return False, even |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
113 in cases where all the data has actually been loaded. |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
114 """ |
16643
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
115 if self._reader: |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
116 return self._reader.done() |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
117 |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
118 def _close(self): |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
119 if self._reader is not None: |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
120 self._reader._close() |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
121 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
122 def readline(self): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
123 """Read a single line from the response body. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
124 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
125 This may block until either a line ending is found or the |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
126 response is complete. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
127 """ |
16643
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
128 # TODO: move this into the reader interface where it can be |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
129 # smarter (and probably avoid copies) |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
130 bytes = [] |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
131 while not bytes: |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
132 try: |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
133 bytes = [self._reader.read(1)] |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
134 except _readers.ReadNotReady: |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
135 self._select() |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
136 while bytes[-1] != '\n' and not self.complete(): |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
137 self._select() |
16643
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
138 bytes.append(self._reader.read(1)) |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
139 if bytes[-1] != '\n': |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
140 next = self._reader.read(1) |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
141 while next and next != '\n': |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
142 bytes.append(next) |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
143 next = self._reader.read(1) |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
144 bytes.append(next) |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
145 return ''.join(bytes) |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
146 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
147 def read(self, length=None): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
148 # if length is None, unbounded read |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
149 while (not self.complete() # never select on a finished read |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
150 and (not length # unbounded, so we wait for complete() |
16643
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
151 or length > self._reader.available_data)): |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
152 self._select() |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
153 if not length: |
16643
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
154 length = self._reader.available_data |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
155 r = self._reader.read(length) |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
156 if self.complete() and self.will_close: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
157 self.sock.close() |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
158 return r |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
159 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
160 def _select(self): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
161 r, _, _ = select.select([self.sock], [], [], self._timeout) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
162 if not r: |
16643
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
163 # socket was not readable. If the response is not |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
164 # complete, raise a timeout. |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
165 if not self.complete(): |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
166 logger.info('timed out with timeout of %s', self._timeout) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
167 raise HTTPTimeoutException('timeout reading data') |
14341
5c3de67e7402
httpclient: import revision b8c3511a8cae from py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14293
diff
changeset
|
168 try: |
5c3de67e7402
httpclient: import revision b8c3511a8cae from py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14293
diff
changeset
|
169 data = self.sock.recv(INCOMING_BUFFER_SIZE) |
5c3de67e7402
httpclient: import revision b8c3511a8cae from py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14293
diff
changeset
|
170 except socket.sslerror, e: |
5c3de67e7402
httpclient: import revision b8c3511a8cae from py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14293
diff
changeset
|
171 if e.args[0] != socket.SSL_ERROR_WANT_READ: |
5c3de67e7402
httpclient: import revision b8c3511a8cae from py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14293
diff
changeset
|
172 raise |
5c3de67e7402
httpclient: import revision b8c3511a8cae from py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14293
diff
changeset
|
173 logger.debug('SSL_WANT_READ in _select, should retry later') |
5c3de67e7402
httpclient: import revision b8c3511a8cae from py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14293
diff
changeset
|
174 return True |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
175 logger.debug('response read %d data during _select', len(data)) |
16643
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
176 # If the socket was readable and no data was read, that means |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
177 # the socket was closed. Inform the reader (if any) so it can |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
178 # raise an exception if this is an invalid situation. |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
179 if not data: |
16643
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
180 if self._reader: |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
181 self._reader._close() |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
182 return False |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
183 else: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
184 self._load_response(data) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
185 return True |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
186 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
187 def _load_response(self, data): |
16643
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
188 # Being here implies we're not at the end of the headers yet, |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
189 # since at the end of this method if headers were completely |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
190 # loaded we replace this method with the load() method of the |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
191 # reader we created. |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
192 self.raw_response += data |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
193 # This is a bogus server with bad line endings |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
194 if self._eol not in self.raw_response: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
195 for bad_eol in ('\n', '\r'): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
196 if (bad_eol in self.raw_response |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
197 # verify that bad_eol is not the end of the incoming data |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
198 # as this could be a response line that just got |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
199 # split between \r and \n. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
200 and (self.raw_response.index(bad_eol) < |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
201 (len(self.raw_response) - 1))): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
202 logger.info('bogus line endings detected, ' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
203 'using %r for EOL', bad_eol) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
204 self._eol = bad_eol |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
205 break |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
206 # exit early if not at end of headers |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
207 if self._end_headers not in self.raw_response or self.headers: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
208 return |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
209 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
210 # handle 100-continue response |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
211 hdrs, body = self.raw_response.split(self._end_headers, 1) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
212 http_ver, status = hdrs.split(' ', 1) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
213 if status.startswith('100'): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
214 self.raw_response = body |
16643
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
215 self.continued = True |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
216 logger.debug('continue seen, setting body to %r', body) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
217 return |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
218 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
219 # arriving here means we should parse response headers |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
220 # as all headers have arrived completely |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
221 hdrs, body = self.raw_response.split(self._end_headers, 1) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
222 del self.raw_response |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
223 if self._eol in hdrs: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
224 self.status_line, hdrs = hdrs.split(self._eol, 1) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
225 else: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
226 self.status_line = hdrs |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
227 hdrs = '' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
228 # TODO HTTP < 1.0 support |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
229 (self.http_version, self.status, |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
230 self.reason) = self.status_line.split(' ', 2) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
231 self.status = int(self.status) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
232 if self._eol != EOL: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
233 hdrs = hdrs.replace(self._eol, '\r\n') |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
234 headers = rfc822.Message(cStringIO.StringIO(hdrs)) |
16643
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
235 content_len = None |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
236 if HDR_CONTENT_LENGTH in headers: |
16643
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
237 content_len = int(headers[HDR_CONTENT_LENGTH]) |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
238 if self.http_version == HTTP_VER_1_0: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
239 self.will_close = True |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
240 elif HDR_CONNECTION_CTRL in headers: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
241 self.will_close = ( |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
242 headers[HDR_CONNECTION_CTRL].lower() == CONNECTION_CLOSE) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
243 if (HDR_XFER_ENCODING in headers |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
244 and headers[HDR_XFER_ENCODING].lower() == XFER_ENCODING_CHUNKED): |
16643
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
245 self._reader = _readers.ChunkedReader(self._eol) |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
246 logger.debug('using a chunked reader') |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
247 else: |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
248 # HEAD responses are forbidden from returning a body, and |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
249 # it's implausible for a CONNECT response to use |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
250 # close-is-end logic for an OK response. |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
251 if (self.method == 'HEAD' or |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
252 (self.method == 'CONNECT' and content_len is None)): |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
253 content_len = 0 |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
254 if content_len is not None: |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
255 logger.debug('using a content-length reader with length %d', |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
256 content_len) |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
257 self._reader = _readers.ContentLengthReader(content_len) |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
258 else: |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
259 # Response body had no length specified and is not |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
260 # chunked, so the end of the body will only be |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
261 # identifiable by the termination of the socket by the |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
262 # server. My interpretation of the spec means that we |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
263 # are correct in hitting this case if |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
264 # transfer-encoding, content-length, and |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
265 # connection-control were left unspecified. |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
266 self._reader = _readers.CloseIsEndReader() |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
267 logger.debug('using a close-is-end reader') |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
268 self.will_close = True |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
269 |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
270 if body: |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
271 self._reader._load(body) |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
272 logger.debug('headers complete') |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
273 self.headers = headers |
16643
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
274 self._load_response = self._reader._load |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
275 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
276 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
277 class HTTPConnection(object): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
278 """Connection to a single http server. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
279 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
280 Supports 100-continue and keepalives natively. Uses select() for |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
281 non-blocking socket operations. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
282 """ |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
283 http_version = HTTP_VER_1_1 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
284 response_class = HTTPResponse |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
285 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
286 def __init__(self, host, port=None, use_ssl=None, ssl_validator=None, |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
287 timeout=TIMEOUT_DEFAULT, |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
288 continue_timeout=TIMEOUT_ASSUME_CONTINUE, |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
289 proxy_hostport=None, **ssl_opts): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
290 """Create a new HTTPConnection. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
291 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
292 Args: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
293 host: The host to which we'll connect. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
294 port: Optional. The port over which we'll connect. Default 80 for |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
295 non-ssl, 443 for ssl. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
296 use_ssl: Optional. Wether to use ssl. Defaults to False if port is |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
297 not 443, true if port is 443. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
298 ssl_validator: a function(socket) to validate the ssl cert |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
299 timeout: Optional. Connection timeout, default is TIMEOUT_DEFAULT. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
300 continue_timeout: Optional. Timeout for waiting on an expected |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
301 "100 Continue" response. Default is TIMEOUT_ASSUME_CONTINUE. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
302 proxy_hostport: Optional. Tuple of (host, port) to use as an http |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
303 proxy for the connection. Default is to not use a proxy. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
304 """ |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
305 if port is None and host.count(':') == 1 or ']:' in host: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
306 host, port = host.rsplit(':', 1) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
307 port = int(port) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
308 if '[' in host: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
309 host = host[1:-1] |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
310 if use_ssl is None and port is None: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
311 use_ssl = False |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
312 port = 80 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
313 elif use_ssl is None: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
314 use_ssl = (port == 443) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
315 elif port is None: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
316 port = (use_ssl and 443 or 80) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
317 self.port = port |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
318 if use_ssl and not socketutil.have_ssl: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
319 raise Exception('ssl requested but unavailable on this Python') |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
320 self.ssl = use_ssl |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
321 self.ssl_opts = ssl_opts |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
322 self._ssl_validator = ssl_validator |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
323 self.host = host |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
324 self.sock = None |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
325 self._current_response = None |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
326 self._current_response_taken = False |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
327 if proxy_hostport is None: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
328 self._proxy_host = self._proxy_port = None |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
329 else: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
330 self._proxy_host, self._proxy_port = proxy_hostport |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
331 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
332 self.timeout = timeout |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
333 self.continue_timeout = continue_timeout |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
334 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
335 def _connect(self): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
336 """Connect to the host and port specified in __init__.""" |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
337 if self.sock: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
338 return |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
339 if self._proxy_host is not None: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
340 logger.info('Connecting to http proxy %s:%s', |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
341 self._proxy_host, self._proxy_port) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
342 sock = socketutil.create_connection((self._proxy_host, |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
343 self._proxy_port)) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
344 if self.ssl: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
345 # TODO proxy header support |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
346 data = self.buildheaders('CONNECT', '%s:%d' % (self.host, |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
347 self.port), |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
348 {}, HTTP_VER_1_0) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
349 sock.send(data) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
350 sock.setblocking(0) |
16643
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
351 r = self.response_class(sock, self.timeout, 'CONNECT') |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
352 timeout_exc = HTTPTimeoutException( |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
353 'Timed out waiting for CONNECT response from proxy') |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
354 while not r.complete(): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
355 try: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
356 if not r._select(): |
16643
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
357 if not r.complete(): |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
358 raise timeout_exc |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
359 except HTTPTimeoutException: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
360 # This raise/except pattern looks goofy, but |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
361 # _select can raise the timeout as well as the |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
362 # loop body. I wish it wasn't this convoluted, |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
363 # but I don't have a better solution |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
364 # immediately handy. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
365 raise timeout_exc |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
366 if r.status != 200: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
367 raise HTTPProxyConnectFailedException( |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
368 'Proxy connection failed: %d %s' % (r.status, |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
369 r.read())) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
370 logger.info('CONNECT (for SSL) to %s:%s via proxy succeeded.', |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
371 self.host, self.port) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
372 else: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
373 sock = socketutil.create_connection((self.host, self.port)) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
374 if self.ssl: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
375 logger.debug('wrapping socket for ssl with options %r', |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
376 self.ssl_opts) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
377 sock = socketutil.wrap_socket(sock, **self.ssl_opts) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
378 if self._ssl_validator: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
379 self._ssl_validator(sock) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
380 sock.setblocking(0) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
381 self.sock = sock |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
382 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
383 def buildheaders(self, method, path, headers, http_ver): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
384 if self.ssl and self.port == 443 or self.port == 80: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
385 # default port for protocol, so leave it out |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
386 hdrhost = self.host |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
387 else: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
388 # include nonstandard port in header |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
389 if ':' in self.host: # must be IPv6 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
390 hdrhost = '[%s]:%d' % (self.host, self.port) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
391 else: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
392 hdrhost = '%s:%d' % (self.host, self.port) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
393 if self._proxy_host and not self.ssl: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
394 # When talking to a regular http proxy we must send the |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
395 # full URI, but in all other cases we must not (although |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
396 # technically RFC 2616 says servers must accept our |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
397 # request if we screw up, experimentally few do that |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
398 # correctly.) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
399 assert path[0] == '/', 'path must start with a /' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
400 path = 'http://%s%s' % (hdrhost, path) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
401 outgoing = ['%s %s %s%s' % (method, path, http_ver, EOL)] |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
402 headers['host'] = ('Host', hdrhost) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
403 headers[HDR_ACCEPT_ENCODING] = (HDR_ACCEPT_ENCODING, 'identity') |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
404 for hdr, val in headers.itervalues(): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
405 outgoing.append('%s: %s%s' % (hdr, val, EOL)) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
406 outgoing.append(EOL) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
407 return ''.join(outgoing) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
408 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
409 def close(self): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
410 """Close the connection to the server. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
411 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
412 This is a no-op if the connection is already closed. The |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
413 connection may automatically close if requessted by the server |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
414 or required by the nature of a response. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
415 """ |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
416 if self.sock is None: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
417 return |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
418 self.sock.close() |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
419 self.sock = None |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
420 logger.info('closed connection to %s on %s', self.host, self.port) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
421 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
422 def busy(self): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
423 """Returns True if this connection object is currently in use. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
424 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
425 If a response is still pending, this will return True, even if |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
426 the request has finished sending. In the future, |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
427 HTTPConnection may transparently juggle multiple connections |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
428 to the server, in which case this will be useful to detect if |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
429 any of those connections is ready for use. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
430 """ |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
431 cr = self._current_response |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
432 if cr is not None: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
433 if self._current_response_taken: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
434 if cr.will_close: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
435 self.sock = None |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
436 self._current_response = None |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
437 return False |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
438 elif cr.complete(): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
439 self._current_response = None |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
440 return False |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
441 return True |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
442 return False |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
443 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
444 def request(self, method, path, body=None, headers={}, |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
445 expect_continue=False): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
446 """Send a request to the server. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
447 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
448 For increased flexibility, this does not return the response |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
449 object. Future versions of HTTPConnection that juggle multiple |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
450 sockets will be able to send (for example) 5 requests all at |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
451 once, and then let the requests arrive as data is |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
452 available. Use the `getresponse()` method to retrieve the |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
453 response. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
454 """ |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
455 if self.busy(): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
456 raise httplib.CannotSendRequest( |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
457 'Can not send another request before ' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
458 'current response is read!') |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
459 self._current_response_taken = False |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
460 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
461 logger.info('sending %s request for %s to %s on port %s', |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
462 method, path, self.host, self.port) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
463 hdrs = dict((k.lower(), (k, v)) for k, v in headers.iteritems()) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
464 if hdrs.get('expect', ('', ''))[1].lower() == '100-continue': |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
465 expect_continue = True |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
466 elif expect_continue: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
467 hdrs['expect'] = ('Expect', '100-Continue') |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
468 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
469 chunked = False |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
470 if body and HDR_CONTENT_LENGTH not in hdrs: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
471 if getattr(body, '__len__', False): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
472 hdrs[HDR_CONTENT_LENGTH] = (HDR_CONTENT_LENGTH, len(body)) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
473 elif getattr(body, 'read', False): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
474 hdrs[HDR_XFER_ENCODING] = (HDR_XFER_ENCODING, |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
475 XFER_ENCODING_CHUNKED) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
476 chunked = True |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
477 else: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
478 raise BadRequestData('body has no __len__() nor read()') |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
479 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
480 self._connect() |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
481 outgoing_headers = self.buildheaders( |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
482 method, path, hdrs, self.http_version) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
483 response = None |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
484 first = True |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
485 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
486 def reconnect(where): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
487 logger.info('reconnecting during %s', where) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
488 self.close() |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
489 self._connect() |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
490 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
491 while ((outgoing_headers or body) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
492 and not (response and response.complete())): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
493 select_timeout = self.timeout |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
494 out = outgoing_headers or body |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
495 blocking_on_continue = False |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
496 if expect_continue and not outgoing_headers and not ( |
16643
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
497 response and (response.headers or response.continued)): |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
498 logger.info( |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
499 'waiting up to %s seconds for' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
500 ' continue response from server', |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
501 self.continue_timeout) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
502 select_timeout = self.continue_timeout |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
503 blocking_on_continue = True |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
504 out = False |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
505 if out: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
506 w = [self.sock] |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
507 else: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
508 w = [] |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
509 r, w, x = select.select([self.sock], w, [], select_timeout) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
510 # if we were expecting a 100 continue and it's been long |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
511 # enough, just go ahead and assume it's ok. This is the |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
512 # recommended behavior from the RFC. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
513 if r == w == x == []: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
514 if blocking_on_continue: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
515 expect_continue = False |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
516 logger.info('no response to continue expectation from ' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
517 'server, optimistically sending request body') |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
518 else: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
519 raise HTTPTimeoutException('timeout sending data') |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
520 was_first = first |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
521 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
522 # incoming data |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
523 if r: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
524 try: |
14341
5c3de67e7402
httpclient: import revision b8c3511a8cae from py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14293
diff
changeset
|
525 try: |
5c3de67e7402
httpclient: import revision b8c3511a8cae from py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14293
diff
changeset
|
526 data = r[0].recv(INCOMING_BUFFER_SIZE) |
5c3de67e7402
httpclient: import revision b8c3511a8cae from py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14293
diff
changeset
|
527 except socket.sslerror, e: |
5c3de67e7402
httpclient: import revision b8c3511a8cae from py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14293
diff
changeset
|
528 if e.args[0] != socket.SSL_ERROR_WANT_READ: |
5c3de67e7402
httpclient: import revision b8c3511a8cae from py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14293
diff
changeset
|
529 raise |
5c3de67e7402
httpclient: import revision b8c3511a8cae from py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14293
diff
changeset
|
530 logger.debug( |
5c3de67e7402
httpclient: import revision b8c3511a8cae from py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14293
diff
changeset
|
531 'SSL_WANT_READ while sending data, retrying...') |
5c3de67e7402
httpclient: import revision b8c3511a8cae from py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14293
diff
changeset
|
532 continue |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
533 if not data: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
534 logger.info('socket appears closed in read') |
14376
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
535 self.sock = None |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
536 self._current_response = None |
16643
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
537 if response is not None: |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
538 response._close() |
14376
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
539 # This if/elif ladder is a bit subtle, |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
540 # comments in each branch should help. |
16643
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
541 if response is not None and response.complete(): |
14376
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
542 # Server responded completely and then |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
543 # closed the socket. We should just shut |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
544 # things down and let the caller get their |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
545 # response. |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
546 logger.info('Got an early response, ' |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
547 'aborting remaining request.') |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
548 break |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
549 elif was_first and response is None: |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
550 # Most likely a keepalive that got killed |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
551 # on the server's end. Commonly happens |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
552 # after getting a really large response |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
553 # from the server. |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
554 logger.info( |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
555 'Connection appeared closed in read on first' |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
556 ' request loop iteration, will retry.') |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
557 reconnect('read') |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
558 continue |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
559 else: |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
560 # We didn't just send the first data hunk, |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
561 # and either have a partial response or no |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
562 # response at all. There's really nothing |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
563 # meaningful we can do here. |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
564 raise HTTPStateError( |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
565 'Connection appears closed after ' |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
566 'some request data was written, but the ' |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
567 'response was missing or incomplete!') |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
568 logger.debug('read %d bytes in request()', len(data)) |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
569 if response is None: |
16643
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
570 response = self.response_class(r[0], self.timeout, method) |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
571 response._load_response(data) |
14376
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
572 # Jump to the next select() call so we load more |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
573 # data if the server is still sending us content. |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
574 continue |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
575 except socket.error, e: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
576 if e[0] != errno.EPIPE and not was_first: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
577 raise |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
578 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
579 # outgoing data |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
580 if w and out: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
581 try: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
582 if getattr(out, 'read', False): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
583 data = out.read(OUTGOING_BUFFER_SIZE) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
584 if not data: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
585 continue |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
586 if len(data) < OUTGOING_BUFFER_SIZE: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
587 if chunked: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
588 body = '0' + EOL + EOL |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
589 else: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
590 body = None |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
591 if chunked: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
592 out = hex(len(data))[2:] + EOL + data + EOL |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
593 else: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
594 out = data |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
595 amt = w[0].send(out) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
596 except socket.error, e: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
597 if e[0] == socket.SSL_ERROR_WANT_WRITE and self.ssl: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
598 # This means that SSL hasn't flushed its buffer into |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
599 # the socket yet. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
600 # TODO: find a way to block on ssl flushing its buffer |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
601 # similar to selecting on a raw socket. |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
602 continue |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
603 elif (e[0] not in (errno.ECONNRESET, errno.EPIPE) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
604 and not first): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
605 raise |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
606 reconnect('write') |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
607 amt = self.sock.send(out) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
608 logger.debug('sent %d', amt) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
609 first = False |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
610 # stash data we think we sent in case the socket breaks |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
611 # when we read from it |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
612 if was_first: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
613 sent_data = out[:amt] |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
614 if out is body: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
615 body = out[amt:] |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
616 else: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
617 outgoing_headers = out[amt:] |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
618 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
619 # close if the server response said to or responded before eating |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
620 # the whole request |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
621 if response is None: |
16643
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
622 response = self.response_class(self.sock, self.timeout, method) |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
623 complete = response.complete() |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
624 data_left = bool(outgoing_headers or body) |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
625 if data_left: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
626 logger.info('stopped sending request early, ' |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
627 'will close the socket to be safe.') |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
628 response.will_close = True |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
629 if response.will_close: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
630 # The socket will be closed by the response, so we disown |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
631 # the socket |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
632 self.sock = None |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
633 self._current_response = response |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
634 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
635 def getresponse(self): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
636 if self._current_response is None: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
637 raise httplib.ResponseNotReady() |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
638 r = self._current_response |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
639 while r.headers is None: |
16643
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
640 if not r._select() and not r.complete(): |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
641 raise _readers.HTTPRemoteClosedError() |
14293
9adbb5ef0964
httpclient: import f4c380237fd5 to fix keepalive not working
Augie Fackler <durin42@gmail.com>
parents:
14243
diff
changeset
|
642 if r.will_close: |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
643 self.sock = None |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
644 self._current_response = None |
14293
9adbb5ef0964
httpclient: import f4c380237fd5 to fix keepalive not working
Augie Fackler <durin42@gmail.com>
parents:
14243
diff
changeset
|
645 elif r.complete(): |
9adbb5ef0964
httpclient: import f4c380237fd5 to fix keepalive not working
Augie Fackler <durin42@gmail.com>
parents:
14243
diff
changeset
|
646 self._current_response = None |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
647 else: |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
648 self._current_response_taken = True |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
649 return r |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
650 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
651 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
652 class HTTPTimeoutException(httplib.HTTPException): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
653 """A timeout occurred while waiting on the server.""" |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
654 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
655 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
656 class BadRequestData(httplib.HTTPException): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
657 """Request body object has neither __len__ nor read.""" |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
658 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
659 |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
660 class HTTPProxyConnectFailedException(httplib.HTTPException): |
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
661 """Connecting to the HTTP proxy failed.""" |
14376
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
662 |
15218
c81dce8a7bb6
httpclient: update to 07d8c356f4d1 of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14376
diff
changeset
|
663 |
14376
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
664 class HTTPStateError(httplib.HTTPException): |
a75e0f4ba0ab
httpclient: import revision fc731618702a of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14341
diff
changeset
|
665 """Invalid internal state encountered.""" |
15218
c81dce8a7bb6
httpclient: update to 07d8c356f4d1 of py-nonblocking-http
Augie Fackler <durin42@gmail.com>
parents:
14376
diff
changeset
|
666 |
16643
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
667 # Forward this exception type from _readers since it needs to be part |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
668 # of the public API. |
24dbef11f477
httpclient: update to revision 892730fe7f46 of httpplus
Augie Fackler <raf@durin42.com>
parents:
15218
diff
changeset
|
669 HTTPRemoteClosedError = _readers.HTTPRemoteClosedError |
14243
861f28212398
Import new http library as mercurial.httpclient.
Augie Fackler <durin42@gmail.com>
parents:
diff
changeset
|
670 # no-check-code |