annotate tests/test-cappedreader.py @ 37048:fc5e261915b9

wireproto: require POST for all HTTPv2 requests Wire protocol version 1 transfers argument data via request headers by default. This has historically caused problems because servers institute limits on the length of individual HTTP headers as well as the total size of all request headers. Mercurial servers can advertise the maximum length of an individual header. But there's no guarantee any intermediate HTTP agents will accept headers up to that length. In the existing wire protocol, server operators typically also key off the HTTP request method to implement authentication. For example, GET requests translate to read-only requests and can be allowed. But read-write commands must use POST and require authentication. This has typically worked because the only wire protocol commands that use POST modify the repo (e.g. the "unbundle" command). There is an experimental feature to enable clients to transmit argument data via POST request bodies. This is technically a better and more robust solution. But we can't enable it by default because of servers assuming POST means write access. In version 2 of the wire protocol, the permissions of a request are encoded in the URL. And with it being a new protocol in a new URL space, we're not constrained by backwards compatibility requirements. This commit adopts the technically superior mechanism of using HTTP request bodies to send argument data by requiring POST for all commands. Strictly speaking, it may be possible to send request bodies on GET requests. But my experience is that not all HTTP stacks support this. POST pretty much always works. Using POST for read-only operations does sacrifice some RESTful design purity. But this API cares about practicality, not about being in Roy T. Fielding's REST ivory tower. There's a chance we may relax this restriction in the future. But for now, I want to see how far we can get with a POST only API. Differential Revision: https://phab.mercurial-scm.org/D2837
author Gregory Szorc <gregory.szorc@gmail.com>
date Tue, 13 Mar 2018 11:57:43 -0700
parents 01e29e885600
children 2372284d9457
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
36362
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
1 from __future__ import absolute_import, print_function
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
2
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
3 import io
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
4 import unittest
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
5
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
6 from mercurial import (
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
7 util,
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
8 )
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
9
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
10 class CappedReaderTests(unittest.TestCase):
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
11 def testreadfull(self):
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
12 source = io.BytesIO(b'x' * 100)
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
13
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
14 reader = util.cappedreader(source, 10)
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
15 res = reader.read(10)
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
16 self.assertEqual(res, b'x' * 10)
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
17 self.assertEqual(source.tell(), 10)
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
18 source.seek(0)
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
19
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
20 reader = util.cappedreader(source, 15)
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
21 res = reader.read(16)
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
22 self.assertEqual(res, b'x' * 15)
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
23 self.assertEqual(source.tell(), 15)
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
24 source.seek(0)
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
25
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
26 reader = util.cappedreader(source, 100)
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
27 res = reader.read(100)
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
28 self.assertEqual(res, b'x' * 100)
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
29 self.assertEqual(source.tell(), 100)
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
30 source.seek(0)
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
31
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
32 reader = util.cappedreader(source, 50)
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
33 res = reader.read()
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
34 self.assertEqual(res, b'x' * 50)
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
35 self.assertEqual(source.tell(), 50)
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
36 source.seek(0)
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
37
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
38 def testreadnegative(self):
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
39 source = io.BytesIO(b'x' * 100)
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
40
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
41 reader = util.cappedreader(source, 20)
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
42 res = reader.read(-1)
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
43 self.assertEqual(res, b'x' * 20)
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
44 self.assertEqual(source.tell(), 20)
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
45 source.seek(0)
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
46
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
47 reader = util.cappedreader(source, 100)
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
48 res = reader.read(-1)
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
49 self.assertEqual(res, b'x' * 100)
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
50 self.assertEqual(source.tell(), 100)
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
51 source.seek(0)
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
52
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
53 def testreadmultiple(self):
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
54 source = io.BytesIO(b'x' * 100)
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
55
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
56 reader = util.cappedreader(source, 10)
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
57 for i in range(10):
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
58 res = reader.read(1)
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
59 self.assertEqual(res, b'x')
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
60 self.assertEqual(source.tell(), i + 1)
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
61
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
62 self.assertEqual(source.tell(), 10)
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
63 res = reader.read(1)
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
64 self.assertEqual(res, b'')
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
65 self.assertEqual(source.tell(), 10)
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
66 source.seek(0)
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
67
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
68 reader = util.cappedreader(source, 45)
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
69 for i in range(4):
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
70 res = reader.read(10)
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
71 self.assertEqual(res, b'x' * 10)
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
72 self.assertEqual(source.tell(), (i + 1) * 10)
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
73
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
74 res = reader.read(10)
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
75 self.assertEqual(res, b'x' * 5)
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
76 self.assertEqual(source.tell(), 45)
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
77
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
78 def readlimitpasteof(self):
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
79 source = io.BytesIO(b'x' * 100)
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
80
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
81 reader = util.cappedreader(source, 1024)
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
82 res = reader.read(1000)
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
83 self.assertEqual(res, b'x' * 100)
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
84 self.assertEqual(source.tell(), 100)
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
85 res = reader.read(1000)
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
86 self.assertEqual(res, b'')
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
87 self.assertEqual(source.tell(), 100)
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
88
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
89 if __name__ == '__main__':
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
90 import silenttestrunner
01e29e885600 util: add a file object proxy that can read at most N bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
91 silenttestrunner.main(__name__)