Mercurial > hg
annotate tests/test-wireproto.py @ 37048:fc5e261915b9
wireproto: require POST for all HTTPv2 requests
Wire protocol version 1 transfers argument data via request
headers by default. This has historically caused problems because
servers institute limits on the length of individual HTTP headers
as well as the total size of all request headers. Mercurial servers
can advertise the maximum length of an individual header. But
there's no guarantee any intermediate HTTP agents will accept
headers up to that length.
In the existing wire protocol, server operators typically also
key off the HTTP request method to implement authentication.
For example, GET requests translate to read-only requests and
can be allowed. But read-write commands must use POST and require
authentication. This has typically worked because the only wire
protocol commands that use POST modify the repo (e.g. the
"unbundle" command).
There is an experimental feature to enable clients to transmit
argument data via POST request bodies. This is technically a
better and more robust solution. But we can't enable it by default
because of servers assuming POST means write access.
In version 2 of the wire protocol, the permissions of a request
are encoded in the URL. And with it being a new protocol in a new
URL space, we're not constrained by backwards compatibility
requirements.
This commit adopts the technically superior mechanism of using
HTTP request bodies to send argument data by requiring POST for
all commands. Strictly speaking, it may be possible to send
request bodies on GET requests. But my experience is that not all
HTTP stacks support this. POST pretty much always works. Using POST
for read-only operations does sacrifice some RESTful design
purity. But this API cares about practicality, not about being
in Roy T. Fielding's REST ivory tower.
There's a chance we may relax this restriction in the future. But
for now, I want to see how far we can get with a POST only API.
Differential Revision: https://phab.mercurial-scm.org/D2837
author | Gregory Szorc <gregory.szorc@gmail.com> |
---|---|
date | Tue, 13 Mar 2018 11:57:43 -0700 |
parents | 65615c29e74f |
children | 45b39c69fae0 |
rev | line source |
---|---|
28675
fcafd84bc9c5
py3: make test-wireproto use print_function
Pulkit Goyal <7895pulkit@gmail.com>
parents:
28438
diff
changeset
|
1 from __future__ import absolute_import, print_function |
27301
5defcb7d6e5f
tests: use absolulte_import in test-wireproto.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25912
diff
changeset
|
2 |
28860
50d11dd8ac02
py3: use multi-line import in test-wireproto.py
timeless <timeless@mozdev.org>
parents:
28675
diff
changeset
|
3 from mercurial import ( |
36074
2f7290555c96
wireproto: introduce type for raw byte responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
33806
diff
changeset
|
4 error, |
36555
c28de190e20a
py3: port tests/test-wireproto.py to Python 3
Pulkit Goyal <7895pulkit@gmail.com>
parents:
36074
diff
changeset
|
5 pycompat, |
36944
65615c29e74f
tests: fix test-wireproto.py to work around serverrepo() not having a ui
Augie Fackler <augie@google.com>
parents:
36801
diff
changeset
|
6 ui as uimod, |
28861
86db5cb55d46
pycompat: switch to util.stringio for py3 compat
timeless <timeless@mozdev.org>
parents:
28860
diff
changeset
|
7 util, |
28860
50d11dd8ac02
py3: use multi-line import in test-wireproto.py
timeless <timeless@mozdev.org>
parents:
28675
diff
changeset
|
8 wireproto, |
36074
2f7290555c96
wireproto: introduce type for raw byte responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
33806
diff
changeset
|
9 wireprototypes, |
28860
50d11dd8ac02
py3: use multi-line import in test-wireproto.py
timeless <timeless@mozdev.org>
parents:
28675
diff
changeset
|
10 ) |
28861
86db5cb55d46
pycompat: switch to util.stringio for py3 compat
timeless <timeless@mozdev.org>
parents:
28860
diff
changeset
|
11 stringio = util.stringio |
14622
bd88561afb4b
wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff
changeset
|
12 |
14764
a7d5816087a9
classes: fix class style problems found by b071cd58af50
Thomas Arendsen Hein <thomas@intevation.de>
parents:
14622
diff
changeset
|
13 class proto(object): |
14622
bd88561afb4b
wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff
changeset
|
14 def __init__(self, args): |
bd88561afb4b
wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff
changeset
|
15 self.args = args |
bd88561afb4b
wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff
changeset
|
16 def getargs(self, spec): |
bd88561afb4b
wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff
changeset
|
17 args = self.args |
36555
c28de190e20a
py3: port tests/test-wireproto.py to Python 3
Pulkit Goyal <7895pulkit@gmail.com>
parents:
36074
diff
changeset
|
18 args.setdefault(b'*', {}) |
14622
bd88561afb4b
wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff
changeset
|
19 names = spec.split() |
bd88561afb4b
wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff
changeset
|
20 return [args[n] for n in names] |
bd88561afb4b
wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff
changeset
|
21 |
36801
66de4555cefd
wireproto: formalize permissions checking as part of protocol interface
Gregory Szorc <gregory.szorc@gmail.com>
parents:
36555
diff
changeset
|
22 def checkperm(self, perm): |
66de4555cefd
wireproto: formalize permissions checking as part of protocol interface
Gregory Szorc <gregory.szorc@gmail.com>
parents:
36555
diff
changeset
|
23 pass |
66de4555cefd
wireproto: formalize permissions checking as part of protocol interface
Gregory Szorc <gregory.szorc@gmail.com>
parents:
36555
diff
changeset
|
24 |
17192
1ac628cd7113
peer: introduce real peer classes
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
14765
diff
changeset
|
25 class clientpeer(wireproto.wirepeer): |
36944
65615c29e74f
tests: fix test-wireproto.py to work around serverrepo() not having a ui
Augie Fackler <augie@google.com>
parents:
36801
diff
changeset
|
26 def __init__(self, serverrepo, ui): |
14622
bd88561afb4b
wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff
changeset
|
27 self.serverrepo = serverrepo |
36944
65615c29e74f
tests: fix test-wireproto.py to work around serverrepo() not having a ui
Augie Fackler <augie@google.com>
parents:
36801
diff
changeset
|
28 self._ui = ui |
25912
cbbdd085c991
batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents:
25708
diff
changeset
|
29 |
33806
dedab036215d
wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents:
33767
diff
changeset
|
30 @property |
dedab036215d
wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents:
33767
diff
changeset
|
31 def ui(self): |
36944
65615c29e74f
tests: fix test-wireproto.py to work around serverrepo() not having a ui
Augie Fackler <augie@google.com>
parents:
36801
diff
changeset
|
32 return self._ui |
33806
dedab036215d
wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents:
33767
diff
changeset
|
33 |
dedab036215d
wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents:
33767
diff
changeset
|
34 def url(self): |
36555
c28de190e20a
py3: port tests/test-wireproto.py to Python 3
Pulkit Goyal <7895pulkit@gmail.com>
parents:
36074
diff
changeset
|
35 return b'test' |
33806
dedab036215d
wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents:
33767
diff
changeset
|
36 |
dedab036215d
wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents:
33767
diff
changeset
|
37 def local(self): |
dedab036215d
wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents:
33767
diff
changeset
|
38 return None |
dedab036215d
wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents:
33767
diff
changeset
|
39 |
dedab036215d
wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents:
33767
diff
changeset
|
40 def peer(self): |
dedab036215d
wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents:
33767
diff
changeset
|
41 return self |
dedab036215d
wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents:
33767
diff
changeset
|
42 |
dedab036215d
wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents:
33767
diff
changeset
|
43 def canpush(self): |
dedab036215d
wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents:
33767
diff
changeset
|
44 return True |
dedab036215d
wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents:
33767
diff
changeset
|
45 |
dedab036215d
wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents:
33767
diff
changeset
|
46 def close(self): |
dedab036215d
wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents:
33767
diff
changeset
|
47 pass |
dedab036215d
wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents:
33767
diff
changeset
|
48 |
dedab036215d
wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents:
33767
diff
changeset
|
49 def capabilities(self): |
36555
c28de190e20a
py3: port tests/test-wireproto.py to Python 3
Pulkit Goyal <7895pulkit@gmail.com>
parents:
36074
diff
changeset
|
50 return [b'batch'] |
25912
cbbdd085c991
batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents:
25708
diff
changeset
|
51 |
14622
bd88561afb4b
wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff
changeset
|
52 def _call(self, cmd, **args): |
36555
c28de190e20a
py3: port tests/test-wireproto.py to Python 3
Pulkit Goyal <7895pulkit@gmail.com>
parents:
36074
diff
changeset
|
53 args = pycompat.byteskwargs(args) |
36074
2f7290555c96
wireproto: introduce type for raw byte responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
33806
diff
changeset
|
54 res = wireproto.dispatch(self.serverrepo, proto(args), cmd) |
2f7290555c96
wireproto: introduce type for raw byte responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
33806
diff
changeset
|
55 if isinstance(res, wireprototypes.bytesresponse): |
2f7290555c96
wireproto: introduce type for raw byte responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
33806
diff
changeset
|
56 return res.data |
2f7290555c96
wireproto: introduce type for raw byte responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
33806
diff
changeset
|
57 elif isinstance(res, bytes): |
2f7290555c96
wireproto: introduce type for raw byte responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
33806
diff
changeset
|
58 return res |
2f7290555c96
wireproto: introduce type for raw byte responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
33806
diff
changeset
|
59 else: |
2f7290555c96
wireproto: introduce type for raw byte responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
33806
diff
changeset
|
60 raise error.Abort('dummy client does not support response type') |
14622
bd88561afb4b
wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff
changeset
|
61 |
28438
48fd02dac1d4
wireproto: make iterbatcher behave streamily over http(s)
Augie Fackler <augie@google.com>
parents:
27301
diff
changeset
|
62 def _callstream(self, cmd, **args): |
28861
86db5cb55d46
pycompat: switch to util.stringio for py3 compat
timeless <timeless@mozdev.org>
parents:
28860
diff
changeset
|
63 return stringio(self._call(cmd, **args)) |
28438
48fd02dac1d4
wireproto: make iterbatcher behave streamily over http(s)
Augie Fackler <augie@google.com>
parents:
27301
diff
changeset
|
64 |
14622
bd88561afb4b
wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff
changeset
|
65 @wireproto.batchable |
bd88561afb4b
wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff
changeset
|
66 def greet(self, name): |
bd88561afb4b
wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff
changeset
|
67 f = wireproto.future() |
36555
c28de190e20a
py3: port tests/test-wireproto.py to Python 3
Pulkit Goyal <7895pulkit@gmail.com>
parents:
36074
diff
changeset
|
68 yield {b'name': mangle(name)}, f |
14622
bd88561afb4b
wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff
changeset
|
69 yield unmangle(f.value) |
bd88561afb4b
wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff
changeset
|
70 |
14764
a7d5816087a9
classes: fix class style problems found by b071cd58af50
Thomas Arendsen Hein <thomas@intevation.de>
parents:
14622
diff
changeset
|
71 class serverrepo(object): |
14622
bd88561afb4b
wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff
changeset
|
72 def greet(self, name): |
36555
c28de190e20a
py3: port tests/test-wireproto.py to Python 3
Pulkit Goyal <7895pulkit@gmail.com>
parents:
36074
diff
changeset
|
73 return b"Hello, " + name |
14622
bd88561afb4b
wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff
changeset
|
74 |
18278
753acee7d6dd
clfilter: make localpeer use a repo with "unserved" filter
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
17192
diff
changeset
|
75 def filtered(self, name): |
753acee7d6dd
clfilter: make localpeer use a repo with "unserved" filter
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
17192
diff
changeset
|
76 return self |
753acee7d6dd
clfilter: make localpeer use a repo with "unserved" filter
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
17192
diff
changeset
|
77 |
14622
bd88561afb4b
wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff
changeset
|
78 def mangle(s): |
36555
c28de190e20a
py3: port tests/test-wireproto.py to Python 3
Pulkit Goyal <7895pulkit@gmail.com>
parents:
36074
diff
changeset
|
79 return b''.join(pycompat.bytechr(ord(c) + 1) for c in pycompat.bytestr(s)) |
14622
bd88561afb4b
wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff
changeset
|
80 def unmangle(s): |
36555
c28de190e20a
py3: port tests/test-wireproto.py to Python 3
Pulkit Goyal <7895pulkit@gmail.com>
parents:
36074
diff
changeset
|
81 return b''.join(pycompat.bytechr(ord(c) - 1) for c in pycompat.bytestr(s)) |
14622
bd88561afb4b
wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff
changeset
|
82 |
bd88561afb4b
wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff
changeset
|
83 def greet(repo, proto, name): |
bd88561afb4b
wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff
changeset
|
84 return mangle(repo.greet(unmangle(name))) |
bd88561afb4b
wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff
changeset
|
85 |
36555
c28de190e20a
py3: port tests/test-wireproto.py to Python 3
Pulkit Goyal <7895pulkit@gmail.com>
parents:
36074
diff
changeset
|
86 wireproto.commands[b'greet'] = (greet, b'name',) |
14622
bd88561afb4b
wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff
changeset
|
87 |
bd88561afb4b
wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff
changeset
|
88 srv = serverrepo() |
36944
65615c29e74f
tests: fix test-wireproto.py to work around serverrepo() not having a ui
Augie Fackler <augie@google.com>
parents:
36801
diff
changeset
|
89 clt = clientpeer(srv, uimod.ui()) |
14622
bd88561afb4b
wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff
changeset
|
90 |
36555
c28de190e20a
py3: port tests/test-wireproto.py to Python 3
Pulkit Goyal <7895pulkit@gmail.com>
parents:
36074
diff
changeset
|
91 print(clt.greet(b"Foobar")) |
33767
b47fe9733d76
peer: remove non iterating batcher (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
28861
diff
changeset
|
92 b = clt.iterbatch() |
36555
c28de190e20a
py3: port tests/test-wireproto.py to Python 3
Pulkit Goyal <7895pulkit@gmail.com>
parents:
36074
diff
changeset
|
93 list(map(b.greet, (b'Fo, =;:<o', b'Bar'))) |
14622
bd88561afb4b
wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff
changeset
|
94 b.submit() |
33767
b47fe9733d76
peer: remove non iterating batcher (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
28861
diff
changeset
|
95 print([r for r in b.results()]) |