Mercurial > hg
annotate tests/sshprotoext.py @ 37048:fc5e261915b9
wireproto: require POST for all HTTPv2 requests
Wire protocol version 1 transfers argument data via request
headers by default. This has historically caused problems because
servers institute limits on the length of individual HTTP headers
as well as the total size of all request headers. Mercurial servers
can advertise the maximum length of an individual header. But
there's no guarantee any intermediate HTTP agents will accept
headers up to that length.
In the existing wire protocol, server operators typically also
key off the HTTP request method to implement authentication.
For example, GET requests translate to read-only requests and
can be allowed. But read-write commands must use POST and require
authentication. This has typically worked because the only wire
protocol commands that use POST modify the repo (e.g. the
"unbundle" command).
There is an experimental feature to enable clients to transmit
argument data via POST request bodies. This is technically a
better and more robust solution. But we can't enable it by default
because of servers assuming POST means write access.
In version 2 of the wire protocol, the permissions of a request
are encoded in the URL. And with it being a new protocol in a new
URL space, we're not constrained by backwards compatibility
requirements.
This commit adopts the technically superior mechanism of using
HTTP request bodies to send argument data by requiring POST for
all commands. Strictly speaking, it may be possible to send
request bodies on GET requests. But my experience is that not all
HTTP stacks support this. POST pretty much always works. Using POST
for read-only operations does sacrifice some RESTful design
purity. But this API cares about practicality, not about being
in Roy T. Fielding's REST ivory tower.
There's a chance we may relax this restriction in the future. But
for now, I want to see how far we can get with a POST only API.
Differential Revision: https://phab.mercurial-scm.org/D2837
author | Gregory Szorc <gregory.szorc@gmail.com> |
---|---|
date | Tue, 13 Mar 2018 11:57:43 -0700 |
parents | 464bedc0fdb4 |
children | b4d85bc122bd |
rev | line source |
---|---|
35930
83d67257ba90
tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
1 # sshprotoext.py - Extension to test behavior of SSH protocol |
83d67257ba90
tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
2 # |
83d67257ba90
tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
3 # Copyright 2018 Gregory Szorc <gregory.szorc@gmail.com> |
83d67257ba90
tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
4 # |
83d67257ba90
tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
5 # This software may be used and distributed according to the terms of the |
83d67257ba90
tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
6 # GNU General Public License version 2 or any later version. |
83d67257ba90
tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
7 |
83d67257ba90
tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
8 # This extension replaces the SSH server started via `hg serve --stdio`. |
83d67257ba90
tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
9 # The server behaves differently depending on environment variables. |
83d67257ba90
tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
10 |
83d67257ba90
tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
11 from __future__ import absolute_import |
83d67257ba90
tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
12 |
83d67257ba90
tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
13 from mercurial import ( |
83d67257ba90
tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
14 error, |
35938
80a2b8ae42a1
sshpeer: move handshake outside of sshpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
35937
diff
changeset
|
15 extensions, |
35930
83d67257ba90
tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
16 registrar, |
83d67257ba90
tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
17 sshpeer, |
83d67257ba90
tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
18 wireproto, |
83d67257ba90
tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
19 wireprotoserver, |
83d67257ba90
tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
20 ) |
83d67257ba90
tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
21 |
83d67257ba90
tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
22 configtable = {} |
83d67257ba90
tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
23 configitem = registrar.configitem(configtable) |
83d67257ba90
tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
24 |
36211
30cc9f9780df
py3: add b'' to config options in test extension
Gregory Szorc <gregory.szorc@gmail.com>
parents:
36122
diff
changeset
|
25 configitem(b'sshpeer', b'mode', default=None) |
30cc9f9780df
py3: add b'' to config options in test extension
Gregory Szorc <gregory.szorc@gmail.com>
parents:
36122
diff
changeset
|
26 configitem(b'sshpeer', b'handshake-mode', default=None) |
35930
83d67257ba90
tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
27 |
83d67257ba90
tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
28 class bannerserver(wireprotoserver.sshserver): |
83d67257ba90
tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
29 """Server that sends a banner to stdout.""" |
83d67257ba90
tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
30 def serve_forever(self): |
83d67257ba90
tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
31 for i in range(10): |
83d67257ba90
tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
32 self._fout.write(b'banner: line %d\n' % i) |
83d67257ba90
tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
33 |
83d67257ba90
tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
34 super(bannerserver, self).serve_forever() |
83d67257ba90
tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
35 |
83d67257ba90
tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
36 class prehelloserver(wireprotoserver.sshserver): |
83d67257ba90
tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
37 """Tests behavior when connecting to <0.9.1 servers. |
83d67257ba90
tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
38 |
83d67257ba90
tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
39 The ``hello`` wire protocol command was introduced in Mercurial |
83d67257ba90
tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
40 0.9.1. Modern clients send the ``hello`` command when connecting |
83d67257ba90
tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
41 to SSH servers. This mock server tests behavior of the handshake |
83d67257ba90
tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
42 when ``hello`` is not supported. |
83d67257ba90
tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
43 """ |
83d67257ba90
tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
44 def serve_forever(self): |
83d67257ba90
tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
45 l = self._fin.readline() |
83d67257ba90
tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
46 assert l == b'hello\n' |
83d67257ba90
tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
47 # Respond to unknown commands with an empty reply. |
36064
5767664d39a5
wireprotoserver: extract SSH response handling functions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
35976
diff
changeset
|
48 wireprotoserver._sshv1respondbytes(self._fout, b'') |
35930
83d67257ba90
tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
49 l = self._fin.readline() |
83d67257ba90
tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
50 assert l == b'between\n' |
36214
3b3a987bbbaa
wireprotoserver: move SSH server operation to a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents:
36211
diff
changeset
|
51 proto = wireprotoserver.sshv1protocolhandler(self._ui, self._fin, |
3b3a987bbbaa
wireprotoserver: move SSH server operation to a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents:
36211
diff
changeset
|
52 self._fout) |
3b3a987bbbaa
wireprotoserver: move SSH server operation to a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents:
36211
diff
changeset
|
53 rsp = wireproto.dispatch(self._repo, proto, b'between') |
36074
2f7290555c96
wireproto: introduce type for raw byte responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
36065
diff
changeset
|
54 wireprotoserver._sshv1respondbytes(self._fout, rsp.data) |
35930
83d67257ba90
tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
55 |
83d67257ba90
tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
56 super(prehelloserver, self).serve_forever() |
83d67257ba90
tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
57 |
35938
80a2b8ae42a1
sshpeer: move handshake outside of sshpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
35937
diff
changeset
|
58 def performhandshake(orig, ui, stdin, stdout, stderr): |
80a2b8ae42a1
sshpeer: move handshake outside of sshpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
35937
diff
changeset
|
59 """Wrapped version of sshpeer._performhandshake to send extra commands.""" |
80a2b8ae42a1
sshpeer: move handshake outside of sshpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
35937
diff
changeset
|
60 mode = ui.config(b'sshpeer', b'handshake-mode') |
80a2b8ae42a1
sshpeer: move handshake outside of sshpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
35937
diff
changeset
|
61 if mode == b'pre-no-args': |
80a2b8ae42a1
sshpeer: move handshake outside of sshpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
35937
diff
changeset
|
62 ui.debug(b'sending no-args command\n') |
80a2b8ae42a1
sshpeer: move handshake outside of sshpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
35937
diff
changeset
|
63 stdin.write(b'no-args\n') |
80a2b8ae42a1
sshpeer: move handshake outside of sshpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
35937
diff
changeset
|
64 stdin.flush() |
80a2b8ae42a1
sshpeer: move handshake outside of sshpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
35937
diff
changeset
|
65 return orig(ui, stdin, stdout, stderr) |
80a2b8ae42a1
sshpeer: move handshake outside of sshpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
35937
diff
changeset
|
66 elif mode == b'pre-multiple-no-args': |
80a2b8ae42a1
sshpeer: move handshake outside of sshpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
35937
diff
changeset
|
67 ui.debug(b'sending unknown1 command\n') |
80a2b8ae42a1
sshpeer: move handshake outside of sshpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
35937
diff
changeset
|
68 stdin.write(b'unknown1\n') |
80a2b8ae42a1
sshpeer: move handshake outside of sshpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
35937
diff
changeset
|
69 ui.debug(b'sending unknown2 command\n') |
80a2b8ae42a1
sshpeer: move handshake outside of sshpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
35937
diff
changeset
|
70 stdin.write(b'unknown2\n') |
80a2b8ae42a1
sshpeer: move handshake outside of sshpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
35937
diff
changeset
|
71 ui.debug(b'sending unknown3 command\n') |
80a2b8ae42a1
sshpeer: move handshake outside of sshpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
35937
diff
changeset
|
72 stdin.write(b'unknown3\n') |
80a2b8ae42a1
sshpeer: move handshake outside of sshpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
35937
diff
changeset
|
73 stdin.flush() |
80a2b8ae42a1
sshpeer: move handshake outside of sshpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
35937
diff
changeset
|
74 return orig(ui, stdin, stdout, stderr) |
80a2b8ae42a1
sshpeer: move handshake outside of sshpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
35937
diff
changeset
|
75 else: |
80a2b8ae42a1
sshpeer: move handshake outside of sshpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
35937
diff
changeset
|
76 raise error.ProgrammingError(b'unknown HANDSHAKECOMMANDMODE: %s' % |
80a2b8ae42a1
sshpeer: move handshake outside of sshpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
35937
diff
changeset
|
77 mode) |
35930
83d67257ba90
tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
78 |
83d67257ba90
tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
79 def extsetup(ui): |
83d67257ba90
tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
80 # It's easier for tests to define the server behavior via environment |
83d67257ba90
tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
81 # variables than config options. This is because `hg serve --stdio` |
83d67257ba90
tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
82 # has to be invoked with a certain form for security reasons and |
83d67257ba90
tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
83 # `dummyssh` can't just add `--config` flags to the command line. |
83d67257ba90
tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
84 servermode = ui.environ.get(b'SSHSERVERMODE') |
83d67257ba90
tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
85 |
83d67257ba90
tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
86 if servermode == b'banner': |
83d67257ba90
tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
87 wireprotoserver.sshserver = bannerserver |
83d67257ba90
tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
88 elif servermode == b'no-hello': |
83d67257ba90
tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
89 wireprotoserver.sshserver = prehelloserver |
83d67257ba90
tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
90 elif servermode: |
83d67257ba90
tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
91 raise error.ProgrammingError(b'unknown server mode: %s' % servermode) |
83d67257ba90
tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
92 |
83d67257ba90
tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
93 peermode = ui.config(b'sshpeer', b'mode') |
83d67257ba90
tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
94 |
83d67257ba90
tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
95 if peermode == b'extra-handshake-commands': |
35938
80a2b8ae42a1
sshpeer: move handshake outside of sshpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
35937
diff
changeset
|
96 extensions.wrapfunction(sshpeer, '_performhandshake', performhandshake) |
35930
83d67257ba90
tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
97 elif peermode: |
83d67257ba90
tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
98 raise error.ProgrammingError(b'unknown peer mode: %s' % peermode) |