annotate tests/sshprotoext.py @ 49269:395f28064826

worker: avoid potential partial write of pickled data Previously, the code wrote the pickled data using os.write(). However, os.write() can write less bytes than passed to it. To trigger the problem, the pickled data had to be larger than 2147479552 bytes on my system. Instead, open a file object and pass it to pickle.dump(). This also has the advantage that it doesn’t buffer the whole pickled data in memory. Note that the opened file must be buffered because pickle doesn’t support unbuffered streams because unbuffered streams’ write() method might write less bytes than passed to it (like os.write()) but pickle.dump() relies on that all bytes are written (see https://github.com/python/cpython/issues/93050). The side effect of using a file object and a with statement is that wfd is explicitly closed now while it seems like before it was implicitly closed by process exit.
author Manuel Jacob <me@manueljacob.de>
date Sun, 22 May 2022 03:50:34 +0200
parents 6000f5b25c9b
children 13c004b54cbe
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
35930
83d67257ba90 tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
1 # sshprotoext.py - Extension to test behavior of SSH protocol
83d67257ba90 tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
2 #
83d67257ba90 tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
3 # Copyright 2018 Gregory Szorc <gregory.szorc@gmail.com>
83d67257ba90 tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
4 #
83d67257ba90 tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
5 # This software may be used and distributed according to the terms of the
83d67257ba90 tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
6 # GNU General Public License version 2 or any later version.
83d67257ba90 tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
7
83d67257ba90 tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
8 # This extension replaces the SSH server started via `hg serve --stdio`.
83d67257ba90 tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
9 # The server behaves differently depending on environment variables.
83d67257ba90 tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
10
83d67257ba90 tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
11
83d67257ba90 tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
12 from mercurial import (
83d67257ba90 tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
13 error,
35938
80a2b8ae42a1 sshpeer: move handshake outside of sshpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35937
diff changeset
14 extensions,
35930
83d67257ba90 tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
15 registrar,
83d67257ba90 tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
16 sshpeer,
83d67257ba90 tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
17 wireprotoserver,
37785
b4d85bc122bd wireproto: rename wireproto to wireprotov1server (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36215
diff changeset
18 wireprotov1server,
35930
83d67257ba90 tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
19 )
83d67257ba90 tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
20
83d67257ba90 tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
21 configtable = {}
83d67257ba90 tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
22 configitem = registrar.configitem(configtable)
83d67257ba90 tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
23
36211
30cc9f9780df py3: add b'' to config options in test extension
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36122
diff changeset
24 configitem(b'sshpeer', b'mode', default=None)
30cc9f9780df py3: add b'' to config options in test extension
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36122
diff changeset
25 configitem(b'sshpeer', b'handshake-mode', default=None)
35930
83d67257ba90 tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
26
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 37785
diff changeset
27
35930
83d67257ba90 tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
28 class bannerserver(wireprotoserver.sshserver):
83d67257ba90 tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
29 """Server that sends a banner to stdout."""
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 37785
diff changeset
30
35930
83d67257ba90 tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
31 def serve_forever(self):
83d67257ba90 tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
32 for i in range(10):
83d67257ba90 tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
33 self._fout.write(b'banner: line %d\n' % i)
83d67257ba90 tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
34
83d67257ba90 tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
35 super(bannerserver, self).serve_forever()
83d67257ba90 tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
36
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 37785
diff changeset
37
35930
83d67257ba90 tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
38 class prehelloserver(wireprotoserver.sshserver):
83d67257ba90 tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
39 """Tests behavior when connecting to <0.9.1 servers.
83d67257ba90 tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
40
83d67257ba90 tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
41 The ``hello`` wire protocol command was introduced in Mercurial
83d67257ba90 tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
42 0.9.1. Modern clients send the ``hello`` command when connecting
83d67257ba90 tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
43 to SSH servers. This mock server tests behavior of the handshake
83d67257ba90 tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
44 when ``hello`` is not supported.
83d67257ba90 tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
45 """
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 37785
diff changeset
46
35930
83d67257ba90 tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
47 def serve_forever(self):
83d67257ba90 tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
48 l = self._fin.readline()
83d67257ba90 tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
49 assert l == b'hello\n'
83d67257ba90 tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
50 # Respond to unknown commands with an empty reply.
36064
5767664d39a5 wireprotoserver: extract SSH response handling functions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35976
diff changeset
51 wireprotoserver._sshv1respondbytes(self._fout, b'')
35930
83d67257ba90 tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
52 l = self._fin.readline()
83d67257ba90 tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
53 assert l == b'between\n'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 37785
diff changeset
54 proto = wireprotoserver.sshv1protocolhandler(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 37785
diff changeset
55 self._ui, self._fin, self._fout
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 37785
diff changeset
56 )
37785
b4d85bc122bd wireproto: rename wireproto to wireprotov1server (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36215
diff changeset
57 rsp = wireprotov1server.dispatch(self._repo, proto, b'between')
36074
2f7290555c96 wireproto: introduce type for raw byte responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36065
diff changeset
58 wireprotoserver._sshv1respondbytes(self._fout, rsp.data)
35930
83d67257ba90 tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
59
83d67257ba90 tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
60 super(prehelloserver, self).serve_forever()
83d67257ba90 tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
61
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 37785
diff changeset
62
35938
80a2b8ae42a1 sshpeer: move handshake outside of sshpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35937
diff changeset
63 def performhandshake(orig, ui, stdin, stdout, stderr):
80a2b8ae42a1 sshpeer: move handshake outside of sshpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35937
diff changeset
64 """Wrapped version of sshpeer._performhandshake to send extra commands."""
80a2b8ae42a1 sshpeer: move handshake outside of sshpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35937
diff changeset
65 mode = ui.config(b'sshpeer', b'handshake-mode')
80a2b8ae42a1 sshpeer: move handshake outside of sshpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35937
diff changeset
66 if mode == b'pre-no-args':
80a2b8ae42a1 sshpeer: move handshake outside of sshpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35937
diff changeset
67 ui.debug(b'sending no-args command\n')
80a2b8ae42a1 sshpeer: move handshake outside of sshpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35937
diff changeset
68 stdin.write(b'no-args\n')
80a2b8ae42a1 sshpeer: move handshake outside of sshpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35937
diff changeset
69 stdin.flush()
80a2b8ae42a1 sshpeer: move handshake outside of sshpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35937
diff changeset
70 return orig(ui, stdin, stdout, stderr)
80a2b8ae42a1 sshpeer: move handshake outside of sshpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35937
diff changeset
71 elif mode == b'pre-multiple-no-args':
80a2b8ae42a1 sshpeer: move handshake outside of sshpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35937
diff changeset
72 ui.debug(b'sending unknown1 command\n')
80a2b8ae42a1 sshpeer: move handshake outside of sshpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35937
diff changeset
73 stdin.write(b'unknown1\n')
80a2b8ae42a1 sshpeer: move handshake outside of sshpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35937
diff changeset
74 ui.debug(b'sending unknown2 command\n')
80a2b8ae42a1 sshpeer: move handshake outside of sshpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35937
diff changeset
75 stdin.write(b'unknown2\n')
80a2b8ae42a1 sshpeer: move handshake outside of sshpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35937
diff changeset
76 ui.debug(b'sending unknown3 command\n')
80a2b8ae42a1 sshpeer: move handshake outside of sshpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35937
diff changeset
77 stdin.write(b'unknown3\n')
80a2b8ae42a1 sshpeer: move handshake outside of sshpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35937
diff changeset
78 stdin.flush()
80a2b8ae42a1 sshpeer: move handshake outside of sshpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35937
diff changeset
79 return orig(ui, stdin, stdout, stderr)
80a2b8ae42a1 sshpeer: move handshake outside of sshpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35937
diff changeset
80 else:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 37785
diff changeset
81 raise error.ProgrammingError(b'unknown HANDSHAKECOMMANDMODE: %s' % mode)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 37785
diff changeset
82
35930
83d67257ba90 tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
83
83d67257ba90 tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
84 def extsetup(ui):
83d67257ba90 tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
85 # It's easier for tests to define the server behavior via environment
83d67257ba90 tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
86 # variables than config options. This is because `hg serve --stdio`
83d67257ba90 tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
87 # has to be invoked with a certain form for security reasons and
83d67257ba90 tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
88 # `dummyssh` can't just add `--config` flags to the command line.
83d67257ba90 tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
89 servermode = ui.environ.get(b'SSHSERVERMODE')
83d67257ba90 tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
90
83d67257ba90 tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
91 if servermode == b'banner':
83d67257ba90 tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
92 wireprotoserver.sshserver = bannerserver
83d67257ba90 tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
93 elif servermode == b'no-hello':
83d67257ba90 tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
94 wireprotoserver.sshserver = prehelloserver
83d67257ba90 tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
95 elif servermode:
83d67257ba90 tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
96 raise error.ProgrammingError(b'unknown server mode: %s' % servermode)
83d67257ba90 tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
97
83d67257ba90 tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
98 peermode = ui.config(b'sshpeer', b'mode')
83d67257ba90 tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
99
83d67257ba90 tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
100 if peermode == b'extra-handshake-commands':
35938
80a2b8ae42a1 sshpeer: move handshake outside of sshpeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35937
diff changeset
101 extensions.wrapfunction(sshpeer, '_performhandshake', performhandshake)
35930
83d67257ba90 tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
102 elif peermode:
83d67257ba90 tests: add low-level SSH protocol tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
103 raise error.ProgrammingError(b'unknown peer mode: %s' % peermode)