tests/test-wireproto.py
author Gregory Szorc <gregory.szorc@gmail.com>
Wed, 28 Mar 2018 10:40:41 -0700
changeset 37295 45b39c69fae0
parent 36944 65615c29e74f
child 37321 e826fe7a08c7
permissions -rw-r--r--
wireproto: separate commands tables for version 1 and 2 commands We can't easily reuse existing command handlers for version 2 commands because the response types will be different. e.g. many commands return nodes encoded as hex. Our new wire protocol is binary safe, so we'll wish to encode nodes as binary. We /could/ teach each command handler to look at the protocol handler and change behavior based on the version in use. However, this would make logic a bit unwieldy over time and would make it harder to design a unified protocol handler interface. I think it's better to create a clean break between version 1 and version 2 of commands on the server. What I imagine happening is we will have separate @wireprotocommand functions for each protocol generation. Those functions will parse the request, dispatch to a common function to process it, then generate the response in its own, transport-specific manner. This commit establishes a separate table for tracking version 1 commands from version 2 commands. The HTTP server pieces have been updated to use this new table. Most commands are marked as both version 1 and version 2, so there is little practical impact to this change. A side-effect of this change is we now rely on transport registration in wireprototypes.TRANSPORTS and certain properties of the protocol interface. So a test had to be updated to conform. Differential Revision: https://phab.mercurial-scm.org/D2982
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
28675
fcafd84bc9c5 py3: make test-wireproto use print_function
Pulkit Goyal <7895pulkit@gmail.com>
parents: 28438
diff changeset
     1
from __future__ import absolute_import, print_function
27301
5defcb7d6e5f tests: use absolulte_import in test-wireproto.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25912
diff changeset
     2
28860
50d11dd8ac02 py3: use multi-line import in test-wireproto.py
timeless <timeless@mozdev.org>
parents: 28675
diff changeset
     3
from mercurial import (
36074
2f7290555c96 wireproto: introduce type for raw byte responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33806
diff changeset
     4
    error,
36555
c28de190e20a py3: port tests/test-wireproto.py to Python 3
Pulkit Goyal <7895pulkit@gmail.com>
parents: 36074
diff changeset
     5
    pycompat,
36944
65615c29e74f tests: fix test-wireproto.py to work around serverrepo() not having a ui
Augie Fackler <augie@google.com>
parents: 36801
diff changeset
     6
    ui as uimod,
28861
86db5cb55d46 pycompat: switch to util.stringio for py3 compat
timeless <timeless@mozdev.org>
parents: 28860
diff changeset
     7
    util,
28860
50d11dd8ac02 py3: use multi-line import in test-wireproto.py
timeless <timeless@mozdev.org>
parents: 28675
diff changeset
     8
    wireproto,
36074
2f7290555c96 wireproto: introduce type for raw byte responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33806
diff changeset
     9
    wireprototypes,
28860
50d11dd8ac02 py3: use multi-line import in test-wireproto.py
timeless <timeless@mozdev.org>
parents: 28675
diff changeset
    10
)
28861
86db5cb55d46 pycompat: switch to util.stringio for py3 compat
timeless <timeless@mozdev.org>
parents: 28860
diff changeset
    11
stringio = util.stringio
14622
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
    12
14764
a7d5816087a9 classes: fix class style problems found by b071cd58af50
Thomas Arendsen Hein <thomas@intevation.de>
parents: 14622
diff changeset
    13
class proto(object):
14622
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
    14
    def __init__(self, args):
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
    15
        self.args = args
37295
45b39c69fae0 wireproto: separate commands tables for version 1 and 2 commands
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36944
diff changeset
    16
        self.name = 'dummyproto'
45b39c69fae0 wireproto: separate commands tables for version 1 and 2 commands
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36944
diff changeset
    17
14622
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
    18
    def getargs(self, spec):
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
    19
        args = self.args
36555
c28de190e20a py3: port tests/test-wireproto.py to Python 3
Pulkit Goyal <7895pulkit@gmail.com>
parents: 36074
diff changeset
    20
        args.setdefault(b'*', {})
14622
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
    21
        names = spec.split()
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
    22
        return [args[n] for n in names]
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
    23
36801
66de4555cefd wireproto: formalize permissions checking as part of protocol interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36555
diff changeset
    24
    def checkperm(self, perm):
66de4555cefd wireproto: formalize permissions checking as part of protocol interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36555
diff changeset
    25
        pass
66de4555cefd wireproto: formalize permissions checking as part of protocol interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36555
diff changeset
    26
37295
45b39c69fae0 wireproto: separate commands tables for version 1 and 2 commands
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36944
diff changeset
    27
wireprototypes.TRANSPORTS['dummyproto'] = {
45b39c69fae0 wireproto: separate commands tables for version 1 and 2 commands
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36944
diff changeset
    28
    'transport': 'dummy',
45b39c69fae0 wireproto: separate commands tables for version 1 and 2 commands
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36944
diff changeset
    29
    'version': 1,
45b39c69fae0 wireproto: separate commands tables for version 1 and 2 commands
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36944
diff changeset
    30
}
45b39c69fae0 wireproto: separate commands tables for version 1 and 2 commands
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36944
diff changeset
    31
17192
1ac628cd7113 peer: introduce real peer classes
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14765
diff changeset
    32
class clientpeer(wireproto.wirepeer):
36944
65615c29e74f tests: fix test-wireproto.py to work around serverrepo() not having a ui
Augie Fackler <augie@google.com>
parents: 36801
diff changeset
    33
    def __init__(self, serverrepo, ui):
14622
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
    34
        self.serverrepo = serverrepo
36944
65615c29e74f tests: fix test-wireproto.py to work around serverrepo() not having a ui
Augie Fackler <augie@google.com>
parents: 36801
diff changeset
    35
        self._ui = ui
25912
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 25708
diff changeset
    36
33806
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
    37
    @property
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
    38
    def ui(self):
36944
65615c29e74f tests: fix test-wireproto.py to work around serverrepo() not having a ui
Augie Fackler <augie@google.com>
parents: 36801
diff changeset
    39
        return self._ui
33806
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
    40
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
    41
    def url(self):
36555
c28de190e20a py3: port tests/test-wireproto.py to Python 3
Pulkit Goyal <7895pulkit@gmail.com>
parents: 36074
diff changeset
    42
        return b'test'
33806
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
    43
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
    44
    def local(self):
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
    45
        return None
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
    46
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
    47
    def peer(self):
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
    48
        return self
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
    49
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
    50
    def canpush(self):
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
    51
        return True
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
    52
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
    53
    def close(self):
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
    54
        pass
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
    55
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
    56
    def capabilities(self):
36555
c28de190e20a py3: port tests/test-wireproto.py to Python 3
Pulkit Goyal <7895pulkit@gmail.com>
parents: 36074
diff changeset
    57
        return [b'batch']
25912
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 25708
diff changeset
    58
14622
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
    59
    def _call(self, cmd, **args):
36555
c28de190e20a py3: port tests/test-wireproto.py to Python 3
Pulkit Goyal <7895pulkit@gmail.com>
parents: 36074
diff changeset
    60
        args = pycompat.byteskwargs(args)
36074
2f7290555c96 wireproto: introduce type for raw byte responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33806
diff changeset
    61
        res = wireproto.dispatch(self.serverrepo, proto(args), cmd)
2f7290555c96 wireproto: introduce type for raw byte responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33806
diff changeset
    62
        if isinstance(res, wireprototypes.bytesresponse):
2f7290555c96 wireproto: introduce type for raw byte responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33806
diff changeset
    63
            return res.data
2f7290555c96 wireproto: introduce type for raw byte responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33806
diff changeset
    64
        elif isinstance(res, bytes):
2f7290555c96 wireproto: introduce type for raw byte responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33806
diff changeset
    65
            return res
2f7290555c96 wireproto: introduce type for raw byte responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33806
diff changeset
    66
        else:
2f7290555c96 wireproto: introduce type for raw byte responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33806
diff changeset
    67
            raise error.Abort('dummy client does not support response type')
14622
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
    68
28438
48fd02dac1d4 wireproto: make iterbatcher behave streamily over http(s)
Augie Fackler <augie@google.com>
parents: 27301
diff changeset
    69
    def _callstream(self, cmd, **args):
28861
86db5cb55d46 pycompat: switch to util.stringio for py3 compat
timeless <timeless@mozdev.org>
parents: 28860
diff changeset
    70
        return stringio(self._call(cmd, **args))
28438
48fd02dac1d4 wireproto: make iterbatcher behave streamily over http(s)
Augie Fackler <augie@google.com>
parents: 27301
diff changeset
    71
14622
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
    72
    @wireproto.batchable
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
    73
    def greet(self, name):
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
    74
        f = wireproto.future()
36555
c28de190e20a py3: port tests/test-wireproto.py to Python 3
Pulkit Goyal <7895pulkit@gmail.com>
parents: 36074
diff changeset
    75
        yield {b'name': mangle(name)}, f
14622
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
    76
        yield unmangle(f.value)
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
    77
14764
a7d5816087a9 classes: fix class style problems found by b071cd58af50
Thomas Arendsen Hein <thomas@intevation.de>
parents: 14622
diff changeset
    78
class serverrepo(object):
14622
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
    79
    def greet(self, name):
36555
c28de190e20a py3: port tests/test-wireproto.py to Python 3
Pulkit Goyal <7895pulkit@gmail.com>
parents: 36074
diff changeset
    80
        return b"Hello, " + name
14622
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
    81
18278
753acee7d6dd clfilter: make localpeer use a repo with "unserved" filter
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 17192
diff changeset
    82
    def filtered(self, name):
753acee7d6dd clfilter: make localpeer use a repo with "unserved" filter
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 17192
diff changeset
    83
        return self
753acee7d6dd clfilter: make localpeer use a repo with "unserved" filter
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 17192
diff changeset
    84
14622
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
    85
def mangle(s):
36555
c28de190e20a py3: port tests/test-wireproto.py to Python 3
Pulkit Goyal <7895pulkit@gmail.com>
parents: 36074
diff changeset
    86
    return b''.join(pycompat.bytechr(ord(c) + 1) for c in pycompat.bytestr(s))
14622
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
    87
def unmangle(s):
36555
c28de190e20a py3: port tests/test-wireproto.py to Python 3
Pulkit Goyal <7895pulkit@gmail.com>
parents: 36074
diff changeset
    88
    return b''.join(pycompat.bytechr(ord(c) - 1) for c in pycompat.bytestr(s))
14622
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
    89
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
    90
def greet(repo, proto, name):
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
    91
    return mangle(repo.greet(unmangle(name)))
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
    92
36555
c28de190e20a py3: port tests/test-wireproto.py to Python 3
Pulkit Goyal <7895pulkit@gmail.com>
parents: 36074
diff changeset
    93
wireproto.commands[b'greet'] = (greet, b'name',)
14622
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
    94
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
    95
srv = serverrepo()
36944
65615c29e74f tests: fix test-wireproto.py to work around serverrepo() not having a ui
Augie Fackler <augie@google.com>
parents: 36801
diff changeset
    96
clt = clientpeer(srv, uimod.ui())
14622
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
    97
36555
c28de190e20a py3: port tests/test-wireproto.py to Python 3
Pulkit Goyal <7895pulkit@gmail.com>
parents: 36074
diff changeset
    98
print(clt.greet(b"Foobar"))
33767
b47fe9733d76 peer: remove non iterating batcher (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28861
diff changeset
    99
b = clt.iterbatch()
36555
c28de190e20a py3: port tests/test-wireproto.py to Python 3
Pulkit Goyal <7895pulkit@gmail.com>
parents: 36074
diff changeset
   100
list(map(b.greet, (b'Fo, =;:<o', b'Bar')))
14622
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
   101
b.submit()
33767
b47fe9733d76 peer: remove non iterating batcher (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28861
diff changeset
   102
print([r for r in b.results()])