annotate mercurial/wireprotov1peer.py @ 44192:d6d4170882cd

profiling: flush stdout before writing profile to stderr On py3, stdout and stderr appear to be buffered and this causes my command's output to be intermixed with the profiling output. Differential Revision: https://phab.mercurial-scm.org/D8024
author Kyle Lippincott <spectral@google.com>
date Mon, 27 Jan 2020 18:16:05 -0800
parents a61287a95dc3
children 89a2afe31e82
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
37614
a81d02ea65db wireproto: move version 1 peer functionality to standalone module (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37613
diff changeset
1 # wireprotov1peer.py - Client-side functionality for wire protocol version 1.
11581
4530b3307fb9 protocol: introduce wireproto.py
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
2 #
4530b3307fb9 protocol: introduce wireproto.py
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
3 # Copyright 2005-2010 Matt Mackall <mpm@selenic.com>
4530b3307fb9 protocol: introduce wireproto.py
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
4 #
4530b3307fb9 protocol: introduce wireproto.py
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
5 # This software may be used and distributed according to the terms of the
4530b3307fb9 protocol: introduce wireproto.py
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
6 # GNU General Public License version 2 or any later version.
4530b3307fb9 protocol: introduce wireproto.py
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
7
25993
0851678be71b wireproto: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25913
diff changeset
8 from __future__ import absolute_import
0851678be71b wireproto: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25913
diff changeset
9
37630
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
10 import sys
37631
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
11 import weakref
11581
4530b3307fb9 protocol: introduce wireproto.py
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
12
25993
0851678be71b wireproto: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25913
diff changeset
13 from .i18n import _
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
14 from .node import bin
43089
c59eb1560c44 py3: manually import getattr where it is needed
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43087
diff changeset
15 from .pycompat import (
c59eb1560c44 py3: manually import getattr where it is needed
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43087
diff changeset
16 getattr,
c59eb1560c44 py3: manually import getattr where it is needed
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43087
diff changeset
17 setattr,
c59eb1560c44 py3: manually import getattr where it is needed
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43087
diff changeset
18 )
25993
0851678be71b wireproto: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25913
diff changeset
19 from . import (
0851678be71b wireproto: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25913
diff changeset
20 bundle2,
0851678be71b wireproto: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25913
diff changeset
21 changegroup as changegroupmod,
0851678be71b wireproto: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25913
diff changeset
22 encoding,
0851678be71b wireproto: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25913
diff changeset
23 error,
0851678be71b wireproto: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25913
diff changeset
24 pushkey as pushkeymod,
30924
48dea083f66d py3: convert the mode argument of os.fdopen to unicodes (1 of 2)
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30914
diff changeset
25 pycompat,
25993
0851678be71b wireproto: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25913
diff changeset
26 util,
36073
cd6ab329c5c7 wireprototypes: move wire protocol response types to new module
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36071
diff changeset
27 wireprototypes,
25993
0851678be71b wireproto: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25913
diff changeset
28 )
42813
268662aac075 interfaces: create a new folder for interfaces and move repository.py in it
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41055
diff changeset
29 from .interfaces import (
268662aac075 interfaces: create a new folder for interfaces and move repository.py in it
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41055
diff changeset
30 repository,
42814
2c4f656c8e9f interfaceutil: move to interfaces/
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 42813
diff changeset
31 util as interfaceutil,
37810
856f381ad74b interfaceutil: module to stub out zope.interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37669
diff changeset
32 )
44060
a61287a95dc3 core: migrate uses of hashlib.sha1 to hashutil.sha1
Augie Fackler <augie@google.com>
parents: 43506
diff changeset
33 from .utils import hashutil
20903
8d477543882b wireproto: introduce an abstractserverproto class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20902
diff changeset
34
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28666
diff changeset
35 urlreq = util.urlreq
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28666
diff changeset
36
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
37
37615
f3dc8239e3a9 peer: scatter module to the wind (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
38 def batchable(f):
f3dc8239e3a9 peer: scatter module to the wind (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
39 '''annotation for batchable methods
f3dc8239e3a9 peer: scatter module to the wind (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
40
f3dc8239e3a9 peer: scatter module to the wind (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
41 Such methods must implement a coroutine as follows:
f3dc8239e3a9 peer: scatter module to the wind (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
42
f3dc8239e3a9 peer: scatter module to the wind (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
43 @batchable
f3dc8239e3a9 peer: scatter module to the wind (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
44 def sample(self, one, two=None):
f3dc8239e3a9 peer: scatter module to the wind (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
45 # Build list of encoded arguments suitable for your wire protocol:
f3dc8239e3a9 peer: scatter module to the wind (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
46 encargs = [('one', encode(one),), ('two', encode(two),)]
f3dc8239e3a9 peer: scatter module to the wind (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
47 # Create future for injection of encoded result:
f3dc8239e3a9 peer: scatter module to the wind (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
48 encresref = future()
f3dc8239e3a9 peer: scatter module to the wind (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
49 # Return encoded arguments and future:
f3dc8239e3a9 peer: scatter module to the wind (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
50 yield encargs, encresref
f3dc8239e3a9 peer: scatter module to the wind (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
51 # Assuming the future to be filled with the result from the batched
f3dc8239e3a9 peer: scatter module to the wind (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
52 # request now. Decode it:
f3dc8239e3a9 peer: scatter module to the wind (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
53 yield decode(encresref.value)
f3dc8239e3a9 peer: scatter module to the wind (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
54
f3dc8239e3a9 peer: scatter module to the wind (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
55 The decorator returns a function which wraps this coroutine as a plain
f3dc8239e3a9 peer: scatter module to the wind (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
56 method, but adds the original method as an attribute called "batchable",
f3dc8239e3a9 peer: scatter module to the wind (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
57 which is used by remotebatch to split the call into separate encoding and
f3dc8239e3a9 peer: scatter module to the wind (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
58 decoding phases.
f3dc8239e3a9 peer: scatter module to the wind (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
59 '''
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
60
37615
f3dc8239e3a9 peer: scatter module to the wind (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
61 def plain(*args, **opts):
f3dc8239e3a9 peer: scatter module to the wind (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
62 batchable = f(*args, **opts)
f3dc8239e3a9 peer: scatter module to the wind (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
63 encargsorres, encresref = next(batchable)
f3dc8239e3a9 peer: scatter module to the wind (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
64 if not encresref:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
65 return encargsorres # a local result in this case
37615
f3dc8239e3a9 peer: scatter module to the wind (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
66 self = args[0]
f3dc8239e3a9 peer: scatter module to the wind (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
67 cmd = pycompat.bytesurl(f.__name__) # ensure cmd is ascii bytestr
f3dc8239e3a9 peer: scatter module to the wind (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
68 encresref.set(self._submitone(cmd, encargsorres))
f3dc8239e3a9 peer: scatter module to the wind (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
69 return next(batchable)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
70
37615
f3dc8239e3a9 peer: scatter module to the wind (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
71 setattr(plain, 'batchable', f)
39588
f15a587d2dfc wireprotov1peer: forward __name__ of wrapped method in batchable decorator
Augie Fackler <raf@durin42.com>
parents: 38783
diff changeset
72 setattr(plain, '__name__', f.__name__)
37615
f3dc8239e3a9 peer: scatter module to the wind (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
73 return plain
f3dc8239e3a9 peer: scatter module to the wind (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
74
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
75
37615
f3dc8239e3a9 peer: scatter module to the wind (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
76 class future(object):
f3dc8239e3a9 peer: scatter module to the wind (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
77 '''placeholder for a value to be set later'''
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
78
37615
f3dc8239e3a9 peer: scatter module to the wind (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
79 def set(self, value):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
80 if util.safehasattr(self, b'value'):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
81 raise error.RepoError(b"future is already set")
37615
f3dc8239e3a9 peer: scatter module to the wind (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
82 self.value = value
f3dc8239e3a9 peer: scatter module to the wind (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
83
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
84
29733
bb04f96df51c wireproto: consolidate code for obtaining "cmds" argument value
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29706
diff changeset
85 def encodebatchcmds(req):
bb04f96df51c wireproto: consolidate code for obtaining "cmds" argument value
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29706
diff changeset
86 """Return a ``cmds`` argument value for the ``batch`` command."""
37612
5e71dea79aae wireproto: move value encoding functions to wireprototypes (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37546
diff changeset
87 escapearg = wireprototypes.escapebatcharg
5e71dea79aae wireproto: move value encoding functions to wireprototypes (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37546
diff changeset
88
29733
bb04f96df51c wireproto: consolidate code for obtaining "cmds" argument value
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29706
diff changeset
89 cmds = []
bb04f96df51c wireproto: consolidate code for obtaining "cmds" argument value
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29706
diff changeset
90 for op, argsdict in req:
29734
62e2e048d068 wireproto: unescape argument names in batch command (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29733
diff changeset
91 # Old servers didn't properly unescape argument names. So prevent
62e2e048d068 wireproto: unescape argument names in batch command (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29733
diff changeset
92 # the sending of argument names that may not be decoded properly by
62e2e048d068 wireproto: unescape argument names in batch command (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29733
diff changeset
93 # servers.
62e2e048d068 wireproto: unescape argument names in batch command (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29733
diff changeset
94 assert all(escapearg(k) == k for k in argsdict)
62e2e048d068 wireproto: unescape argument names in batch command (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29733
diff changeset
95
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
96 args = b','.join(
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
97 b'%s=%s' % (escapearg(k), escapearg(v))
43106
d783f945a701 py3: finish porting iteritems() to pycompat and remove source transformer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43089
diff changeset
98 for k, v in pycompat.iteritems(argsdict)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
99 )
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
100 cmds.append(b'%s %s' % (op, args))
29733
bb04f96df51c wireproto: consolidate code for obtaining "cmds" argument value
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29706
diff changeset
101
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
102 return b';'.join(cmds)
29733
bb04f96df51c wireproto: consolidate code for obtaining "cmds" argument value
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29706
diff changeset
103
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
104
37631
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
105 class unsentfuture(pycompat.futures.Future):
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
106 """A Future variation to represent an unsent command.
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
107
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
108 Because we buffer commands and don't submit them immediately, calling
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
109 ``result()`` on an unsent future could deadlock. Futures for buffered
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
110 commands are represented by this type, which wraps ``result()`` to
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
111 call ``sendcommands()``.
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
112 """
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
113
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
114 def result(self, timeout=None):
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
115 if self.done():
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
116 return pycompat.futures.Future.result(self, timeout)
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
117
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
118 self._peerexecutor.sendcommands()
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
119
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
120 # This looks like it will infinitely recurse. However,
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
121 # sendcommands() should modify __class__. This call serves as a check
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
122 # on that.
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
123 return self.result(timeout)
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
124
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
125
37810
856f381ad74b interfaceutil: module to stub out zope.interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37669
diff changeset
126 @interfaceutil.implementer(repository.ipeercommandexecutor)
37630
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
127 class peerexecutor(object):
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
128 def __init__(self, peer):
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
129 self._peer = peer
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
130 self._sent = False
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
131 self._closed = False
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
132 self._calls = []
37631
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
133 self._futures = weakref.WeakSet()
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
134 self._responseexecutor = None
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
135 self._responsef = None
37630
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
136
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
137 def __enter__(self):
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
138 return self
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
139
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
140 def __exit__(self, exctype, excvalee, exctb):
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
141 self.close()
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
142
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
143 def callcommand(self, command, args):
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
144 if self._sent:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
145 raise error.ProgrammingError(
43117
8ff1ecfadcd1 cleanup: join string literals that are already on one line
Martin von Zweigbergk <martinvonz@google.com>
parents: 43106
diff changeset
146 b'callcommand() cannot be used after commands are sent'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
147 )
37630
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
148
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
149 if self._closed:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
150 raise error.ProgrammingError(
43117
8ff1ecfadcd1 cleanup: join string literals that are already on one line
Martin von Zweigbergk <martinvonz@google.com>
parents: 43106
diff changeset
151 b'callcommand() cannot be used after close()'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
152 )
37630
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
153
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
154 # Commands are dispatched through methods on the peer.
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
155 fn = getattr(self._peer, pycompat.sysstr(command), None)
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
156
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
157 if not fn:
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
158 raise error.ProgrammingError(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
159 b'cannot call command %s: method of same name not available '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
160 b'on peer' % command
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
161 )
37630
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
162
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
163 # Commands are either batchable or they aren't. If a command
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
164 # isn't batchable, we send it immediately because the executor
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
165 # can no longer accept new commands after a non-batchable command.
37631
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
166 # If a command is batchable, we queue it for later. But we have
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
167 # to account for the case of a non-batchable command arriving after
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
168 # a batchable one and refuse to service it.
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
169
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
170 def addcall():
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
171 f = pycompat.futures.Future()
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
172 self._futures.add(f)
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
173 self._calls.append((command, args, fn, f))
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
174 return f
37630
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
175
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
176 if getattr(fn, 'batchable', False):
37631
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
177 f = addcall()
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
178
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
179 # But since we don't issue it immediately, we wrap its result()
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
180 # to trigger sending so we avoid deadlocks.
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
181 f.__class__ = unsentfuture
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
182 f._peerexecutor = self
37630
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
183 else:
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
184 if self._calls:
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
185 raise error.ProgrammingError(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
186 b'%s is not batchable and cannot be called on a command '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
187 b'executor along with other commands' % command
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
188 )
37630
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
189
37631
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
190 f = addcall()
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
191
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
192 # Non-batchable commands can never coexist with another command
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
193 # in this executor. So send the command immediately.
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
194 self.sendcommands()
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
195
37630
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
196 return f
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
197
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
198 def sendcommands(self):
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
199 if self._sent:
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
200 return
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
201
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
202 if not self._calls:
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
203 return
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
204
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
205 self._sent = True
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
206
37631
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
207 # Unhack any future types so caller seens a clean type and to break
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
208 # cycle between us and futures.
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
209 for f in self._futures:
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
210 if isinstance(f, unsentfuture):
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
211 f.__class__ = pycompat.futures.Future
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
212 f._peerexecutor = None
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
213
37630
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
214 calls = self._calls
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
215 # Mainly to destroy references to futures.
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
216 self._calls = None
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
217
37631
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
218 # Simple case of a single command. We call it synchronously.
37630
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
219 if len(calls) == 1:
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
220 command, args, fn, f = calls[0]
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
221
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
222 # Future was cancelled. Ignore it.
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
223 if not f.set_running_or_notify_cancel():
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
224 return
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
225
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
226 try:
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
227 result = fn(**pycompat.strkwargs(args))
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
228 except Exception:
37669
1cb54e6193a6 py3: paper over differences in future exception handling
Augie Fackler <augie@google.com>
parents: 37650
diff changeset
229 pycompat.future_set_exception_info(f, sys.exc_info()[1:])
37630
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
230 else:
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
231 f.set_result(result)
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
232
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
233 return
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
234
37631
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
235 # Batch commands are a bit harder. First, we have to deal with the
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
236 # @batchable coroutine. That's a bit annoying. Furthermore, we also
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
237 # need to preserve streaming. i.e. it should be possible for the
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
238 # futures to resolve as data is coming in off the wire without having
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
239 # to wait for the final byte of the final response. We do this by
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
240 # spinning up a thread to read the responses.
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
241
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
242 requests = []
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
243 states = []
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
244
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
245 for command, args, fn, f in calls:
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
246 # Future was cancelled. Ignore it.
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
247 if not f.set_running_or_notify_cancel():
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
248 continue
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
249
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
250 try:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
251 batchable = fn.batchable(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
252 fn.__self__, **pycompat.strkwargs(args)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
253 )
37631
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
254 except Exception:
37669
1cb54e6193a6 py3: paper over differences in future exception handling
Augie Fackler <augie@google.com>
parents: 37650
diff changeset
255 pycompat.future_set_exception_info(f, sys.exc_info()[1:])
37631
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
256 return
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
257
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
258 # Encoded arguments and future holding remote result.
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
259 try:
41055
55e8da487b8a wireproto: in batch queries, support queries with immediate responses
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 39588
diff changeset
260 encargsorres, fremote = next(batchable)
37631
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
261 except Exception:
37669
1cb54e6193a6 py3: paper over differences in future exception handling
Augie Fackler <augie@google.com>
parents: 37650
diff changeset
262 pycompat.future_set_exception_info(f, sys.exc_info()[1:])
37631
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
263 return
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
264
41055
55e8da487b8a wireproto: in batch queries, support queries with immediate responses
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 39588
diff changeset
265 if not fremote:
55e8da487b8a wireproto: in batch queries, support queries with immediate responses
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 39588
diff changeset
266 f.set_result(encargsorres)
55e8da487b8a wireproto: in batch queries, support queries with immediate responses
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 39588
diff changeset
267 else:
55e8da487b8a wireproto: in batch queries, support queries with immediate responses
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 39588
diff changeset
268 requests.append((command, encargsorres))
55e8da487b8a wireproto: in batch queries, support queries with immediate responses
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 39588
diff changeset
269 states.append((command, f, batchable, fremote))
37631
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
270
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
271 if not requests:
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
272 return
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
273
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
274 # This will emit responses in order they were executed.
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
275 wireresults = self._peer._submitbatch(requests)
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
276
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
277 # The use of a thread pool executor here is a bit weird for something
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
278 # that only spins up a single thread. However, thread management is
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
279 # hard and it is easy to encounter race conditions, deadlocks, etc.
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
280 # concurrent.futures already solves these problems and its thread pool
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
281 # executor has minimal overhead. So we use it.
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
282 self._responseexecutor = pycompat.futures.ThreadPoolExecutor(1)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
283 self._responsef = self._responseexecutor.submit(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
284 self._readbatchresponse, states, wireresults
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
285 )
37630
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
286
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
287 def close(self):
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
288 self.sendcommands()
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
289
37631
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
290 if self._closed:
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
291 return
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
292
37630
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
293 self._closed = True
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
294
37631
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
295 if not self._responsef:
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
296 return
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
297
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
298 # We need to wait on our in-flight response and then shut down the
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
299 # executor once we have a result.
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
300 try:
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
301 self._responsef.result()
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
302 finally:
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
303 self._responseexecutor.shutdown(wait=True)
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
304 self._responsef = None
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
305 self._responseexecutor = None
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
306
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
307 # If any of our futures are still in progress, mark them as
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
308 # errored. Otherwise a result() could wait indefinitely.
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
309 for f in self._futures:
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
310 if not f.done():
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
311 f.set_exception(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
312 error.ResponseError(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
313 _(b'unfulfilled batch command response')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
314 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
315 )
37631
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
316
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
317 self._futures = None
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
318
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
319 def _readbatchresponse(self, states, wireresults):
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
320 # Executes in a thread to read data off the wire.
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
321
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
322 for command, f, batchable, fremote in states:
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
323 # Grab raw result off the wire and teach the internal future
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
324 # about it.
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
325 remoteresult = next(wireresults)
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
326 fremote.set(remoteresult)
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
327
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
328 # And ask the coroutine to decode that value.
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
329 try:
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
330 result = next(batchable)
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
331 except Exception:
37669
1cb54e6193a6 py3: paper over differences in future exception handling
Augie Fackler <augie@google.com>
parents: 37650
diff changeset
332 pycompat.future_set_exception_info(f, sys.exc_info()[1:])
37631
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
333 else:
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
334 f.set_result(result)
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
335
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
336
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
337 @interfaceutil.implementer(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
338 repository.ipeercommands, repository.ipeerlegacycommands
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
339 )
37635
cc8c06835097 wireproto: convert legacy commands to command executor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37633
diff changeset
340 class wirepeer(repository.peer):
27243
3abee2ba27af wireproto: add docstring for wirepeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26911
diff changeset
341 """Client-side interface for communicating with a peer repository.
14622
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14621
diff changeset
342
27243
3abee2ba27af wireproto: add docstring for wirepeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26911
diff changeset
343 Methods commonly call wire protocol commands of the same name.
3abee2ba27af wireproto: add docstring for wirepeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26911
diff changeset
344
3abee2ba27af wireproto: add docstring for wirepeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26911
diff changeset
345 See also httppeer.py and sshpeer.py for protocol-specific
3abee2ba27af wireproto: add docstring for wirepeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26911
diff changeset
346 implementations of this interface.
3abee2ba27af wireproto: add docstring for wirepeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26911
diff changeset
347 """
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
348
37630
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
349 def commandexecutor(self):
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
350 return peerexecutor(self)
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
351
37320
39f7d4ee8bcd repository: port peer interfaces to zope.interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37295
diff changeset
352 # Begin of ipeercommands interface.
14622
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14621
diff changeset
353
37649
a168799687e5 wireproto: properly call clonebundles command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37646
diff changeset
354 def clonebundles(self):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
355 self.requirecap(b'clonebundles', _(b'clone bundles'))
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
356 return self._call(b'clonebundles')
37649
a168799687e5 wireproto: properly call clonebundles command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37646
diff changeset
357
14623
e7c9fdbbb902 wireproto: make a number of commands batchable
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14622
diff changeset
358 @batchable
11586
ddaaaa23bb8f protocol: move basic ssh client commands to wirerepository
Matt Mackall <mpm@selenic.com>
parents: 11585
diff changeset
359 def lookup(self, key):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
360 self.requirecap(b'lookup', _(b'look up remote revision'))
14623
e7c9fdbbb902 wireproto: make a number of commands batchable
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14622
diff changeset
361 f = future()
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
362 yield {b'key': encoding.fromlocal(key)}, f
14623
e7c9fdbbb902 wireproto: make a number of commands batchable
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14622
diff changeset
363 d = f.value
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
364 success, data = d[:-1].split(b" ", 1)
11586
ddaaaa23bb8f protocol: move basic ssh client commands to wirerepository
Matt Mackall <mpm@selenic.com>
parents: 11585
diff changeset
365 if int(success):
14623
e7c9fdbbb902 wireproto: make a number of commands batchable
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14622
diff changeset
366 yield bin(data)
34062
6c6169f71b8d wireproto: do not abort after successful lookup
Kyle Lippincott <spectral@google.com>
parents: 33806
diff changeset
367 else:
6c6169f71b8d wireproto: do not abort after successful lookup
Kyle Lippincott <spectral@google.com>
parents: 33806
diff changeset
368 self._abort(error.RepoError(data))
11586
ddaaaa23bb8f protocol: move basic ssh client commands to wirerepository
Matt Mackall <mpm@selenic.com>
parents: 11585
diff changeset
369
14623
e7c9fdbbb902 wireproto: make a number of commands batchable
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14622
diff changeset
370 @batchable
11586
ddaaaa23bb8f protocol: move basic ssh client commands to wirerepository
Matt Mackall <mpm@selenic.com>
parents: 11585
diff changeset
371 def heads(self):
14623
e7c9fdbbb902 wireproto: make a number of commands batchable
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14622
diff changeset
372 f = future()
e7c9fdbbb902 wireproto: make a number of commands batchable
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14622
diff changeset
373 yield {}, f
e7c9fdbbb902 wireproto: make a number of commands batchable
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14622
diff changeset
374 d = f.value
11586
ddaaaa23bb8f protocol: move basic ssh client commands to wirerepository
Matt Mackall <mpm@selenic.com>
parents: 11585
diff changeset
375 try:
37612
5e71dea79aae wireproto: move value encoding functions to wireprototypes (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37546
diff changeset
376 yield wireprototypes.decodelist(d[:-1])
13726
378522bdc059 wireproto: avoid naked excepts
Matt Mackall <mpm@selenic.com>
parents: 13723
diff changeset
377 except ValueError:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
378 self._abort(error.ResponseError(_(b"unexpected response:"), d))
11586
ddaaaa23bb8f protocol: move basic ssh client commands to wirerepository
Matt Mackall <mpm@selenic.com>
parents: 11585
diff changeset
379
14623
e7c9fdbbb902 wireproto: make a number of commands batchable
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14622
diff changeset
380 @batchable
13723
e615765fdcc7 wireproto: add known([id]) function
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13722
diff changeset
381 def known(self, nodes):
14623
e7c9fdbbb902 wireproto: make a number of commands batchable
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14622
diff changeset
382 f = future()
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
383 yield {b'nodes': wireprototypes.encodelist(nodes)}, f
14623
e7c9fdbbb902 wireproto: make a number of commands batchable
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14622
diff changeset
384 d = f.value
13723
e615765fdcc7 wireproto: add known([id]) function
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13722
diff changeset
385 try:
38328
aa9dd805234d py3: fix loop over byte string in wireprotov1peer
Yuya Nishihara <yuya@tcha.org>
parents: 37810
diff changeset
386 yield [bool(int(b)) for b in pycompat.iterbytestr(d)]
13726
378522bdc059 wireproto: avoid naked excepts
Matt Mackall <mpm@selenic.com>
parents: 13723
diff changeset
387 except ValueError:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
388 self._abort(error.ResponseError(_(b"unexpected response:"), d))
13723
e615765fdcc7 wireproto: add known([id]) function
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13722
diff changeset
389
14623
e7c9fdbbb902 wireproto: make a number of commands batchable
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14622
diff changeset
390 @batchable
11586
ddaaaa23bb8f protocol: move basic ssh client commands to wirerepository
Matt Mackall <mpm@selenic.com>
parents: 11585
diff changeset
391 def branchmap(self):
14623
e7c9fdbbb902 wireproto: make a number of commands batchable
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14622
diff changeset
392 f = future()
e7c9fdbbb902 wireproto: make a number of commands batchable
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14622
diff changeset
393 yield {}, f
e7c9fdbbb902 wireproto: make a number of commands batchable
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14622
diff changeset
394 d = f.value
11586
ddaaaa23bb8f protocol: move basic ssh client commands to wirerepository
Matt Mackall <mpm@selenic.com>
parents: 11585
diff changeset
395 try:
ddaaaa23bb8f protocol: move basic ssh client commands to wirerepository
Matt Mackall <mpm@selenic.com>
parents: 11585
diff changeset
396 branchmap = {}
ddaaaa23bb8f protocol: move basic ssh client commands to wirerepository
Matt Mackall <mpm@selenic.com>
parents: 11585
diff changeset
397 for branchpart in d.splitlines():
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
398 branchname, branchheads = branchpart.split(b' ', 1)
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28666
diff changeset
399 branchname = encoding.tolocal(urlreq.unquote(branchname))
37612
5e71dea79aae wireproto: move value encoding functions to wireprototypes (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37546
diff changeset
400 branchheads = wireprototypes.decodelist(branchheads)
11586
ddaaaa23bb8f protocol: move basic ssh client commands to wirerepository
Matt Mackall <mpm@selenic.com>
parents: 11585
diff changeset
401 branchmap[branchname] = branchheads
14623
e7c9fdbbb902 wireproto: make a number of commands batchable
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14622
diff changeset
402 yield branchmap
11586
ddaaaa23bb8f protocol: move basic ssh client commands to wirerepository
Matt Mackall <mpm@selenic.com>
parents: 11585
diff changeset
403 except TypeError:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
404 self._abort(error.ResponseError(_(b"unexpected response:"), d))
11586
ddaaaa23bb8f protocol: move basic ssh client commands to wirerepository
Matt Mackall <mpm@selenic.com>
parents: 11585
diff changeset
405
33806
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
406 @batchable
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
407 def listkeys(self, namespace):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
408 if not self.capable(b'pushkey'):
33806
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
409 yield {}, None
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
410 f = future()
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
411 self.ui.debug(b'preparing listkeys for "%s"\n' % namespace)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
412 yield {b'namespace': encoding.fromlocal(namespace)}, f
33806
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
413 d = f.value
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
414 self.ui.debug(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
415 b'received listkey for "%s": %i bytes\n' % (namespace, len(d))
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
416 )
33806
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
417 yield pushkeymod.decodekeys(d)
11586
ddaaaa23bb8f protocol: move basic ssh client commands to wirerepository
Matt Mackall <mpm@selenic.com>
parents: 11585
diff changeset
418
14623
e7c9fdbbb902 wireproto: make a number of commands batchable
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14622
diff changeset
419 @batchable
11586
ddaaaa23bb8f protocol: move basic ssh client commands to wirerepository
Matt Mackall <mpm@selenic.com>
parents: 11585
diff changeset
420 def pushkey(self, namespace, key, old, new):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
421 if not self.capable(b'pushkey'):
14623
e7c9fdbbb902 wireproto: make a number of commands batchable
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14622
diff changeset
422 yield False, None
e7c9fdbbb902 wireproto: make a number of commands batchable
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14622
diff changeset
423 f = future()
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
424 self.ui.debug(b'preparing pushkey for "%s:%s"\n' % (namespace, key))
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
425 yield {
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
426 b'namespace': encoding.fromlocal(namespace),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
427 b'key': encoding.fromlocal(key),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
428 b'old': encoding.fromlocal(old),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
429 b'new': encoding.fromlocal(new),
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
430 }, f
14623
e7c9fdbbb902 wireproto: make a number of commands batchable
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14622
diff changeset
431 d = f.value
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
432 d, output = d.split(b'\n', 1)
13450
b3f9af7c22c5 wireproto: catch possible cast error in pushkey
David Soria Parra <dsp@php.net>
parents: 13050
diff changeset
433 try:
b3f9af7c22c5 wireproto: catch possible cast error in pushkey
David Soria Parra <dsp@php.net>
parents: 13050
diff changeset
434 d = bool(int(d))
b3f9af7c22c5 wireproto: catch possible cast error in pushkey
David Soria Parra <dsp@php.net>
parents: 13050
diff changeset
435 except ValueError:
b3f9af7c22c5 wireproto: catch possible cast error in pushkey
David Soria Parra <dsp@php.net>
parents: 13050
diff changeset
436 raise error.ResponseError(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
437 _(b'push failed (unexpected response):'), d
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
438 )
15652
ca6accdad79c wireproto: handle other server output in pushkey
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 15585
diff changeset
439 for l in output.splitlines(True):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
440 self.ui.status(_(b'remote: '), l)
14623
e7c9fdbbb902 wireproto: make a number of commands batchable
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14622
diff changeset
441 yield d
11586
ddaaaa23bb8f protocol: move basic ssh client commands to wirerepository
Matt Mackall <mpm@selenic.com>
parents: 11585
diff changeset
442
11588
8a1f625e971d protocol: unify stream_out client code
Matt Mackall <mpm@selenic.com>
parents: 11587
diff changeset
443 def stream_out(self):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
444 return self._callstream(b'stream_out')
11588
8a1f625e971d protocol: unify stream_out client code
Matt Mackall <mpm@selenic.com>
parents: 11587
diff changeset
445
21646
ce25f465e572 getbundle: declare type of parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21631
diff changeset
446 def getbundle(self, source, **kwargs):
34739
b880cc11da5d wireproto: bounce kwargs to/from bytes/str as needed
Augie Fackler <augie@google.com>
parents: 34731
diff changeset
447 kwargs = pycompat.byteskwargs(kwargs)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
448 self.requirecap(b'getbundle', _(b'look up remote changes'))
13741
b51bf961b3cb wireproto: add getbundle() function
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13726
diff changeset
449 opts = {}
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
450 bundlecaps = kwargs.get(b'bundlecaps') or set()
43106
d783f945a701 py3: finish porting iteritems() to pycompat and remove source transformer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43089
diff changeset
451 for key, value in pycompat.iteritems(kwargs):
21646
ce25f465e572 getbundle: declare type of parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21631
diff changeset
452 if value is None:
ce25f465e572 getbundle: declare type of parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21631
diff changeset
453 continue
37613
96d735601ca1 wireproto: move gboptsmap to wireprototypes and rename (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37612
diff changeset
454 keytype = wireprototypes.GETBUNDLE_ARGUMENTS.get(key)
21646
ce25f465e572 getbundle: declare type of parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21631
diff changeset
455 if keytype is None:
34730
6be264009841 wireproto: use a proper exception instead of `assert False`
Augie Fackler <augie@google.com>
parents: 34729
diff changeset
456 raise error.ProgrammingError(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
457 b'Unexpectedly None keytype for key %s' % key
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
458 )
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
459 elif keytype == b'nodes':
37612
5e71dea79aae wireproto: move value encoding functions to wireprototypes (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37546
diff changeset
460 value = wireprototypes.encodelist(value)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
461 elif keytype == b'csv':
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
462 value = b','.join(value)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
463 elif keytype == b'scsv':
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
464 value = b','.join(sorted(value))
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
465 elif keytype == b'boolean':
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
466 value = b'%i' % bool(value)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
467 elif keytype != b'plain':
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
468 raise KeyError(b'unknown getbundle option type %s' % keytype)
21646
ce25f465e572 getbundle: declare type of parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21631
diff changeset
469 opts[key] = value
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
470 f = self._callcompressable(b"getbundle", **pycompat.strkwargs(opts))
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
471 if any((cap.startswith(b'HG2') for cap in bundlecaps)):
24641
60fecc5b14a4 unbundle20: retrieve unbundler instances through a factory function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23878
diff changeset
472 return bundle2.getunbundler(self.ui, f)
21069
0a9cae236738 bundle2: allow bundle2 for pulling over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21064
diff changeset
473 else:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
474 return changegroupmod.cg1unpacker(f, b'UN')
13741
b51bf961b3cb wireproto: add getbundle() function
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13726
diff changeset
475
37646
72e26319f3b8 wireproto: use command executor for unbundle
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37635
diff changeset
476 def unbundle(self, bundle, heads, url):
11592
26e0782b8380 protocol: unify client unbundle support
Matt Mackall <mpm@selenic.com>
parents: 11591
diff changeset
477 '''Send cg (a readable file-like object representing the
26e0782b8380 protocol: unify client unbundle support
Matt Mackall <mpm@selenic.com>
parents: 11591
diff changeset
478 changegroup to push, typically a chunkbuffer object) to the
21075
438803e4bd97 bundle2: support for push over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21072
diff changeset
479 remote server as a bundle.
438803e4bd97 bundle2: support for push over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21072
diff changeset
480
438803e4bd97 bundle2: support for push over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21072
diff changeset
481 When pushing a bundle10 stream, return an integer indicating the
32880
4c2a46f89f08 wireproto: update reference to deleted addchangegroup()
Martin von Zweigbergk <martinvonz@google.com>
parents: 32744
diff changeset
482 result of the push (see changegroup.apply()).
21075
438803e4bd97 bundle2: support for push over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21072
diff changeset
483
29706
7f6130c7ffe1 wirepeer: rename confusing `source` parameter
Augie Fackler <augie@google.com>
parents: 29590
diff changeset
484 When pushing a bundle20 stream, return a bundle20 stream.
7f6130c7ffe1 wirepeer: rename confusing `source` parameter
Augie Fackler <augie@google.com>
parents: 29590
diff changeset
485
7f6130c7ffe1 wirepeer: rename confusing `source` parameter
Augie Fackler <augie@google.com>
parents: 29590
diff changeset
486 `url` is the url the client thinks it's pushing to, which is
7f6130c7ffe1 wirepeer: rename confusing `source` parameter
Augie Fackler <augie@google.com>
parents: 29590
diff changeset
487 visible to hooks.
7f6130c7ffe1 wirepeer: rename confusing `source` parameter
Augie Fackler <augie@google.com>
parents: 29590
diff changeset
488 '''
11592
26e0782b8380 protocol: unify client unbundle support
Matt Mackall <mpm@selenic.com>
parents: 11591
diff changeset
489
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
490 if heads != [b'force'] and self.capable(b'unbundlehash'):
37612
5e71dea79aae wireproto: move value encoding functions to wireprototypes (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37546
diff changeset
491 heads = wireprototypes.encodelist(
44060
a61287a95dc3 core: migrate uses of hashlib.sha1 to hashutil.sha1
Augie Fackler <augie@google.com>
parents: 43506
diff changeset
492 [b'hashed', hashutil.sha1(b''.join(sorted(heads))).digest()]
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
493 )
13942
88f0e41d8802 wireproto: allow unbundle with hashed heads parameter (issue2126)
Shuhei Takahashi <takahashi.shuhei@gmail.com>
parents: 13741
diff changeset
494 else:
37612
5e71dea79aae wireproto: move value encoding functions to wireprototypes (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37546
diff changeset
495 heads = wireprototypes.encodelist(heads)
13942
88f0e41d8802 wireproto: allow unbundle with hashed heads parameter (issue2126)
Shuhei Takahashi <takahashi.shuhei@gmail.com>
parents: 13741
diff changeset
496
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
497 if util.safehasattr(bundle, b'deltaheader'):
21075
438803e4bd97 bundle2: support for push over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21072
diff changeset
498 # this a bundle10, do the old style call sequence
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
499 ret, output = self._callpush(b"unbundle", bundle, heads=heads)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
500 if ret == b"":
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
501 raise error.ResponseError(_(b'push failed:'), output)
21075
438803e4bd97 bundle2: support for push over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21072
diff changeset
502 try:
438803e4bd97 bundle2: support for push over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21072
diff changeset
503 ret = int(ret)
438803e4bd97 bundle2: support for push over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21072
diff changeset
504 except ValueError:
438803e4bd97 bundle2: support for push over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21072
diff changeset
505 raise error.ResponseError(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
506 _(b'push failed (unexpected response):'), ret
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
507 )
11592
26e0782b8380 protocol: unify client unbundle support
Matt Mackall <mpm@selenic.com>
parents: 11591
diff changeset
508
21075
438803e4bd97 bundle2: support for push over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21072
diff changeset
509 for l in output.splitlines(True):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
510 self.ui.status(_(b'remote: '), l)
21075
438803e4bd97 bundle2: support for push over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21072
diff changeset
511 else:
438803e4bd97 bundle2: support for push over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21072
diff changeset
512 # bundle2 push. Send a stream, fetch a stream.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
513 stream = self._calltwowaystream(b'unbundle', bundle, heads=heads)
24641
60fecc5b14a4 unbundle20: retrieve unbundler instances through a factory function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23878
diff changeset
514 ret = bundle2.getunbundler(self.ui, stream)
11592
26e0782b8380 protocol: unify client unbundle support
Matt Mackall <mpm@selenic.com>
parents: 11591
diff changeset
515 return ret
26e0782b8380 protocol: unify client unbundle support
Matt Mackall <mpm@selenic.com>
parents: 11591
diff changeset
516
37320
39f7d4ee8bcd repository: port peer interfaces to zope.interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37295
diff changeset
517 # End of ipeercommands interface.
33806
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
518
37320
39f7d4ee8bcd repository: port peer interfaces to zope.interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37295
diff changeset
519 # Begin of ipeerlegacycommands interface.
33806
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
520
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
521 def branches(self, nodes):
37612
5e71dea79aae wireproto: move value encoding functions to wireprototypes (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37546
diff changeset
522 n = wireprototypes.encodelist(nodes)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
523 d = self._call(b"branches", nodes=n)
33806
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
524 try:
37612
5e71dea79aae wireproto: move value encoding functions to wireprototypes (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37546
diff changeset
525 br = [tuple(wireprototypes.decodelist(b)) for b in d.splitlines()]
33806
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
526 return br
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
527 except ValueError:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
528 self._abort(error.ResponseError(_(b"unexpected response:"), d))
33806
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
529
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
530 def between(self, pairs):
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
531 batch = 8 # avoid giant requests
33806
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
532 r = []
38783
e7aa113b14f7 global: use pycompat.xrange()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38328
diff changeset
533 for i in pycompat.xrange(0, len(pairs), batch):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
534 n = b" ".join(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
535 [
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
536 wireprototypes.encodelist(p, b'-')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
537 for p in pairs[i : i + batch]
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
538 ]
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
539 )
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
540 d = self._call(b"between", pairs=n)
33806
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
541 try:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
542 r.extend(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
543 l and wireprototypes.decodelist(l) or []
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
544 for l in d.splitlines()
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
545 )
33806
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
546 except ValueError:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
547 self._abort(error.ResponseError(_(b"unexpected response:"), d))
33806
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
548 return r
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
549
37635
cc8c06835097 wireproto: convert legacy commands to command executor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37633
diff changeset
550 def changegroup(self, nodes, source):
37612
5e71dea79aae wireproto: move value encoding functions to wireprototypes (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37546
diff changeset
551 n = wireprototypes.encodelist(nodes)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
552 f = self._callcompressable(b"changegroup", roots=n)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
553 return changegroupmod.cg1unpacker(f, b'UN')
33806
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
554
37635
cc8c06835097 wireproto: convert legacy commands to command executor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37633
diff changeset
555 def changegroupsubset(self, bases, heads, source):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
556 self.requirecap(b'changegroupsubset', _(b'look up remote changes'))
37612
5e71dea79aae wireproto: move value encoding functions to wireprototypes (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37546
diff changeset
557 bases = wireprototypes.encodelist(bases)
5e71dea79aae wireproto: move value encoding functions to wireprototypes (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37546
diff changeset
558 heads = wireprototypes.encodelist(heads)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
559 f = self._callcompressable(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
560 b"changegroupsubset", bases=bases, heads=heads
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
561 )
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
562 return changegroupmod.cg1unpacker(f, b'UN')
33806
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
563
37320
39f7d4ee8bcd repository: port peer interfaces to zope.interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37295
diff changeset
564 # End of ipeerlegacycommands interface.
33806
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
565
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
566 def _submitbatch(self, req):
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
567 """run batch request <req> on the server
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
568
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
569 Returns an iterator of the raw responses from the server.
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
570 """
36945
4901d1e22b27 peer-request: include more details about batch commands
Boris Feld <boris.feld@octobus.net>
parents: 36840
diff changeset
571 ui = self.ui
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
572 if ui.debugflag and ui.configbool(b'devel', b'debug.peer-request'):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
573 ui.debug(b'devel-peer-request: batched-content\n')
36945
4901d1e22b27 peer-request: include more details about batch commands
Boris Feld <boris.feld@octobus.net>
parents: 36840
diff changeset
574 for op, args in req:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
575 msg = b'devel-peer-request: - %s (%d arguments)\n'
36945
4901d1e22b27 peer-request: include more details about batch commands
Boris Feld <boris.feld@octobus.net>
parents: 36840
diff changeset
576 ui.debug(msg % (op, len(args)))
4901d1e22b27 peer-request: include more details about batch commands
Boris Feld <boris.feld@octobus.net>
parents: 36840
diff changeset
577
37612
5e71dea79aae wireproto: move value encoding functions to wireprototypes (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37546
diff changeset
578 unescapearg = wireprototypes.unescapebatcharg
5e71dea79aae wireproto: move value encoding functions to wireprototypes (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37546
diff changeset
579
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
580 rsp = self._callstream(b"batch", cmds=encodebatchcmds(req))
33806
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
581 chunk = rsp.read(1024)
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
582 work = [chunk]
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
583 while chunk:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
584 while b';' not in chunk and chunk:
33806
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
585 chunk = rsp.read(1024)
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
586 work.append(chunk)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
587 merged = b''.join(work)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
588 while b';' in merged:
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
589 one, merged = merged.split(b';', 1)
33806
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
590 yield unescapearg(one)
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
591 chunk = rsp.read(1024)
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
592 work = [merged, chunk]
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
593 yield unescapearg(b''.join(work))
33806
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
594
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
595 def _submitone(self, op, args):
34739
b880cc11da5d wireproto: bounce kwargs to/from bytes/str as needed
Augie Fackler <augie@google.com>
parents: 34731
diff changeset
596 return self._call(op, **pycompat.strkwargs(args))
33806
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
597
14048
58e58406ed19 wireproto: add test for new optional arg missing on server
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13942
diff changeset
598 def debugwireargs(self, one, two, three=None, four=None, five=None):
13720
9c4e04fe267e debug: add debugwireargs to test argument passing over the wire
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13450
diff changeset
599 # don't pass optional arguments left at their default value
9c4e04fe267e debug: add debugwireargs to test argument passing over the wire
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13450
diff changeset
600 opts = {}
9c4e04fe267e debug: add debugwireargs to test argument passing over the wire
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13450
diff changeset
601 if three is not None:
43506
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
602 opts['three'] = three
13720
9c4e04fe267e debug: add debugwireargs to test argument passing over the wire
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13450
diff changeset
603 if four is not None:
43506
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
604 opts['four'] = four
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
605 return self._call(b'debugwireargs', one=one, two=two, **opts)
13720
9c4e04fe267e debug: add debugwireargs to test argument passing over the wire
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13450
diff changeset
606
20904
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20903
diff changeset
607 def _call(self, cmd, **args):
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20903
diff changeset
608 """execute <cmd> on the server
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20903
diff changeset
609
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20903
diff changeset
610 The command is expected to return a simple string.
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20903
diff changeset
611
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20903
diff changeset
612 returns the server reply as a string."""
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20903
diff changeset
613 raise NotImplementedError()
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20903
diff changeset
614
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20903
diff changeset
615 def _callstream(self, cmd, **args):
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20903
diff changeset
616 """execute <cmd> on the server
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20903
diff changeset
617
28435
176736afa886 wireproto: document quirk of _callstream between http and ssh
Augie Fackler <augie@google.com>
parents: 27633
diff changeset
618 The command is expected to return a stream. Note that if the
176736afa886 wireproto: document quirk of _callstream between http and ssh
Augie Fackler <augie@google.com>
parents: 27633
diff changeset
619 command doesn't return a stream, _callstream behaves
176736afa886 wireproto: document quirk of _callstream between http and ssh
Augie Fackler <augie@google.com>
parents: 27633
diff changeset
620 differently for ssh and http peers.
20904
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20903
diff changeset
621
28435
176736afa886 wireproto: document quirk of _callstream between http and ssh
Augie Fackler <augie@google.com>
parents: 27633
diff changeset
622 returns the server reply as a file like object.
176736afa886 wireproto: document quirk of _callstream between http and ssh
Augie Fackler <augie@google.com>
parents: 27633
diff changeset
623 """
20904
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20903
diff changeset
624 raise NotImplementedError()
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20903
diff changeset
625
20905
167047ba3cfa wireproto: drop the _decompress method in favor a new call type
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20904
diff changeset
626 def _callcompressable(self, cmd, **args):
167047ba3cfa wireproto: drop the _decompress method in favor a new call type
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20904
diff changeset
627 """execute <cmd> on the server
167047ba3cfa wireproto: drop the _decompress method in favor a new call type
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20904
diff changeset
628
167047ba3cfa wireproto: drop the _decompress method in favor a new call type
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20904
diff changeset
629 The command is expected to return a stream.
167047ba3cfa wireproto: drop the _decompress method in favor a new call type
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20904
diff changeset
630
21024
7731a2281cf0 spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents: 20969
diff changeset
631 The stream may have been compressed in some implementations. This
20905
167047ba3cfa wireproto: drop the _decompress method in favor a new call type
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20904
diff changeset
632 function takes care of the decompression. This is the only difference
167047ba3cfa wireproto: drop the _decompress method in favor a new call type
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20904
diff changeset
633 with _callstream.
167047ba3cfa wireproto: drop the _decompress method in favor a new call type
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20904
diff changeset
634
167047ba3cfa wireproto: drop the _decompress method in favor a new call type
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20904
diff changeset
635 returns the server reply as a file like object.
167047ba3cfa wireproto: drop the _decompress method in favor a new call type
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20904
diff changeset
636 """
167047ba3cfa wireproto: drop the _decompress method in favor a new call type
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20904
diff changeset
637 raise NotImplementedError()
167047ba3cfa wireproto: drop the _decompress method in favor a new call type
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20904
diff changeset
638
20904
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20903
diff changeset
639 def _callpush(self, cmd, fp, **args):
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20903
diff changeset
640 """execute a <cmd> on server
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20903
diff changeset
641
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20903
diff changeset
642 The command is expected to be related to a push. Push has a special
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20903
diff changeset
643 return method.
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20903
diff changeset
644
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20903
diff changeset
645 returns the server reply as a (ret, output) tuple. ret is either
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20903
diff changeset
646 empty (error) or a stringified int.
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20903
diff changeset
647 """
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20903
diff changeset
648 raise NotImplementedError()
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20903
diff changeset
649
21072
0879352d67d8 wireproto: add a _calltwowaystream method on wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21069
diff changeset
650 def _calltwowaystream(self, cmd, fp, **args):
0879352d67d8 wireproto: add a _calltwowaystream method on wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21069
diff changeset
651 """execute <cmd> on server
0879352d67d8 wireproto: add a _calltwowaystream method on wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21069
diff changeset
652
0879352d67d8 wireproto: add a _calltwowaystream method on wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21069
diff changeset
653 The command will send a stream to the server and get a stream in reply.
0879352d67d8 wireproto: add a _calltwowaystream method on wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21069
diff changeset
654 """
0879352d67d8 wireproto: add a _calltwowaystream method on wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21069
diff changeset
655 raise NotImplementedError()
0879352d67d8 wireproto: add a _calltwowaystream method on wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21069
diff changeset
656
20904
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20903
diff changeset
657 def _abort(self, exception):
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20903
diff changeset
658 """clearly abort the wire protocol connection and raise the exception
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20903
diff changeset
659 """
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20903
diff changeset
660 raise NotImplementedError()