Mercurial > hg
annotate mercurial/wireprotov2peer.py @ 39561:d06834e0f48e
wireprotov2peer: stream decoded responses
Previously, wire protocol version 2 would buffer all response data.
Only once all data was received did we CBOR decode it and resolve
the future associated with the command. This was obviously not
desirable. In future commits that introduce large response payloads,
this caused significant memory bloat and slowed down client
operations due to waiting on the server.
This commit refactors the response handling code so that response
data can be streamed.
Command response objects now contain a buffered CBOR decoder. As
new data arrives, it is fed into the decoder. Decoded objects are
made available to the generator as they are decoded.
Because there is a separate thread processing incoming frames and
feeding data into the response object, there is the potential for
race conditions when mutating response objects. So a lock has been
added to guard access to critical state variables.
Because the generator emitting decoded objects needs to wait on
those objects to become available, we've added an Event for the
generator to wait on so it doesn't busy loop. This does mean
there is the potential for deadlocks. And I'm pretty sure they can
occur in some scenarios. We already have a handful of TODOs around
this. But I've added some more. Fixing this will likely require
moving the background thread receiving frames into clienthandler.
We likely would have done this anyway when implementing the client
bits for the SSH transport.
Test output changes because the initial CBOR map holding the overall
response state is now always handled internally by the response
object.
Differential Revision: https://phab.mercurial-scm.org/D4474
author | Gregory Szorc <gregory.szorc@gmail.com> |
---|---|
date | Wed, 29 Aug 2018 15:17:11 -0700 |
parents | 07b58266bce3 |
children | f5a05bb48116 |
rev | line source |
---|---|
37719
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
1 # wireprotov2peer.py - client side code for wire protocol version 2 |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
2 # |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
3 # Copyright 2018 Gregory Szorc <gregory.szorc@gmail.com> |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
4 # |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
5 # This software may be used and distributed according to the terms of the |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
6 # GNU General Public License version 2 or any later version. |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
7 |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
8 from __future__ import absolute_import |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
9 |
39561
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
10 import threading |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
11 |
37719
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
12 from .i18n import _ |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
13 from . import ( |
37721
f7673845b167
wireprotov2: decode responses to their expected types
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37720
diff
changeset
|
14 encoding, |
37719
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
15 error, |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
16 wireprotoframing, |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
17 ) |
39450
9f51fd22ed50
wireprotov2peer: use our CBOR decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39438
diff
changeset
|
18 from .utils import ( |
9f51fd22ed50
wireprotov2peer: use our CBOR decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39438
diff
changeset
|
19 cborutil, |
9f51fd22ed50
wireprotov2peer: use our CBOR decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39438
diff
changeset
|
20 ) |
37719
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
21 |
37725
3ea8323d6f95
wireprotov2: change command response protocol to include a leading map
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37722
diff
changeset
|
22 def formatrichmessage(atoms): |
3ea8323d6f95
wireprotov2: change command response protocol to include a leading map
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37722
diff
changeset
|
23 """Format an encoded message from the framing protocol.""" |
3ea8323d6f95
wireprotov2: change command response protocol to include a leading map
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37722
diff
changeset
|
24 |
3ea8323d6f95
wireprotov2: change command response protocol to include a leading map
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37722
diff
changeset
|
25 chunks = [] |
3ea8323d6f95
wireprotov2: change command response protocol to include a leading map
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37722
diff
changeset
|
26 |
3ea8323d6f95
wireprotov2: change command response protocol to include a leading map
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37722
diff
changeset
|
27 for atom in atoms: |
3ea8323d6f95
wireprotov2: change command response protocol to include a leading map
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37722
diff
changeset
|
28 msg = _(atom[b'msg']) |
3ea8323d6f95
wireprotov2: change command response protocol to include a leading map
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37722
diff
changeset
|
29 |
3ea8323d6f95
wireprotov2: change command response protocol to include a leading map
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37722
diff
changeset
|
30 if b'args' in atom: |
39486
43d92d68ac88
wireprotov2peer: properly format errors
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39485
diff
changeset
|
31 msg = msg % tuple(atom[b'args']) |
37725
3ea8323d6f95
wireprotov2: change command response protocol to include a leading map
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37722
diff
changeset
|
32 |
3ea8323d6f95
wireprotov2: change command response protocol to include a leading map
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37722
diff
changeset
|
33 chunks.append(msg) |
3ea8323d6f95
wireprotov2: change command response protocol to include a leading map
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37722
diff
changeset
|
34 |
3ea8323d6f95
wireprotov2: change command response protocol to include a leading map
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37722
diff
changeset
|
35 return b''.join(chunks) |
3ea8323d6f95
wireprotov2: change command response protocol to include a leading map
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37722
diff
changeset
|
36 |
37720
d715a85003c8
wireprotov2: establish a type for representing command response
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37719
diff
changeset
|
37 class commandresponse(object): |
39561
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
38 """Represents the response to a command request. |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
39 |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
40 Instances track the state of the command and hold its results. |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
41 |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
42 An external entity is required to update the state of the object when |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
43 events occur. |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
44 """ |
37720
d715a85003c8
wireprotov2: establish a type for representing command response
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37719
diff
changeset
|
45 |
d715a85003c8
wireprotov2: establish a type for representing command response
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37719
diff
changeset
|
46 def __init__(self, requestid, command): |
d715a85003c8
wireprotov2: establish a type for representing command response
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37719
diff
changeset
|
47 self.requestid = requestid |
d715a85003c8
wireprotov2: establish a type for representing command response
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37719
diff
changeset
|
48 self.command = command |
d715a85003c8
wireprotov2: establish a type for representing command response
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37719
diff
changeset
|
49 |
39561
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
50 # Whether all remote input related to this command has been |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
51 # received. |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
52 self._inputcomplete = False |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
53 |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
54 # We have a lock that is acquired when important object state is |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
55 # mutated. This is to prevent race conditions between 1 thread |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
56 # sending us new data and another consuming it. |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
57 self._lock = threading.RLock() |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
58 |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
59 # An event is set when state of the object changes. This event |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
60 # is waited on by the generator emitting objects. |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
61 self._serviceable = threading.Event() |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
62 |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
63 self._pendingevents = [] |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
64 self._decoder = cborutil.bufferingdecoder() |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
65 self._seeninitial = False |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
66 |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
67 def _oninputcomplete(self): |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
68 with self._lock: |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
69 self._inputcomplete = True |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
70 self._serviceable.set() |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
71 |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
72 def _onresponsedata(self, data): |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
73 available, readcount, wanted = self._decoder.decode(data) |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
74 |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
75 if not available: |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
76 return |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
77 |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
78 with self._lock: |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
79 for o in self._decoder.getavailable(): |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
80 if not self._seeninitial: |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
81 self._handleinitial(o) |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
82 continue |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
83 |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
84 self._pendingevents.append(o) |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
85 |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
86 self._serviceable.set() |
37720
d715a85003c8
wireprotov2: establish a type for representing command response
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37719
diff
changeset
|
87 |
39561
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
88 def _handleinitial(self, o): |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
89 self._seeninitial = True |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
90 if o[b'status'] == 'ok': |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
91 return |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
92 |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
93 atoms = [{'msg': o[b'error'][b'message']}] |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
94 if b'args' in o[b'error']: |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
95 atoms[0]['args'] = o[b'error'][b'args'] |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
96 |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
97 raise error.RepoError(formatrichmessage(atoms)) |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
98 |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
99 def objects(self): |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
100 """Obtained decoded objects from this response. |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
101 |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
102 This is a generator of data structures that were decoded from the |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
103 command response. |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
104 |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
105 Obtaining the next member of the generator may block due to waiting |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
106 on external data to become available. |
37720
d715a85003c8
wireprotov2: establish a type for representing command response
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37719
diff
changeset
|
107 |
39561
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
108 If the server encountered an error in the middle of serving the data |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
109 or if another error occurred, an exception may be raised when |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
110 advancing the generator. |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
111 """ |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
112 while True: |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
113 # TODO this can infinite loop if self._inputcomplete is never |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
114 # set. We likely want to tie the lifetime of this object/state |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
115 # to that of the background thread receiving frames and updating |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
116 # our state. |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
117 self._serviceable.wait(1.0) |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
118 |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
119 with self._lock: |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
120 self._serviceable.clear() |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
121 |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
122 # Make copies because objects could be mutated during |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
123 # iteration. |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
124 stop = self._inputcomplete |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
125 pending = list(self._pendingevents) |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
126 self._pendingevents[:] = [] |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
127 |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
128 for o in pending: |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
129 yield o |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
130 |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
131 if stop: |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
132 break |
37720
d715a85003c8
wireprotov2: establish a type for representing command response
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37719
diff
changeset
|
133 |
37719
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
134 class clienthandler(object): |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
135 """Object to handle higher-level client activities. |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
136 |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
137 The ``clientreactor`` is used to hold low-level state about the frame-based |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
138 protocol, such as which requests and streams are active. This type is used |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
139 for higher-level operations, such as reading frames from a socket, exposing |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
140 and managing a higher-level primitive for representing command responses, |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
141 etc. This class is what peers should probably use to bridge wire activity |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
142 with the higher-level peer API. |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
143 """ |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
144 |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
145 def __init__(self, ui, clientreactor): |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
146 self._ui = ui |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
147 self._reactor = clientreactor |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
148 self._requests = {} |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
149 self._futures = {} |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
150 self._responses = {} |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
151 |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
152 def callcommand(self, command, args, f): |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
153 """Register a request to call a command. |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
154 |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
155 Returns an iterable of frames that should be sent over the wire. |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
156 """ |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
157 request, action, meta = self._reactor.callcommand(command, args) |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
158 |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
159 if action != 'noop': |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
160 raise error.ProgrammingError('%s not yet supported' % action) |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
161 |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
162 rid = request.requestid |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
163 self._requests[rid] = request |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
164 self._futures[rid] = f |
39561
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
165 # TODO we need some kind of lifetime on response instances otherwise |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
166 # objects() may deadlock. |
37720
d715a85003c8
wireprotov2: establish a type for representing command response
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37719
diff
changeset
|
167 self._responses[rid] = commandresponse(rid, command) |
37719
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
168 |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
169 return iter(()) |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
170 |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
171 def flushcommands(self): |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
172 """Flush all queued commands. |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
173 |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
174 Returns an iterable of frames that should be sent over the wire. |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
175 """ |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
176 action, meta = self._reactor.flushcommands() |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
177 |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
178 if action != 'sendframes': |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
179 raise error.ProgrammingError('%s not yet supported' % action) |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
180 |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
181 return meta['framegen'] |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
182 |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
183 def readframe(self, fh): |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
184 """Attempt to read and process a frame. |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
185 |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
186 Returns None if no frame was read. Presumably this means EOF. |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
187 """ |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
188 frame = wireprotoframing.readframe(fh) |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
189 if frame is None: |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
190 # TODO tell reactor? |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
191 return |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
192 |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
193 self._ui.note(_('received %r\n') % frame) |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
194 self._processframe(frame) |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
195 |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
196 return True |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
197 |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
198 def _processframe(self, frame): |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
199 """Process a single read frame.""" |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
200 |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
201 action, meta = self._reactor.onframerecv(frame) |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
202 |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
203 if action == 'error': |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
204 e = error.RepoError(meta['message']) |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
205 |
39561
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
206 if frame.requestid in self._responses: |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
207 self._responses[frame.requestid]._oninputcomplete() |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
208 |
37719
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
209 if frame.requestid in self._futures: |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
210 self._futures[frame.requestid].set_exception(e) |
39561
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
211 del self._futures[frame.requestid] |
37719
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
212 else: |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
213 raise e |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
214 |
39559
07b58266bce3
wireprotov2: implement commands as a generator of objects
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39486
diff
changeset
|
215 return |
07b58266bce3
wireprotov2: implement commands as a generator of objects
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39486
diff
changeset
|
216 |
37719
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
217 if frame.requestid not in self._requests: |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
218 raise error.ProgrammingError( |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
219 'received frame for unknown request; this is either a bug in ' |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
220 'the clientreactor not screening for this or this instance was ' |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
221 'never told about this request: %r' % frame) |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
222 |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
223 response = self._responses[frame.requestid] |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
224 |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
225 if action == 'responsedata': |
39485
42bc1c70a6b8
wireprotov2peer: report exceptions in frame handling against request future
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39450
diff
changeset
|
226 # Any failures processing this frame should bubble up to the |
42bc1c70a6b8
wireprotov2peer: report exceptions in frame handling against request future
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39450
diff
changeset
|
227 # future tracking the request. |
42bc1c70a6b8
wireprotov2peer: report exceptions in frame handling against request future
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39450
diff
changeset
|
228 try: |
42bc1c70a6b8
wireprotov2peer: report exceptions in frame handling against request future
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39450
diff
changeset
|
229 self._processresponsedata(frame, meta, response) |
42bc1c70a6b8
wireprotov2peer: report exceptions in frame handling against request future
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39450
diff
changeset
|
230 except BaseException as e: |
42bc1c70a6b8
wireprotov2peer: report exceptions in frame handling against request future
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39450
diff
changeset
|
231 self._futures[frame.requestid].set_exception(e) |
39561
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
232 del self._futures[frame.requestid] |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
233 response._oninputcomplete() |
37719
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
234 else: |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
235 raise error.ProgrammingError( |
a656cba08a04
wireprotov2: move response handling out of httppeer
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
236 'unhandled action from clientreactor: %s' % action) |
37721
f7673845b167
wireprotov2: decode responses to their expected types
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37720
diff
changeset
|
237 |
39438
c734a5c82f38
wireprotov2peer: split responsedata handling into separate function
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39435
diff
changeset
|
238 def _processresponsedata(self, frame, meta, response): |
39561
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
239 # This can raise. The caller can handle it. |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
240 response._onresponsedata(meta['data']) |
39438
c734a5c82f38
wireprotov2peer: split responsedata handling into separate function
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39435
diff
changeset
|
241 |
c734a5c82f38
wireprotov2peer: split responsedata handling into separate function
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39435
diff
changeset
|
242 if meta['eos']: |
39561
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
243 response._oninputcomplete() |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
244 del self._requests[frame.requestid] |
39438
c734a5c82f38
wireprotov2peer: split responsedata handling into separate function
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39435
diff
changeset
|
245 |
39561
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
246 # If the command has a decoder, we wait until all input has been |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
247 # received before resolving the future. Otherwise we resolve the |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
248 # future immediately. |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
249 if frame.requestid not in self._futures: |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
250 return |
39438
c734a5c82f38
wireprotov2peer: split responsedata handling into separate function
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39435
diff
changeset
|
251 |
39561
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
252 if response.command not in COMMAND_DECODERS: |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
253 self._futures[frame.requestid].set_result(response.objects()) |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
254 del self._futures[frame.requestid] |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
255 elif response._inputcomplete: |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
256 decoded = COMMAND_DECODERS[response.command](response.objects()) |
d06834e0f48e
wireprotov2peer: stream decoded responses
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39559
diff
changeset
|
257 self._futures[frame.requestid].set_result(decoded) |
39438
c734a5c82f38
wireprotov2peer: split responsedata handling into separate function
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39435
diff
changeset
|
258 del self._futures[frame.requestid] |
c734a5c82f38
wireprotov2peer: split responsedata handling into separate function
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39435
diff
changeset
|
259 |
37725
3ea8323d6f95
wireprotov2: change command response protocol to include a leading map
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37722
diff
changeset
|
260 def decodebranchmap(objs): |
37721
f7673845b167
wireprotov2: decode responses to their expected types
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37720
diff
changeset
|
261 # Response should be a single CBOR map of branch name to array of nodes. |
37725
3ea8323d6f95
wireprotov2: change command response protocol to include a leading map
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37722
diff
changeset
|
262 bm = next(objs) |
37721
f7673845b167
wireprotov2: decode responses to their expected types
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37720
diff
changeset
|
263 |
f7673845b167
wireprotov2: decode responses to their expected types
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37720
diff
changeset
|
264 return {encoding.tolocal(k): v for k, v in bm.items()} |
f7673845b167
wireprotov2: decode responses to their expected types
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37720
diff
changeset
|
265 |
37725
3ea8323d6f95
wireprotov2: change command response protocol to include a leading map
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37722
diff
changeset
|
266 def decodeheads(objs): |
37721
f7673845b167
wireprotov2: decode responses to their expected types
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37720
diff
changeset
|
267 # Array of node bytestrings. |
37725
3ea8323d6f95
wireprotov2: change command response protocol to include a leading map
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37722
diff
changeset
|
268 return next(objs) |
37721
f7673845b167
wireprotov2: decode responses to their expected types
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37720
diff
changeset
|
269 |
37725
3ea8323d6f95
wireprotov2: change command response protocol to include a leading map
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37722
diff
changeset
|
270 def decodeknown(objs): |
37721
f7673845b167
wireprotov2: decode responses to their expected types
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37720
diff
changeset
|
271 # Bytestring where each byte is a 0 or 1. |
37725
3ea8323d6f95
wireprotov2: change command response protocol to include a leading map
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37722
diff
changeset
|
272 raw = next(objs) |
37721
f7673845b167
wireprotov2: decode responses to their expected types
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37720
diff
changeset
|
273 |
f7673845b167
wireprotov2: decode responses to their expected types
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37720
diff
changeset
|
274 return [True if c == '1' else False for c in raw] |
f7673845b167
wireprotov2: decode responses to their expected types
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37720
diff
changeset
|
275 |
37725
3ea8323d6f95
wireprotov2: change command response protocol to include a leading map
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37722
diff
changeset
|
276 def decodelistkeys(objs): |
37721
f7673845b167
wireprotov2: decode responses to their expected types
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37720
diff
changeset
|
277 # Map with bytestring keys and values. |
37725
3ea8323d6f95
wireprotov2: change command response protocol to include a leading map
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37722
diff
changeset
|
278 return next(objs) |
37721
f7673845b167
wireprotov2: decode responses to their expected types
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37720
diff
changeset
|
279 |
37725
3ea8323d6f95
wireprotov2: change command response protocol to include a leading map
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37722
diff
changeset
|
280 def decodelookup(objs): |
3ea8323d6f95
wireprotov2: change command response protocol to include a leading map
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37722
diff
changeset
|
281 return next(objs) |
37721
f7673845b167
wireprotov2: decode responses to their expected types
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37720
diff
changeset
|
282 |
37725
3ea8323d6f95
wireprotov2: change command response protocol to include a leading map
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37722
diff
changeset
|
283 def decodepushkey(objs): |
3ea8323d6f95
wireprotov2: change command response protocol to include a leading map
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37722
diff
changeset
|
284 return next(objs) |
37721
f7673845b167
wireprotov2: decode responses to their expected types
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37720
diff
changeset
|
285 |
f7673845b167
wireprotov2: decode responses to their expected types
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37720
diff
changeset
|
286 COMMAND_DECODERS = { |
f7673845b167
wireprotov2: decode responses to their expected types
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37720
diff
changeset
|
287 'branchmap': decodebranchmap, |
f7673845b167
wireprotov2: decode responses to their expected types
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37720
diff
changeset
|
288 'heads': decodeheads, |
f7673845b167
wireprotov2: decode responses to their expected types
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37720
diff
changeset
|
289 'known': decodeknown, |
f7673845b167
wireprotov2: decode responses to their expected types
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37720
diff
changeset
|
290 'listkeys': decodelistkeys, |
f7673845b167
wireprotov2: decode responses to their expected types
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37720
diff
changeset
|
291 'lookup': decodelookup, |
f7673845b167
wireprotov2: decode responses to their expected types
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37720
diff
changeset
|
292 'pushkey': decodepushkey, |
f7673845b167
wireprotov2: decode responses to their expected types
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37720
diff
changeset
|
293 } |