Mercurial > hg

comparison tests/test-wireproto-caching.t @ 40021:c537144fdbef

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
wireprotov2: support response caching One of the things I've learned from managing VCS servers over the years is that they are hard to scale. It is well known that some companies have very beefy (read: very expensive) servers to power their VCS needs. It is also known that specialized servers for various VCS exist in order to facilitate scaling servers. (Mercurial is in this boat.) One of the aspects that make a VCS server hard to scale is the high CPU load incurred by constant client clone/pull operations. To alleviate the scaling pain associated with data retrieval operations, I want to integrate caching into the Mercurial wire protocol server as robustly as possible such that servers can aggressively cache responses and defer as much server load as possible. This commit represents the initial implementation of a general caching layer in wire protocol version 2. We define a new interface and behavior for a wire protocol cacher in repository.py. (This is probably where a reviewer should look first to understand what is going on.) The bulk of the added code is in wireprotov2server.py, where we define how a command can opt in to being cached and integrate caching into command dispatching. From a very high-level: * A command can declare itself as cacheable by providing a callable that can be used to derive a cache key. * At dispatch time, if a command is cacheable, we attempt to construct a cacher and use it for serving the request and/or caching the request. * The dispatch layer handles the bulk of the business logic for caching, making cachers mostly "dumb content stores." * The mechanism for invalidating cached entries (one of the harder parts about caching in general) is by varying the cache key when state changes. As such, cachers don't need to be concerned with cache invalidation. Initially, we've hooked up support for caching "manifestdata" and "filedata" commands. These are the simplest to cache, as they should be immutable over time. Caching of commands related to changeset data is a bit harder (because cache validation is impacted by changes to bookmarks, phases, etc). This will be implemented later. (Strictly speaking, censoring a file should invalidate caches. I've added an inline TODO to track this edge case.) To prove it works, this commit implements a test-only extension providing in-memory caching backed by an lrucachedict. A new test showing this extension behaving properly is added. FWIW, the cacher is ~50 lines of code, demonstrating the relative ease with which a cache can be added to a server. While the test cacher is not suitable for production workloads, just for kicks I performed a clone of just the changeset and manifest data for the mozilla-unified repository. With a fully warmed cache (of just the manifest data since changeset data is not cached), server-side CPU usage dropped from ~73s to ~28s. That's pretty significant and demonstrates the potential that response caching has on server scalability! Differential Revision: https://phab.mercurial-scm.org/D4773
author Gregory Szorc <gregory.szorc@gmail.com>
date Wed, 26 Sep 2018 17:16:56 -0700
parents
children 393e44324037
comparison
equal deleted inserted replaced
40020:ed919b90acda 40021:c537144fdbef
1 $ . $TESTDIR/wireprotohelpers.sh
2 $ cat >> $HGRCPATH << EOF
3 > [extensions]
4 > blackbox =
5 > [blackbox]
6 > track = simplecache
7 > EOF
8 $ hg init server
9 $ enablehttpv2 server
10 $ cd server
11 $ cat >> .hg/hgrc << EOF
12 > [extensions]
13 > simplecache = $TESTDIR/wireprotosimplecache.py
14 > EOF
15
16 $ echo a0 > a
17 $ echo b0 > b
18 $ hg -q commit -A -m 'commit 0'
19 $ echo a1 > a
20 $ hg commit -m 'commit 1'
21 $ echo b1 > b
22 $ hg commit -m 'commit 2'
23 $ echo a2 > a
24 $ echo b2 > b
25 $ hg commit -m 'commit 3'
26
27 $ hg log -G -T '{rev}:{node} {desc}'
28 @ 3:50590a86f3ff5d1e9a1624a7a6957884565cc8e8 commit 3
29 |
30 o 2:4d01eda50c6ac5f7e89cbe1880143a32f559c302 commit 2
31 |
32 o 1:4432d83626e8a98655f062ec1f2a43b07f7fbbb0 commit 1
33 |
34 o 0:3390ef850073fbc2f0dfff2244342c8e9229013a commit 0
35
36
37 $ hg --debug debugindex -m
38 rev linkrev nodeid p1 p2
39 0 0 992f4779029a3df8d0666d00bb924f69634e2641 0000000000000000000000000000000000000000 0000000000000000000000000000000000000000
40 1 1 a988fb43583e871d1ed5750ee074c6d840bbbfc8 992f4779029a3df8d0666d00bb924f69634e2641 0000000000000000000000000000000000000000
41 2 2 a8853dafacfca6fc807055a660d8b835141a3bb4 a988fb43583e871d1ed5750ee074c6d840bbbfc8 0000000000000000000000000000000000000000
42 3 3 3fe11dfbb13645782b0addafbe75a87c210ffddc a8853dafacfca6fc807055a660d8b835141a3bb4 0000000000000000000000000000000000000000
43
44 $ hg serve -p $HGPORT -d --pid-file hg.pid -E error.log
45 $ cat hg.pid > $DAEMON_PIDS
46
47 Performing the same request should result in same result, with 2nd response
48 coming from cache.
49
50 $ sendhttpv2peer << EOF
51 > command manifestdata
52 > nodes eval:[b'\x99\x2f\x47\x79\x02\x9a\x3d\xf8\xd0\x66\x6d\x00\xbb\x92\x4f\x69\x63\x4e\x26\x41']
53 > tree eval:b''
54 > fields eval:[b'parents']
55 > EOF
56 creating http peer for wire protocol version 2
57 sending manifestdata command
58 s> POST /api/exp-http-v2-0002/ro/manifestdata HTTP/1.1\r\n
59 s> Accept-Encoding: identity\r\n
60 s> accept: application/mercurial-exp-framing-0005\r\n
61 s> content-type: application/mercurial-exp-framing-0005\r\n
62 s> content-length: 83\r\n
63 s> host: $LOCALIP:$HGPORT\r\n (glob)
64 s> user-agent: Mercurial debugwireproto\r\n
65 s> \r\n
66 s> K\x00\x00\x01\x00\x01\x01\x11\xa2Dargs\xa3Ffields\x81GparentsEnodes\x81T\x99/Gy\x02\x9a=\xf8\xd0fm\x00\xbb\x92OicN&ADtree@DnameLmanifestdata
67 s> makefile('rb', None)
68 s> HTTP/1.1 200 OK\r\n
69 s> Server: testing stub value\r\n
70 s> Date: $HTTP_DATE$\r\n
71 s> Content-Type: application/mercurial-exp-framing-0005\r\n
72 s> Transfer-Encoding: chunked\r\n
73 s> \r\n
74 s> 13\r\n
75 s> \x0b\x00\x00\x01\x00\x02\x011
76 s> \xa1FstatusBok
77 s> \r\n
78 received frame(size=11; request=1; stream=2; streamflags=stream-begin; type=command-response; flags=continuation)
79 s> 63\r\n
80 s> [\x00\x00\x01\x00\x02\x001
81 s> \xa1Jtotalitems\x01\xa2DnodeT\x99/Gy\x02\x9a=\xf8\xd0fm\x00\xbb\x92OicN&AGparents\x82T\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00T\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00
82 s> \r\n
83 received frame(size=91; request=1; stream=2; streamflags=; type=command-response; flags=continuation)
84 s> 8\r\n
85 s> \x00\x00\x00\x01\x00\x02\x002
86 s> \r\n
87 s> 0\r\n
88 s> \r\n
89 received frame(size=0; request=1; stream=2; streamflags=; type=command-response; flags=eos)
90 response: gen[
91 {
92 b'totalitems': 1
93 },
94 {
95 b'node': b'\x99/Gy\x02\x9a=\xf8\xd0fm\x00\xbb\x92OicN&A',
96 b'parents': [
97 b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00',
98 b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
99 ]
100 }
101 ]
102
103 $ sendhttpv2peer << EOF
104 > command manifestdata
105 > nodes eval:[b'\x99\x2f\x47\x79\x02\x9a\x3d\xf8\xd0\x66\x6d\x00\xbb\x92\x4f\x69\x63\x4e\x26\x41']
106 > tree eval:b''
107 > fields eval:[b'parents']
108 > EOF
109 creating http peer for wire protocol version 2
110 sending manifestdata command
111 s> POST /api/exp-http-v2-0002/ro/manifestdata HTTP/1.1\r\n
112 s> Accept-Encoding: identity\r\n
113 s> accept: application/mercurial-exp-framing-0005\r\n
114 s> content-type: application/mercurial-exp-framing-0005\r\n
115 s> content-length: 83\r\n
116 s> host: $LOCALIP:$HGPORT\r\n (glob)
117 s> user-agent: Mercurial debugwireproto\r\n
118 s> \r\n
119 s> K\x00\x00\x01\x00\x01\x01\x11\xa2Dargs\xa3Ffields\x81GparentsEnodes\x81T\x99/Gy\x02\x9a=\xf8\xd0fm\x00\xbb\x92OicN&ADtree@DnameLmanifestdata
120 s> makefile('rb', None)
121 s> HTTP/1.1 200 OK\r\n
122 s> Server: testing stub value\r\n
123 s> Date: $HTTP_DATE$\r\n
124 s> Content-Type: application/mercurial-exp-framing-0005\r\n
125 s> Transfer-Encoding: chunked\r\n
126 s> \r\n
127 s> 13\r\n
128 s> \x0b\x00\x00\x01\x00\x02\x011
129 s> \xa1FstatusBok
130 s> \r\n
131 received frame(size=11; request=1; stream=2; streamflags=stream-begin; type=command-response; flags=continuation)
132 s> 63\r\n
133 s> [\x00\x00\x01\x00\x02\x001
134 s> \xa1Jtotalitems\x01\xa2DnodeT\x99/Gy\x02\x9a=\xf8\xd0fm\x00\xbb\x92OicN&AGparents\x82T\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00T\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00
135 s> \r\n
136 received frame(size=91; request=1; stream=2; streamflags=; type=command-response; flags=continuation)
137 s> 8\r\n
138 s> \x00\x00\x00\x01\x00\x02\x002
139 s> \r\n
140 s> 0\r\n
141 s> \r\n
142 received frame(size=0; request=1; stream=2; streamflags=; type=command-response; flags=eos)
143 response: gen[
144 {
145 b'totalitems': 1
146 },
147 {
148 b'node': b'\x99/Gy\x02\x9a=\xf8\xd0fm\x00\xbb\x92OicN&A',
149 b'parents': [
150 b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00',
151 b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
152 ]
153 }
154 ]
155
156 Sending different request doesn't yield cache hit.
157
158 $ sendhttpv2peer << EOF
159 > command manifestdata
160 > nodes eval:[b'\x99\x2f\x47\x79\x02\x9a\x3d\xf8\xd0\x66\x6d\x00\xbb\x92\x4f\x69\x63\x4e\x26\x41', b'\xa9\x88\xfb\x43\x58\x3e\x87\x1d\x1e\xd5\x75\x0e\xe0\x74\xc6\xd8\x40\xbb\xbf\xc8']
161 > tree eval:b''
162 > fields eval:[b'parents']
163 > EOF
164 creating http peer for wire protocol version 2
165 sending manifestdata command
166 s> POST /api/exp-http-v2-0002/ro/manifestdata HTTP/1.1\r\n
167 s> Accept-Encoding: identity\r\n
168 s> accept: application/mercurial-exp-framing-0005\r\n
169 s> content-type: application/mercurial-exp-framing-0005\r\n
170 s> content-length: 104\r\n
171 s> host: $LOCALIP:$HGPORT\r\n (glob)
172 s> user-agent: Mercurial debugwireproto\r\n
173 s> \r\n
174 s> `\x00\x00\x01\x00\x01\x01\x11\xa2Dargs\xa3Ffields\x81GparentsEnodes\x82T\x99/Gy\x02\x9a=\xf8\xd0fm\x00\xbb\x92OicN&AT\xa9\x88\xfbCX>\x87\x1d\x1e\xd5u\x0e\xe0t\xc6\xd8@\xbb\xbf\xc8Dtree@DnameLmanifestdata
175 s> makefile('rb', None)
176 s> HTTP/1.1 200 OK\r\n
177 s> Server: testing stub value\r\n
178 s> Date: $HTTP_DATE$\r\n
179 s> Content-Type: application/mercurial-exp-framing-0005\r\n
180 s> Transfer-Encoding: chunked\r\n
181 s> \r\n
182 s> 13\r\n
183 s> \x0b\x00\x00\x01\x00\x02\x011
184 s> \xa1FstatusBok
185 s> \r\n
186 received frame(size=11; request=1; stream=2; streamflags=stream-begin; type=command-response; flags=continuation)
187 s> b1\r\n
188 s> \xa9\x00\x00\x01\x00\x02\x001
189 s> \xa1Jtotalitems\x02\xa2DnodeT\x99/Gy\x02\x9a=\xf8\xd0fm\x00\xbb\x92OicN&AGparents\x82T\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00T\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\xa2DnodeT\xa9\x88\xfbCX>\x87\x1d\x1e\xd5u\x0e\xe0t\xc6\xd8@\xbb\xbf\xc8Gparents\x82T\x99/Gy\x02\x9a=\xf8\xd0fm\x00\xbb\x92OicN&AT\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00
190 s> \r\n
191 received frame(size=169; request=1; stream=2; streamflags=; type=command-response; flags=continuation)
192 s> 8\r\n
193 s> \x00\x00\x00\x01\x00\x02\x002
194 s> \r\n
195 s> 0\r\n
196 s> \r\n
197 received frame(size=0; request=1; stream=2; streamflags=; type=command-response; flags=eos)
198 response: gen[
199 {
200 b'totalitems': 2
201 },
202 {
203 b'node': b'\x99/Gy\x02\x9a=\xf8\xd0fm\x00\xbb\x92OicN&A',
204 b'parents': [
205 b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00',
206 b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
207 ]
208 },
209 {
210 b'node': b'\xa9\x88\xfbCX>\x87\x1d\x1e\xd5u\x0e\xe0t\xc6\xd8@\xbb\xbf\xc8',
211 b'parents': [
212 b'\x99/Gy\x02\x9a=\xf8\xd0fm\x00\xbb\x92OicN&A',
213 b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
214 ]
215 }
216 ]
217
218 $ cat .hg/blackbox.log
219 *> cacher constructed for manifestdata (glob)
220 *> cache miss for c045a581599d58608efd3d93d8129841f2af04a0 (glob)
221 *> storing cache entry for c045a581599d58608efd3d93d8129841f2af04a0 (glob)
222 *> cacher constructed for manifestdata (glob)
223 *> cache hit for c045a581599d58608efd3d93d8129841f2af04a0 (glob)
224 *> cacher constructed for manifestdata (glob)
225 *> cache miss for 6ed2f740a1cdd12c9e99c4f27695543143c26a11 (glob)
226 *> storing cache entry for 6ed2f740a1cdd12c9e99c4f27695543143c26a11 (glob)
227
228 $ cat error.log
229
230 $ killdaemons.py
231 $ rm .hg/blackbox.log
232
233 Try with object caching mode
234
235 $ cat >> .hg/hgrc << EOF
236 > [simplecache]
237 > cacheobjects = true
238 > EOF
239
240 $ hg serve -p $HGPORT -d --pid-file hg.pid -E error.log
241 $ cat hg.pid > $DAEMON_PIDS
242
243 $ sendhttpv2peer << EOF
244 > command manifestdata
245 > nodes eval:[b'\x99\x2f\x47\x79\x02\x9a\x3d\xf8\xd0\x66\x6d\x00\xbb\x92\x4f\x69\x63\x4e\x26\x41']
246 > tree eval:b''
247 > fields eval:[b'parents']
248 > EOF
249 creating http peer for wire protocol version 2
250 sending manifestdata command
251 s> POST /api/exp-http-v2-0002/ro/manifestdata HTTP/1.1\r\n
252 s> Accept-Encoding: identity\r\n
253 s> accept: application/mercurial-exp-framing-0005\r\n
254 s> content-type: application/mercurial-exp-framing-0005\r\n
255 s> content-length: 83\r\n
256 s> host: $LOCALIP:$HGPORT\r\n (glob)
257 s> user-agent: Mercurial debugwireproto\r\n
258 s> \r\n
259 s> K\x00\x00\x01\x00\x01\x01\x11\xa2Dargs\xa3Ffields\x81GparentsEnodes\x81T\x99/Gy\x02\x9a=\xf8\xd0fm\x00\xbb\x92OicN&ADtree@DnameLmanifestdata
260 s> makefile('rb', None)
261 s> HTTP/1.1 200 OK\r\n
262 s> Server: testing stub value\r\n
263 s> Date: $HTTP_DATE$\r\n
264 s> Content-Type: application/mercurial-exp-framing-0005\r\n
265 s> Transfer-Encoding: chunked\r\n
266 s> \r\n
267 s> 13\r\n
268 s> \x0b\x00\x00\x01\x00\x02\x011
269 s> \xa1FstatusBok
270 s> \r\n
271 received frame(size=11; request=1; stream=2; streamflags=stream-begin; type=command-response; flags=continuation)
272 s> 63\r\n
273 s> [\x00\x00\x01\x00\x02\x001
274 s> \xa1Jtotalitems\x01\xa2DnodeT\x99/Gy\x02\x9a=\xf8\xd0fm\x00\xbb\x92OicN&AGparents\x82T\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00T\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00
275 s> \r\n
276 received frame(size=91; request=1; stream=2; streamflags=; type=command-response; flags=continuation)
277 s> 8\r\n
278 s> \x00\x00\x00\x01\x00\x02\x002
279 s> \r\n
280 s> 0\r\n
281 s> \r\n
282 received frame(size=0; request=1; stream=2; streamflags=; type=command-response; flags=eos)
283 response: gen[
284 {
285 b'totalitems': 1
286 },
287 {
288 b'node': b'\x99/Gy\x02\x9a=\xf8\xd0fm\x00\xbb\x92OicN&A',
289 b'parents': [
290 b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00',
291 b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
292 ]
293 }
294 ]
295
296 $ sendhttpv2peer << EOF
297 > command manifestdata
298 > nodes eval:[b'\x99\x2f\x47\x79\x02\x9a\x3d\xf8\xd0\x66\x6d\x00\xbb\x92\x4f\x69\x63\x4e\x26\x41']
299 > tree eval:b''
300 > fields eval:[b'parents']
301 > EOF
302 creating http peer for wire protocol version 2
303 sending manifestdata command
304 s> POST /api/exp-http-v2-0002/ro/manifestdata HTTP/1.1\r\n
305 s> Accept-Encoding: identity\r\n
306 s> accept: application/mercurial-exp-framing-0005\r\n
307 s> content-type: application/mercurial-exp-framing-0005\r\n
308 s> content-length: 83\r\n
309 s> host: $LOCALIP:$HGPORT\r\n (glob)
310 s> user-agent: Mercurial debugwireproto\r\n
311 s> \r\n
312 s> K\x00\x00\x01\x00\x01\x01\x11\xa2Dargs\xa3Ffields\x81GparentsEnodes\x81T\x99/Gy\x02\x9a=\xf8\xd0fm\x00\xbb\x92OicN&ADtree@DnameLmanifestdata
313 s> makefile('rb', None)
314 s> HTTP/1.1 200 OK\r\n
315 s> Server: testing stub value\r\n
316 s> Date: $HTTP_DATE$\r\n
317 s> Content-Type: application/mercurial-exp-framing-0005\r\n
318 s> Transfer-Encoding: chunked\r\n
319 s> \r\n
320 s> 13\r\n
321 s> \x0b\x00\x00\x01\x00\x02\x011
322 s> \xa1FstatusBok
323 s> \r\n
324 received frame(size=11; request=1; stream=2; streamflags=stream-begin; type=command-response; flags=continuation)
325 s> 63\r\n
326 s> [\x00\x00\x01\x00\x02\x001
327 s> \xa1Jtotalitems\x01\xa2DnodeT\x99/Gy\x02\x9a=\xf8\xd0fm\x00\xbb\x92OicN&AGparents\x82T\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00T\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00
328 s> \r\n
329 received frame(size=91; request=1; stream=2; streamflags=; type=command-response; flags=continuation)
330 s> 8\r\n
331 s> \x00\x00\x00\x01\x00\x02\x002
332 s> \r\n
333 s> 0\r\n
334 s> \r\n
335 received frame(size=0; request=1; stream=2; streamflags=; type=command-response; flags=eos)
336 response: gen[
337 {
338 b'totalitems': 1
339 },
340 {
341 b'node': b'\x99/Gy\x02\x9a=\xf8\xd0fm\x00\xbb\x92OicN&A',
342 b'parents': [
343 b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00',
344 b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
345 ]
346 }
347 ]
348
349 $ cat .hg/blackbox.log
350 *> cacher constructed for manifestdata (glob)
351 *> cache miss for c045a581599d58608efd3d93d8129841f2af04a0 (glob)
352 *> storing cache entry for c045a581599d58608efd3d93d8129841f2af04a0 (glob)
353 *> cacher constructed for manifestdata (glob)
354 *> cache hit for c045a581599d58608efd3d93d8129841f2af04a0 (glob)
355
356 $ cat error.log
357
358 $ killdaemons.py
359 $ rm .hg/blackbox.log
360
361 A non-cacheable command does not instantiate cacher
362
363 $ hg serve -p $HGPORT -d --pid-file hg.pid -E error.log
364 $ cat hg.pid > $DAEMON_PIDS
365 $ sendhttpv2peer << EOF
366 > command capabilities
367 > EOF
368 creating http peer for wire protocol version 2
369 sending capabilities command
370 s> POST /api/exp-http-v2-0002/ro/capabilities HTTP/1.1\r\n
371 s> Accept-Encoding: identity\r\n
372 s> accept: application/mercurial-exp-framing-0005\r\n
373 s> content-type: application/mercurial-exp-framing-0005\r\n
374 s> content-length: 27\r\n
375 s> host: $LOCALIP:$HGPORT\r\n (glob)
376 s> user-agent: Mercurial debugwireproto\r\n
377 s> \r\n
378 s> \x13\x00\x00\x01\x00\x01\x01\x11\xa1DnameLcapabilities
379 s> makefile('rb', None)
380 s> HTTP/1.1 200 OK\r\n
381 s> Server: testing stub value\r\n
382 s> Date: $HTTP_DATE$\r\n
383 s> Content-Type: application/mercurial-exp-framing-0005\r\n
384 s> Transfer-Encoding: chunked\r\n
385 s> \r\n
386 s> 13\r\n
387 s> \x0b\x00\x00\x01\x00\x02\x011
388 s> \xa1FstatusBok
389 s> \r\n
390 received frame(size=11; request=1; stream=2; streamflags=stream-begin; type=command-response; flags=continuation)
391 s> 52b\r\n
392 s> #\x05\x00\x01\x00\x02\x001
393 s> \xa5Hcommands\xaaIbranchmap\xa2Dargs\xa0Kpermissions\x81DpullLcapabilities\xa2Dargs\xa0Kpermissions\x81DpullMchangesetdata\xa2Dargs\xa4Ffields\xa4Gdefault\xd9\x01\x02\x80Hrequired\xf4DtypeCsetKvalidvalues\xd9\x01\x02\x84IbookmarksGparentsEphaseHrevisionInoderange\xa3Gdefault\xf6Hrequired\xf4DtypeDlistEnodes\xa3Gdefault\xf6Hrequired\xf4DtypeDlistJnodesdepth\xa3Gdefault\xf6Hrequired\xf4DtypeCintKpermissions\x81DpullHfiledata\xa2Dargs\xa4Ffields\xa4Gdefault\xd9\x01\x02\x80Hrequired\xf4DtypeCsetKvalidvalues\xd9\x01\x02\x82GparentsHrevisionKhaveparents\xa3Gdefault\xf4Hrequired\xf4DtypeDboolEnodes\xa2Hrequired\xf5DtypeDlistDpath\xa2Hrequired\xf5DtypeEbytesKpermissions\x81DpullEheads\xa2Dargs\xa1Jpubliconly\xa3Gdefault\xf4Hrequired\xf4DtypeDboolKpermissions\x81DpullEknown\xa2Dargs\xa1Enodes\xa3Gdefault\x80Hrequired\xf4DtypeDlistKpermissions\x81DpullHlistkeys\xa2Dargs\xa1Inamespace\xa2Hrequired\xf5DtypeEbytesKpermissions\x81DpullFlookup\xa2Dargs\xa1Ckey\xa2Hrequired\xf5DtypeEbytesKpermissions\x81DpullLmanifestdata\xa2Dargs\xa4Ffields\xa4Gdefault\xd9\x01\x02\x80Hrequired\xf4DtypeCsetKvalidvalues\xd9\x01\x02\x82GparentsHrevisionKhaveparents\xa3Gdefault\xf4Hrequired\xf4DtypeDboolEnodes\xa2Hrequired\xf5DtypeDlistDtree\xa2Hrequired\xf5DtypeEbytesKpermissions\x81DpullGpushkey\xa2Dargs\xa4Ckey\xa2Hrequired\xf5DtypeEbytesInamespace\xa2Hrequired\xf5DtypeEbytesCnew\xa2Hrequired\xf5DtypeEbytesCold\xa2Hrequired\xf5DtypeEbytesKpermissions\x81DpushKcompression\x82\xa1DnameDzstd\xa1DnameDzlibQframingmediatypes\x81X&application/mercurial-exp-framing-0005Rpathfilterprefixes\xd9\x01\x02\x82Epath:Lrootfilesin:Nrawrepoformats\x82LgeneraldeltaHrevlogv1
394 s> \r\n
395 received frame(size=1315; request=1; stream=2; streamflags=; type=command-response; flags=continuation)
396 s> 8\r\n
397 s> \x00\x00\x00\x01\x00\x02\x002
398 s> \r\n
399 s> 0\r\n
400 s> \r\n
401 received frame(size=0; request=1; stream=2; streamflags=; type=command-response; flags=eos)
402 response: gen[
403 {
404 b'commands': {
405 b'branchmap': {
406 b'args': {},
407 b'permissions': [
408 b'pull'
409 ]
410 },
411 b'capabilities': {
412 b'args': {},
413 b'permissions': [
414 b'pull'
415 ]
416 },
417 b'changesetdata': {
418 b'args': {
419 b'fields': {
420 b'default': set([]),
421 b'required': False,
422 b'type': b'set',
423 b'validvalues': set([
424 b'bookmarks',
425 b'parents',
426 b'phase',
427 b'revision'
428 ])
429 },
430 b'noderange': {
431 b'default': None,
432 b'required': False,
433 b'type': b'list'
434 },
435 b'nodes': {
436 b'default': None,
437 b'required': False,
438 b'type': b'list'
439 },
440 b'nodesdepth': {
441 b'default': None,
442 b'required': False,
443 b'type': b'int'
444 }
445 },
446 b'permissions': [
447 b'pull'
448 ]
449 },
450 b'filedata': {
451 b'args': {
452 b'fields': {
453 b'default': set([]),
454 b'required': False,
455 b'type': b'set',
456 b'validvalues': set([
457 b'parents',
458 b'revision'
459 ])
460 },
461 b'haveparents': {
462 b'default': False,
463 b'required': False,
464 b'type': b'bool'
465 },
466 b'nodes': {
467 b'required': True,
468 b'type': b'list'
469 },
470 b'path': {
471 b'required': True,
472 b'type': b'bytes'
473 }
474 },
475 b'permissions': [
476 b'pull'
477 ]
478 },
479 b'heads': {
480 b'args': {
481 b'publiconly': {
482 b'default': False,
483 b'required': False,
484 b'type': b'bool'
485 }
486 },
487 b'permissions': [
488 b'pull'
489 ]
490 },
491 b'known': {
492 b'args': {
493 b'nodes': {
494 b'default': [],
495 b'required': False,
496 b'type': b'list'
497 }
498 },
499 b'permissions': [
500 b'pull'
501 ]
502 },
503 b'listkeys': {
504 b'args': {
505 b'namespace': {
506 b'required': True,
507 b'type': b'bytes'
508 }
509 },
510 b'permissions': [
511 b'pull'
512 ]
513 },
514 b'lookup': {
515 b'args': {
516 b'key': {
517 b'required': True,
518 b'type': b'bytes'
519 }
520 },
521 b'permissions': [
522 b'pull'
523 ]
524 },
525 b'manifestdata': {
526 b'args': {
527 b'fields': {
528 b'default': set([]),
529 b'required': False,
530 b'type': b'set',
531 b'validvalues': set([
532 b'parents',
533 b'revision'
534 ])
535 },
536 b'haveparents': {
537 b'default': False,
538 b'required': False,
539 b'type': b'bool'
540 },
541 b'nodes': {
542 b'required': True,
543 b'type': b'list'
544 },
545 b'tree': {
546 b'required': True,
547 b'type': b'bytes'
548 }
549 },
550 b'permissions': [
551 b'pull'
552 ]
553 },
554 b'pushkey': {
555 b'args': {
556 b'key': {
557 b'required': True,
558 b'type': b'bytes'
559 },
560 b'namespace': {
561 b'required': True,
562 b'type': b'bytes'
563 },
564 b'new': {
565 b'required': True,
566 b'type': b'bytes'
567 },
568 b'old': {
569 b'required': True,
570 b'type': b'bytes'
571 }
572 },
573 b'permissions': [
574 b'push'
575 ]
576 }
577 },
578 b'compression': [
579 {
580 b'name': b'zstd'
581 },
582 {
583 b'name': b'zlib'
584 }
585 ],
586 b'framingmediatypes': [
587 b'application/mercurial-exp-framing-0005'
588 ],
589 b'pathfilterprefixes': set([
590 b'path:',
591 b'rootfilesin:'
592 ]),
593 b'rawrepoformats': [
594 b'generaldelta',
595 b'revlogv1'
596 ]
597 }
598 ]
599
600 $ test -f .hg/blackbox.log
601 [1]
602
603 An error is not cached
604
605 $ sendhttpv2peer << EOF
606 > command manifestdata
607 > nodes eval:[b'\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa']
608 > tree eval:b''
609 > fields eval:[b'parents']
610 > EOF
611 creating http peer for wire protocol version 2
612 sending manifestdata command
613 s> POST /api/exp-http-v2-0002/ro/manifestdata HTTP/1.1\r\n
614 s> Accept-Encoding: identity\r\n
615 s> accept: application/mercurial-exp-framing-0005\r\n
616 s> content-type: application/mercurial-exp-framing-0005\r\n
617 s> content-length: 83\r\n
618 s> host: $LOCALIP:$HGPORT\r\n (glob)
619 s> user-agent: Mercurial debugwireproto\r\n
620 s> \r\n
621 s> K\x00\x00\x01\x00\x01\x01\x11\xa2Dargs\xa3Ffields\x81GparentsEnodes\x81T\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaaDtree@DnameLmanifestdata
622 s> makefile('rb', None)
623 s> HTTP/1.1 200 OK\r\n
624 s> Server: testing stub value\r\n
625 s> Date: $HTTP_DATE$\r\n
626 s> Content-Type: application/mercurial-exp-framing-0005\r\n
627 s> Transfer-Encoding: chunked\r\n
628 s> \r\n
629 s> 51\r\n
630 s> I\x00\x00\x01\x00\x02\x012
631 s> \xa2Eerror\xa2Dargs\x81T\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaaGmessagePunknown node: %sFstatusEerror
632 s> \r\n
633 received frame(size=73; request=1; stream=2; streamflags=stream-begin; type=command-response; flags=eos)
634 s> 0\r\n
635 s> \r\n
636 abort: unknown node: \xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa! (esc)
637 [255]
638
639 $ cat .hg/blackbox.log
640 *> cacher constructed for manifestdata (glob)
641 *> cache miss for 9d1bb421d99e913d45f2d099aa49728514292dd2 (glob)
642 *> cacher exiting due to error (glob)
643
644 $ killdaemons.py
645 $ rm .hg/blackbox.log