annotate tests/test-batching.py @ 42621:99ebde4fec99

commit: improve the files field of changelog for merges Currently, the files list of merge commits repeats all the deletions (either actual deletions, or files that got renamed) that happened between base and p2 of the merge. If p2 is the main branch, the list can easily be much bigger than the change being merged. This results in various problems worth improving: - changelog is bigger than necessary - `hg log directory` lists many unrelated merge commits, and `hg log -v -r commit` frequently fills multiple screens worth of files - it possibly slows down adjustlinkrev, by forcing it to read more manifests, and that function can certainly be a bottleneck - the server side of pulls can waste a lot of time simply opening the filelogs for pointless files (the constant factors for opening even a tiny filelog is apparently pretty bad) So stop listing such files as described in the code. Impacted merge commits and their descendants get a different hash than they would have without this. This doesn't seem problematic, except for convert. The previous commit helped with that in the hg->hg case (but if you do svn->hg twice from scratch, hashes can still change). The rest of the description is numbers. I don't have much to report, because recreating the files list of existing repositories is not easy: - debugupgradeformat and bundle/unbundle don't recreate the list - export/import tends to choke quickly applying patches or on description that contain diffs, - merge commits from the convert extension don't have the right files list for reasons orthogonal to the current commit - replaying the merge with hg update/hg merge/hg revert --all/hg commit can end up failing in hg revert - I wasn't sure that using debugsetparents + debugrebuilddirstate would really build the right thing I measured commit time before and after this change, in a case with no files filtered out, several files filtered out (no difference) and 5k files filtered out (+1% time). Recreating the 100 more recent merges in a private repo, the concatenated uncompressed files lists goes from 1.12MB to 0.52MB. Excluding 3 merges that are not representative, then the size goes from 570k to 15k. I converted part of mozilla-central, and observed file list shrinking quite a bit too, starting at the very first merge, 733641d9feaf, going from 550 files to 10 files (although they have relatively few merges, so they probably wouldn't care). Differential Revision: https://phab.mercurial-scm.org/D6613
author Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
date Tue, 02 Jul 2019 12:59:58 -0400
parents b81ca9a3f4e4
children 2372284d9457
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
14621
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
1 # test-batching.py - tests for transparent command batching
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
2 #
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
3 # Copyright 2011 Peter Arrenbrecht <peter@arrenbrecht.ch>
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
4 #
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
5 # This software may be used and distributed according to the terms of the
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
6 # GNU General Public License version 2 or any later version.
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
7
28732
e2b118592c63 py3: use print_function in test-batching.py
Robert Stanca <robert.stanca7@gmail.com>
parents: 28731
diff changeset
8 from __future__ import absolute_import, print_function
28800
544908ae36ce test-batching: stop direct symbol import of mercurial modules
Yuya Nishihara <yuya@tcha.org>
parents: 28732
diff changeset
9
37633
33a6eee08db2 wireproto: remove iterbatch() from peer interface (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
10 import contextlib
33a6eee08db2 wireproto: remove iterbatch() from peer interface (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
11
28800
544908ae36ce test-batching: stop direct symbol import of mercurial modules
Yuya Nishihara <yuya@tcha.org>
parents: 28732
diff changeset
12 from mercurial import (
37615
f3dc8239e3a9 peer: scatter module to the wind (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
13 localrepo,
41335
b81ca9a3f4e4 py3: port test-batching.py to python3
Augie Fackler <augie@google.com>
parents: 37633
diff changeset
14 pycompat,
37614
a81d02ea65db wireproto: move version 1 peer functionality to standalone module (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33766
diff changeset
15 wireprotov1peer,
41335
b81ca9a3f4e4 py3: port test-batching.py to python3
Augie Fackler <augie@google.com>
parents: 37633
diff changeset
16 )
37615
f3dc8239e3a9 peer: scatter module to the wind (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
17
41335
b81ca9a3f4e4 py3: port test-batching.py to python3
Augie Fackler <augie@google.com>
parents: 37633
diff changeset
18 def bprint(*bs):
b81ca9a3f4e4 py3: port test-batching.py to python3
Augie Fackler <augie@google.com>
parents: 37633
diff changeset
19 print(*[pycompat.sysstr(b) for b in bs])
14621
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
20
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
21 # equivalent of repo.repository
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
22 class thing(object):
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
23 def hello(self):
41335
b81ca9a3f4e4 py3: port test-batching.py to python3
Augie Fackler <augie@google.com>
parents: 37633
diff changeset
24 return b"Ready."
14621
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
25
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
26 # equivalent of localrepo.localrepository
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
27 class localthing(thing):
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
28 def foo(self, one, two=None):
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
29 if one:
41335
b81ca9a3f4e4 py3: port test-batching.py to python3
Augie Fackler <augie@google.com>
parents: 37633
diff changeset
30 return b"%s and %s" % (one, two,)
b81ca9a3f4e4 py3: port test-batching.py to python3
Augie Fackler <augie@google.com>
parents: 37633
diff changeset
31 return b"Nope"
14621
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
32 def bar(self, b, a):
41335
b81ca9a3f4e4 py3: port test-batching.py to python3
Augie Fackler <augie@google.com>
parents: 37633
diff changeset
33 return b"%s und %s" % (b, a,)
14621
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
34 def greet(self, name=None):
41335
b81ca9a3f4e4 py3: port test-batching.py to python3
Augie Fackler <augie@google.com>
parents: 37633
diff changeset
35 return b"Hello, %s" % name
37633
33a6eee08db2 wireproto: remove iterbatch() from peer interface (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
36
33a6eee08db2 wireproto: remove iterbatch() from peer interface (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
37 @contextlib.contextmanager
33a6eee08db2 wireproto: remove iterbatch() from peer interface (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
38 def commandexecutor(self):
33a6eee08db2 wireproto: remove iterbatch() from peer interface (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
39 e = localrepo.localcommandexecutor(self)
33a6eee08db2 wireproto: remove iterbatch() from peer interface (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
40 try:
33a6eee08db2 wireproto: remove iterbatch() from peer interface (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
41 yield e
33a6eee08db2 wireproto: remove iterbatch() from peer interface (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
42 finally:
33a6eee08db2 wireproto: remove iterbatch() from peer interface (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
43 e.close()
14621
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
44
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
45 # usage of "thing" interface
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
46 def use(it):
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
47
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
48 # Direct call to base method shared between client and server.
41335
b81ca9a3f4e4 py3: port test-batching.py to python3
Augie Fackler <augie@google.com>
parents: 37633
diff changeset
49 bprint(it.hello())
14621
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
50
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
51 # Direct calls to proxied methods. They cause individual roundtrips.
41335
b81ca9a3f4e4 py3: port test-batching.py to python3
Augie Fackler <augie@google.com>
parents: 37633
diff changeset
52 bprint(it.foo(b"Un", two=b"Deux"))
b81ca9a3f4e4 py3: port test-batching.py to python3
Augie Fackler <augie@google.com>
parents: 37633
diff changeset
53 bprint(it.bar(b"Eins", b"Zwei"))
14621
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
54
33766
4c706037adef wireproto: overhaul iterating batcher code (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33765
diff changeset
55 # Batched call to a couple of proxied methods.
4c706037adef wireproto: overhaul iterating batcher code (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33765
diff changeset
56
37633
33a6eee08db2 wireproto: remove iterbatch() from peer interface (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
57 with it.commandexecutor() as e:
41335
b81ca9a3f4e4 py3: port test-batching.py to python3
Augie Fackler <augie@google.com>
parents: 37633
diff changeset
58 ffoo = e.callcommand(b'foo', {b'one': b'One', b'two': b'Two'})
b81ca9a3f4e4 py3: port test-batching.py to python3
Augie Fackler <augie@google.com>
parents: 37633
diff changeset
59 fbar = e.callcommand(b'bar', {b'b': b'Eins', b'a': b'Zwei'})
b81ca9a3f4e4 py3: port test-batching.py to python3
Augie Fackler <augie@google.com>
parents: 37633
diff changeset
60 fbar2 = e.callcommand(b'bar', {b'b': b'Uno', b'a': b'Due'})
14621
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
61
41335
b81ca9a3f4e4 py3: port test-batching.py to python3
Augie Fackler <augie@google.com>
parents: 37633
diff changeset
62 bprint(ffoo.result())
b81ca9a3f4e4 py3: port test-batching.py to python3
Augie Fackler <augie@google.com>
parents: 37633
diff changeset
63 bprint(fbar.result())
b81ca9a3f4e4 py3: port test-batching.py to python3
Augie Fackler <augie@google.com>
parents: 37633
diff changeset
64 bprint(fbar2.result())
33766
4c706037adef wireproto: overhaul iterating batcher code (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33765
diff changeset
65
14621
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
66 # local usage
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
67 mylocal = localthing()
28732
e2b118592c63 py3: use print_function in test-batching.py
Robert Stanca <robert.stanca7@gmail.com>
parents: 28731
diff changeset
68 print()
41335
b81ca9a3f4e4 py3: port test-batching.py to python3
Augie Fackler <augie@google.com>
parents: 37633
diff changeset
69 bprint(b"== Local")
14621
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
70 use(mylocal)
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
71
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
72 # demo remoting; mimicks what wireproto and HTTP/SSH do
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
73
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
74 # shared
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
75
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
76 def escapearg(plain):
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
77 return (plain
41335
b81ca9a3f4e4 py3: port test-batching.py to python3
Augie Fackler <augie@google.com>
parents: 37633
diff changeset
78 .replace(b':', b'::')
b81ca9a3f4e4 py3: port test-batching.py to python3
Augie Fackler <augie@google.com>
parents: 37633
diff changeset
79 .replace(b',', b':,')
b81ca9a3f4e4 py3: port test-batching.py to python3
Augie Fackler <augie@google.com>
parents: 37633
diff changeset
80 .replace(b';', b':;')
b81ca9a3f4e4 py3: port test-batching.py to python3
Augie Fackler <augie@google.com>
parents: 37633
diff changeset
81 .replace(b'=', b':='))
14621
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
82 def unescapearg(escaped):
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
83 return (escaped
41335
b81ca9a3f4e4 py3: port test-batching.py to python3
Augie Fackler <augie@google.com>
parents: 37633
diff changeset
84 .replace(b':=', b'=')
b81ca9a3f4e4 py3: port test-batching.py to python3
Augie Fackler <augie@google.com>
parents: 37633
diff changeset
85 .replace(b':;', b';')
b81ca9a3f4e4 py3: port test-batching.py to python3
Augie Fackler <augie@google.com>
parents: 37633
diff changeset
86 .replace(b':,', b',')
b81ca9a3f4e4 py3: port test-batching.py to python3
Augie Fackler <augie@google.com>
parents: 37633
diff changeset
87 .replace(b'::', b':'))
14621
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
88
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
89 # server side
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
90
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
91 # equivalent of wireproto's global functions
14764
a7d5816087a9 classes: fix class style problems found by b071cd58af50
Thomas Arendsen Hein <thomas@intevation.de>
parents: 14621
diff changeset
92 class server(object):
14621
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
93 def __init__(self, local):
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
94 self.local = local
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
95 def _call(self, name, args):
41335
b81ca9a3f4e4 py3: port test-batching.py to python3
Augie Fackler <augie@google.com>
parents: 37633
diff changeset
96 args = dict(arg.split(b'=', 1) for arg in args)
14621
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
97 return getattr(self, name)(**args)
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
98 def perform(self, req):
41335
b81ca9a3f4e4 py3: port test-batching.py to python3
Augie Fackler <augie@google.com>
parents: 37633
diff changeset
99 bprint(b"REQ:", req)
b81ca9a3f4e4 py3: port test-batching.py to python3
Augie Fackler <augie@google.com>
parents: 37633
diff changeset
100 name, args = req.split(b'?', 1)
b81ca9a3f4e4 py3: port test-batching.py to python3
Augie Fackler <augie@google.com>
parents: 37633
diff changeset
101 args = args.split(b'&')
b81ca9a3f4e4 py3: port test-batching.py to python3
Augie Fackler <augie@google.com>
parents: 37633
diff changeset
102 vals = dict(arg.split(b'=', 1) for arg in args)
b81ca9a3f4e4 py3: port test-batching.py to python3
Augie Fackler <augie@google.com>
parents: 37633
diff changeset
103 res = getattr(self, pycompat.sysstr(name))(**pycompat.strkwargs(vals))
b81ca9a3f4e4 py3: port test-batching.py to python3
Augie Fackler <augie@google.com>
parents: 37633
diff changeset
104 bprint(b" ->", res)
14621
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
105 return res
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
106 def batch(self, cmds):
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
107 res = []
41335
b81ca9a3f4e4 py3: port test-batching.py to python3
Augie Fackler <augie@google.com>
parents: 37633
diff changeset
108 for pair in cmds.split(b';'):
b81ca9a3f4e4 py3: port test-batching.py to python3
Augie Fackler <augie@google.com>
parents: 37633
diff changeset
109 name, args = pair.split(b':', 1)
14621
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
110 vals = {}
41335
b81ca9a3f4e4 py3: port test-batching.py to python3
Augie Fackler <augie@google.com>
parents: 37633
diff changeset
111 for a in args.split(b','):
14621
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
112 if a:
41335
b81ca9a3f4e4 py3: port test-batching.py to python3
Augie Fackler <augie@google.com>
parents: 37633
diff changeset
113 n, v = a.split(b'=')
14621
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
114 vals[n] = unescapearg(v)
41335
b81ca9a3f4e4 py3: port test-batching.py to python3
Augie Fackler <augie@google.com>
parents: 37633
diff changeset
115 res.append(escapearg(getattr(self, pycompat.sysstr(name))(
b81ca9a3f4e4 py3: port test-batching.py to python3
Augie Fackler <augie@google.com>
parents: 37633
diff changeset
116 **pycompat.strkwargs(vals))))
b81ca9a3f4e4 py3: port test-batching.py to python3
Augie Fackler <augie@google.com>
parents: 37633
diff changeset
117 return b';'.join(res)
14621
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
118 def foo(self, one, two):
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
119 return mangle(self.local.foo(unmangle(one), unmangle(two)))
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
120 def bar(self, b, a):
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
121 return mangle(self.local.bar(unmangle(b), unmangle(a)))
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
122 def greet(self, name):
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
123 return mangle(self.local.greet(unmangle(name)))
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
124 myserver = server(mylocal)
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
125
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
126 # local side
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
127
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
128 # equivalent of wireproto.encode/decodelist, that is, type-specific marshalling
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
129 # here we just transform the strings a bit to check we're properly en-/decoding
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
130 def mangle(s):
41335
b81ca9a3f4e4 py3: port test-batching.py to python3
Augie Fackler <augie@google.com>
parents: 37633
diff changeset
131 return b''.join(pycompat.bytechr(ord(c) + 1) for c in pycompat.bytestr(s))
14621
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
132 def unmangle(s):
41335
b81ca9a3f4e4 py3: port test-batching.py to python3
Augie Fackler <augie@google.com>
parents: 37633
diff changeset
133 return b''.join(pycompat.bytechr(ord(c) - 1) for c in pycompat.bytestr(s))
14621
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
134
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
135 # equivalent of wireproto.wirerepository and something like http's wire format
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
136 class remotething(thing):
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
137 def __init__(self, server):
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
138 self.server = server
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
139 def _submitone(self, name, args):
41335
b81ca9a3f4e4 py3: port test-batching.py to python3
Augie Fackler <augie@google.com>
parents: 37633
diff changeset
140 req = name + b'?' + b'&'.join([b'%s=%s' % (n, v) for n, v in args])
14621
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
141 return self.server.perform(req)
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
142 def _submitbatch(self, cmds):
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
143 req = []
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
144 for name, args in cmds:
41335
b81ca9a3f4e4 py3: port test-batching.py to python3
Augie Fackler <augie@google.com>
parents: 37633
diff changeset
145 args = b','.join(n + b'=' + escapearg(v) for n, v in args)
b81ca9a3f4e4 py3: port test-batching.py to python3
Augie Fackler <augie@google.com>
parents: 37633
diff changeset
146 req.append(name + b':' + args)
b81ca9a3f4e4 py3: port test-batching.py to python3
Augie Fackler <augie@google.com>
parents: 37633
diff changeset
147 req = b';'.join(req)
b81ca9a3f4e4 py3: port test-batching.py to python3
Augie Fackler <augie@google.com>
parents: 37633
diff changeset
148 res = self._submitone(b'batch', [(b'cmds', req,)])
b81ca9a3f4e4 py3: port test-batching.py to python3
Augie Fackler <augie@google.com>
parents: 37633
diff changeset
149 for r in res.split(b';'):
33766
4c706037adef wireproto: overhaul iterating batcher code (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33765
diff changeset
150 yield r
14621
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
151
37633
33a6eee08db2 wireproto: remove iterbatch() from peer interface (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
152 @contextlib.contextmanager
33a6eee08db2 wireproto: remove iterbatch() from peer interface (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
153 def commandexecutor(self):
33a6eee08db2 wireproto: remove iterbatch() from peer interface (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
154 e = wireprotov1peer.peerexecutor(self)
33a6eee08db2 wireproto: remove iterbatch() from peer interface (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
155 try:
33a6eee08db2 wireproto: remove iterbatch() from peer interface (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
156 yield e
33a6eee08db2 wireproto: remove iterbatch() from peer interface (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
157 finally:
33a6eee08db2 wireproto: remove iterbatch() from peer interface (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
158 e.close()
14621
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
159
37615
f3dc8239e3a9 peer: scatter module to the wind (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
160 @wireprotov1peer.batchable
14621
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
161 def foo(self, one, two=None):
41335
b81ca9a3f4e4 py3: port test-batching.py to python3
Augie Fackler <augie@google.com>
parents: 37633
diff changeset
162 encargs = [(b'one', mangle(one),), (b'two', mangle(two),)]
37615
f3dc8239e3a9 peer: scatter module to the wind (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
163 encresref = wireprotov1peer.future()
14621
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
164 yield encargs, encresref
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
165 yield unmangle(encresref.value)
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
166
37615
f3dc8239e3a9 peer: scatter module to the wind (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
167 @wireprotov1peer.batchable
14621
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
168 def bar(self, b, a):
37615
f3dc8239e3a9 peer: scatter module to the wind (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
169 encresref = wireprotov1peer.future()
41335
b81ca9a3f4e4 py3: port test-batching.py to python3
Augie Fackler <augie@google.com>
parents: 37633
diff changeset
170 yield [(b'b', mangle(b),), (b'a', mangle(a),)], encresref
14621
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
171 yield unmangle(encresref.value)
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
172
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
173 # greet is coded directly. It therefore does not support batching. If it
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
174 # does appear in a batch, the batch is split around greet, and the call to
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
175 # greet is done in its own roundtrip.
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
176 def greet(self, name=None):
41335
b81ca9a3f4e4 py3: port test-batching.py to python3
Augie Fackler <augie@google.com>
parents: 37633
diff changeset
177 return unmangle(self._submitone(b'greet', [(b'name', mangle(name),)]))
14621
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
178
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
179 # demo remote usage
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
180
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
181 myproxy = remotething(myserver)
28732
e2b118592c63 py3: use print_function in test-batching.py
Robert Stanca <robert.stanca7@gmail.com>
parents: 28731
diff changeset
182 print()
41335
b81ca9a3f4e4 py3: port test-batching.py to python3
Augie Fackler <augie@google.com>
parents: 37633
diff changeset
183 bprint(b"== Remote")
14621
84094c0d2724 wireproto: add basic command batching infrastructure
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
184 use(myproxy)