annotate mercurial/peer.py @ 25912:cbbdd085c991

batching: migrate basic noop batching into peer.peer "Real" batching only makes sense for wirepeers, but it greatly simplifies the clients of peer instances if they can be ignorant to actual batching capabilities of that peer. By moving the not-really-batched batching code into peer.peer, all peer instances now work with the batching API, thus simplifying users. This leaves a couple of name forwards in wirepeer.py. Originally I had planned to clean those up, but it kind of unclarifies other bits of code that want to use batching, so I think it makes sense for the names to stay exposed by wireproto. Specifically, almost nothing is currently aware of peer (see largefiles.proto for an example), so making them be aware of the peer module *and* the wireproto module seems like some abstraction leakage. I *think* the right long-term fix would actually be to make wireproto an implementation detail that clients wouldn't need to know about, but I don't really know what that would entail at the moment. As far as I'm aware, no clients of batching in third-party extensions will need updating, which is nice icing.
author Augie Fackler <augie@google.com>
date Wed, 05 Aug 2015 14:51:34 -0400
parents 4ed6b3a24661
children e6b56b2c1f26
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
17192
1ac628cd7113 peer: introduce real peer classes
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 17191
diff changeset
1 # peer.py - repository base classes for mercurial
1089
142b5d5ec9cc Break apart hg.py
mpm@selenic.com
parents:
diff changeset
2 #
4635
63b9d2deed48 Updated copyright notices and add "and others" to "hg version"
Thomas Arendsen Hein <thomas@intevation.de>
parents: 3930
diff changeset
3 # Copyright 2005, 2006 Matt Mackall <mpm@selenic.com>
2859
345bac2bc4ec update copyrights.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2612
diff changeset
4 # Copyright 2006 Vadim Gelfer <vadim.gelfer@gmail.com>
1089
142b5d5ec9cc Break apart hg.py
mpm@selenic.com
parents:
diff changeset
5 #
8225
46293a0c7e9f updated license to be explicit about GPL version 2
Martin Geisler <mg@lazybytes.net>
parents: 7873
diff changeset
6 # This software may be used and distributed according to the terms of the
10263
25e572394f5c Update license to GPLv2+
Matt Mackall <mpm@selenic.com>
parents: 9436
diff changeset
7 # GNU General Public License version 2 or any later version.
1089
142b5d5ec9cc Break apart hg.py
mpm@selenic.com
parents:
diff changeset
8
5455
08d6e8754388 import gettext since '_' is used
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 5259
diff changeset
9 from i18n import _
7873
4a4c7f6a5912 cleanup: drop unused imports
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 7637
diff changeset
10 import error
25912
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
11 import util
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
12
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
13 # abstract batching support
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
14
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
15 class future(object):
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
16 '''placeholder for a value to be set later'''
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
17 def set(self, value):
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
18 if util.safehasattr(self, 'value'):
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
19 raise error.RepoError("future is already set")
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
20 self.value = value
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
21
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
22 class batcher(object):
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
23 '''base class for batches of commands submittable in a single request
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
24
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
25 All methods invoked on instances of this class are simply queued and
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
26 return a a future for the result. Once you call submit(), all the queued
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
27 calls are performed and the results set in their respective futures.
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
28 '''
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
29 def __init__(self):
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
30 self.calls = []
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
31 def __getattr__(self, name):
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
32 def call(*args, **opts):
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
33 resref = future()
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
34 self.calls.append((name, args, opts, resref,))
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
35 return resref
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
36 return call
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
37 def submit(self):
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
38 pass
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
39
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
40 class localbatch(batcher):
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
41 '''performs the queued calls directly'''
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
42 def __init__(self, local):
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
43 batcher.__init__(self)
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
44 self.local = local
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
45 def submit(self):
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
46 for name, args, opts, resref in self.calls:
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
47 resref.set(getattr(self.local, name)(*args, **opts))
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
48
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
49 def batchable(f):
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
50 '''annotation for batchable methods
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
51
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
52 Such methods must implement a coroutine as follows:
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
53
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
54 @batchable
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
55 def sample(self, one, two=None):
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
56 # Handle locally computable results first:
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
57 if not one:
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
58 yield "a local result", None
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
59 # Build list of encoded arguments suitable for your wire protocol:
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
60 encargs = [('one', encode(one),), ('two', encode(two),)]
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
61 # Create future for injection of encoded result:
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
62 encresref = future()
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
63 # Return encoded arguments and future:
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
64 yield encargs, encresref
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
65 # Assuming the future to be filled with the result from the batched
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
66 # request now. Decode it:
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
67 yield decode(encresref.value)
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
68
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
69 The decorator returns a function which wraps this coroutine as a plain
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
70 method, but adds the original method as an attribute called "batchable",
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
71 which is used by remotebatch to split the call into separate encoding and
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
72 decoding phases.
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
73 '''
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
74 def plain(*args, **opts):
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
75 batchable = f(*args, **opts)
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
76 encargsorres, encresref = batchable.next()
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
77 if not encresref:
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
78 return encargsorres # a local result in this case
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
79 self = args[0]
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
80 encresref.set(self._submitone(f.func_name, encargsorres))
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
81 return batchable.next()
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
82 setattr(plain, 'batchable', f)
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
83 return plain
5455
08d6e8754388 import gettext since '_' is used
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 5259
diff changeset
84
17192
1ac628cd7113 peer: introduce real peer classes
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 17191
diff changeset
85 class peerrepository(object):
1ac628cd7113 peer: introduce real peer classes
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 17191
diff changeset
86
25912
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
87 def batch(self):
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
88 return localbatch(self)
cbbdd085c991 batching: migrate basic noop batching into peer.peer
Augie Fackler <augie@google.com>
parents: 17273
diff changeset
89
2612
ffb895f16925 add support for streaming clone.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 1089
diff changeset
90 def capable(self, name):
ffb895f16925 add support for streaming clone.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 1089
diff changeset
91 '''tell whether repo supports named capability.
ffb895f16925 add support for streaming clone.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 1089
diff changeset
92 return False if not supported.
ffb895f16925 add support for streaming clone.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 1089
diff changeset
93 if boolean capability, return True.
ffb895f16925 add support for streaming clone.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 1089
diff changeset
94 if string capability, return string.'''
17192
1ac628cd7113 peer: introduce real peer classes
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 17191
diff changeset
95 caps = self._capabilities()
1ac628cd7113 peer: introduce real peer classes
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 17191
diff changeset
96 if name in caps:
5259
65dc707606ed Push capability checking into protocol-level code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 4635
diff changeset
97 return True
2612
ffb895f16925 add support for streaming clone.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 1089
diff changeset
98 name_eq = name + '='
17192
1ac628cd7113 peer: introduce real peer classes
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 17191
diff changeset
99 for cap in caps:
2612
ffb895f16925 add support for streaming clone.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 1089
diff changeset
100 if cap.startswith(name_eq):
ffb895f16925 add support for streaming clone.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 1089
diff changeset
101 return cap[len(name_eq):]
ffb895f16925 add support for streaming clone.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 1089
diff changeset
102 return False
5259
65dc707606ed Push capability checking into protocol-level code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 4635
diff changeset
103
65dc707606ed Push capability checking into protocol-level code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 4635
diff changeset
104 def requirecap(self, name, purpose):
65dc707606ed Push capability checking into protocol-level code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 4635
diff changeset
105 '''raise an exception if the given capability is not present'''
65dc707606ed Push capability checking into protocol-level code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 4635
diff changeset
106 if not self.capable(name):
7637
1d54e2f6c0b7 error: move repo errors
Matt Mackall <mpm@selenic.com>
parents: 6526
diff changeset
107 raise error.CapabilityError(
1d54e2f6c0b7 error: move repo errors
Matt Mackall <mpm@selenic.com>
parents: 6526
diff changeset
108 _('cannot %s; remote repository does not '
1d54e2f6c0b7 error: move repo errors
Matt Mackall <mpm@selenic.com>
parents: 6526
diff changeset
109 'support the %r capability') % (purpose, name))
6311
a079cf630065 Add default local() and cancopy() methods to repository base class
John Mulligan <phlogistonjohn@asynchrono.us>
parents: 5455
diff changeset
110
a079cf630065 Add default local() and cancopy() methods to repository base class
John Mulligan <phlogistonjohn@asynchrono.us>
parents: 5455
diff changeset
111 def local(self):
17192
1ac628cd7113 peer: introduce real peer classes
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 17191
diff changeset
112 '''return peer as a localrepo, or None'''
1ac628cd7113 peer: introduce real peer classes
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 17191
diff changeset
113 return None
1ac628cd7113 peer: introduce real peer classes
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 17191
diff changeset
114
1ac628cd7113 peer: introduce real peer classes
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 17191
diff changeset
115 def peer(self):
1ac628cd7113 peer: introduce real peer classes
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 17191
diff changeset
116 return self
6311
a079cf630065 Add default local() and cancopy() methods to repository base class
John Mulligan <phlogistonjohn@asynchrono.us>
parents: 5455
diff changeset
117
17193
1d710fe5ee0e peer: introduce canpush and improve error message
Sune Foldager <cryo@cyanite.org>
parents: 17192
diff changeset
118 def canpush(self):
1d710fe5ee0e peer: introduce canpush and improve error message
Sune Foldager <cryo@cyanite.org>
parents: 17192
diff changeset
119 return True
1d710fe5ee0e peer: introduce canpush and improve error message
Sune Foldager <cryo@cyanite.org>
parents: 17192
diff changeset
120
13382
d747774ca9da Make sure bundlerepo doesn't leak temp files (issue2491)
Adrian Buehlmann <adrian@cadifra.com>
parents: 12035
diff changeset
121 def close(self):
d747774ca9da Make sure bundlerepo doesn't leak temp files (issue2491)
Adrian Buehlmann <adrian@cadifra.com>
parents: 12035
diff changeset
122 pass