annotate mercurial/wireproto.py @ 37781:352932a11905

wireproto: move command registration types to wireprototypes These are shared across wire protocol implementations. wireprototypes is our module for common code. Differential Revision: https://phab.mercurial-scm.org/D3396
author Gregory Szorc <gregory.szorc@gmail.com>
date Mon, 16 Apr 2018 21:52:33 -0700
parents 8acd3a9ac4fd
children 99accae4cc59
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
11581
4530b3307fb9 protocol: introduce wireproto.py
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
1 # wireproto.py - generic wire protocol support functions
4530b3307fb9 protocol: introduce wireproto.py
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
2 #
4530b3307fb9 protocol: introduce wireproto.py
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
3 # Copyright 2005-2010 Matt Mackall <mpm@selenic.com>
4530b3307fb9 protocol: introduce wireproto.py
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
4 #
4530b3307fb9 protocol: introduce wireproto.py
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
5 # This software may be used and distributed according to the terms of the
4530b3307fb9 protocol: introduce wireproto.py
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
6 # GNU General Public License version 2 or any later version.
4530b3307fb9 protocol: introduce wireproto.py
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
7
25993
0851678be71b wireproto: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25913
diff changeset
8 from __future__ import absolute_import
0851678be71b wireproto: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25913
diff changeset
9
0851678be71b wireproto: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25913
diff changeset
10 import os
0851678be71b wireproto: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25913
diff changeset
11 import tempfile
11581
4530b3307fb9 protocol: introduce wireproto.py
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
12
25993
0851678be71b wireproto: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25913
diff changeset
13 from .i18n import _
0851678be71b wireproto: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25913
diff changeset
14 from .node import (
0851678be71b wireproto: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25913
diff changeset
15 hex,
32300
d0d9a4fca59b clone: add a server-side option to disable full getbundles (pull-based clones)
Siddharth Agarwal <sid0@fb.com>
parents: 31460
diff changeset
16 nullid,
25993
0851678be71b wireproto: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25913
diff changeset
17 )
0851678be71b wireproto: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25913
diff changeset
18
0851678be71b wireproto: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25913
diff changeset
19 from . import (
0851678be71b wireproto: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25913
diff changeset
20 bundle2,
0851678be71b wireproto: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25913
diff changeset
21 changegroup as changegroupmod,
34112
f7d41b85bbf6 changegroup: replace changegroupsubset with makechangegroup
Durham Goode <durham@fb.com>
parents: 34079
diff changeset
22 discovery,
25993
0851678be71b wireproto: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25913
diff changeset
23 encoding,
0851678be71b wireproto: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25913
diff changeset
24 error,
0851678be71b wireproto: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25913
diff changeset
25 exchange,
0851678be71b wireproto: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25913
diff changeset
26 pushkey as pushkeymod,
30944
48dea083f66d py3: convert the mode argument of os.fdopen to unicodes (1 of 2)
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30876
diff changeset
27 pycompat,
26443
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25993
diff changeset
28 streamclone,
25993
0851678be71b wireproto: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25913
diff changeset
29 util,
36111
cd6ab329c5c7 wireprototypes: move wire protocol response types to new module
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36109
diff changeset
30 wireprototypes,
25993
0851678be71b wireproto: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25913
diff changeset
31 )
20903
8d477543882b wireproto: introduce an abstractserverproto class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20902
diff changeset
32
37087
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module
Yuya Nishihara <yuya@tcha.org>
parents: 37056
diff changeset
33 from .utils import (
37122
d4a2e0d5d042 procutil: bulk-replace util.std* to point to new module
Yuya Nishihara <yuya@tcha.org>
parents: 37087
diff changeset
34 procutil,
37087
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module
Yuya Nishihara <yuya@tcha.org>
parents: 37056
diff changeset
35 stringutil,
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module
Yuya Nishihara <yuya@tcha.org>
parents: 37056
diff changeset
36 )
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module
Yuya Nishihara <yuya@tcha.org>
parents: 37056
diff changeset
37
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28666
diff changeset
38 urlerr = util.urlerr
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28666
diff changeset
39 urlreq = util.urlreq
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28666
diff changeset
40
30871
d554e624c5fe bundle1: fix bundle1-denied reporting for push over ssh
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 30762
diff changeset
41 bundle2requiredmain = _('incompatible Mercurial client; bundle2 required')
d554e624c5fe bundle1: fix bundle1-denied reporting for push over ssh
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 30762
diff changeset
42 bundle2requiredhint = _('see https://www.mercurial-scm.org/wiki/'
d554e624c5fe bundle1: fix bundle1-denied reporting for push over ssh
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 30762
diff changeset
43 'IncompatibleClient')
d554e624c5fe bundle1: fix bundle1-denied reporting for push over ssh
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 30762
diff changeset
44 bundle2required = '%s\n(%s)\n' % (bundle2requiredmain, bundle2requiredhint)
27246
b288fb2724bf wireproto: config options to disable bundle1
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27243
diff changeset
45
37393
afcfdf53e4b5 wireproto: provide accessors for client capabilities
Joerg Sonnenberger <joerg@bec.de>
parents: 37353
diff changeset
46 def clientcompressionsupport(proto):
afcfdf53e4b5 wireproto: provide accessors for client capabilities
Joerg Sonnenberger <joerg@bec.de>
parents: 37353
diff changeset
47 """Returns a list of compression methods supported by the client.
afcfdf53e4b5 wireproto: provide accessors for client capabilities
Joerg Sonnenberger <joerg@bec.de>
parents: 37353
diff changeset
48
afcfdf53e4b5 wireproto: provide accessors for client capabilities
Joerg Sonnenberger <joerg@bec.de>
parents: 37353
diff changeset
49 Returns a list of the compression methods supported by the client
afcfdf53e4b5 wireproto: provide accessors for client capabilities
Joerg Sonnenberger <joerg@bec.de>
parents: 37353
diff changeset
50 according to the protocol capabilities. If no such capability has
afcfdf53e4b5 wireproto: provide accessors for client capabilities
Joerg Sonnenberger <joerg@bec.de>
parents: 37353
diff changeset
51 been announced, fallback to the default of zlib and uncompressed.
afcfdf53e4b5 wireproto: provide accessors for client capabilities
Joerg Sonnenberger <joerg@bec.de>
parents: 37353
diff changeset
52 """
afcfdf53e4b5 wireproto: provide accessors for client capabilities
Joerg Sonnenberger <joerg@bec.de>
parents: 37353
diff changeset
53 for cap in proto.getprotocaps():
afcfdf53e4b5 wireproto: provide accessors for client capabilities
Joerg Sonnenberger <joerg@bec.de>
parents: 37353
diff changeset
54 if cap.startswith('comp='):
afcfdf53e4b5 wireproto: provide accessors for client capabilities
Joerg Sonnenberger <joerg@bec.de>
parents: 37353
diff changeset
55 return cap[5:].split(',')
afcfdf53e4b5 wireproto: provide accessors for client capabilities
Joerg Sonnenberger <joerg@bec.de>
parents: 37353
diff changeset
56 return ['zlib', 'none']
afcfdf53e4b5 wireproto: provide accessors for client capabilities
Joerg Sonnenberger <joerg@bec.de>
parents: 37353
diff changeset
57
20902
1e4fda2f5cf1 wireproto: document possible return type
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20775
diff changeset
58 # wire protocol command can either return a string or one of these classes.
15017
f4522df38c65 wireproto: add out-of-band error class to allow remote repo to report errors
Andrew Pritchard <andrewp@fogcreek.com>
parents: 14970
diff changeset
59
29590
84c1a5942f1d wireproto: extract repo filtering to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29341
diff changeset
60 def getdispatchrepo(repo, proto, command):
84c1a5942f1d wireproto: extract repo filtering to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29341
diff changeset
61 """Obtain the repo used for processing wire protocol commands.
84c1a5942f1d wireproto: extract repo filtering to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29341
diff changeset
62
84c1a5942f1d wireproto: extract repo filtering to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29341
diff changeset
63 The intent of this function is to serve as a monkeypatch point for
84c1a5942f1d wireproto: extract repo filtering to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29341
diff changeset
64 extensions that need commands to operate on different repo views under
84c1a5942f1d wireproto: extract repo filtering to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29341
diff changeset
65 specialized circumstances.
84c1a5942f1d wireproto: extract repo filtering to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29341
diff changeset
66 """
84c1a5942f1d wireproto: extract repo filtering to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29341
diff changeset
67 return repo.filtered('served')
84c1a5942f1d wireproto: extract repo filtering to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29341
diff changeset
68
11581
4530b3307fb9 protocol: introduce wireproto.py
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
69 def dispatch(repo, proto, command):
29590
84c1a5942f1d wireproto: extract repo filtering to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29341
diff changeset
70 repo = getdispatchrepo(repo, proto, command)
37295
45b39c69fae0 wireproto: separate commands tables for version 1 and 2 commands
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37293
diff changeset
71
45b39c69fae0 wireproto: separate commands tables for version 1 and 2 commands
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37293
diff changeset
72 transportversion = wireprototypes.TRANSPORTS[proto.name]['version']
45b39c69fae0 wireproto: separate commands tables for version 1 and 2 commands
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37293
diff changeset
73 commandtable = commandsv2 if transportversion == 2 else commands
45b39c69fae0 wireproto: separate commands tables for version 1 and 2 commands
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37293
diff changeset
74 func, spec = commandtable[command]
45b39c69fae0 wireproto: separate commands tables for version 1 and 2 commands
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37293
diff changeset
75
11581
4530b3307fb9 protocol: introduce wireproto.py
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
76 args = proto.getargs(spec)
37485
0b7475ea38cf wireproto: port heads command to wire protocol v2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37414
diff changeset
77
0b7475ea38cf wireproto: port heads command to wire protocol v2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37414
diff changeset
78 # Version 1 protocols define arguments as a list. Version 2 uses a dict.
0b7475ea38cf wireproto: port heads command to wire protocol v2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37414
diff changeset
79 if isinstance(args, list):
0b7475ea38cf wireproto: port heads command to wire protocol v2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37414
diff changeset
80 return func(repo, proto, *args)
0b7475ea38cf wireproto: port heads command to wire protocol v2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37414
diff changeset
81 elif isinstance(args, dict):
0b7475ea38cf wireproto: port heads command to wire protocol v2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37414
diff changeset
82 return func(repo, proto, **args)
0b7475ea38cf wireproto: port heads command to wire protocol v2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37414
diff changeset
83 else:
0b7475ea38cf wireproto: port heads command to wire protocol v2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37414
diff changeset
84 raise error.ProgrammingError('unexpected type returned from '
0b7475ea38cf wireproto: port heads command to wire protocol v2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37414
diff changeset
85 'proto.getargs(): %s' % type(args))
11581
4530b3307fb9 protocol: introduce wireproto.py
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
86
13721
3458c15ab2f0 wireproto: fix handling of '*' args for HTTP and SSH
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13720
diff changeset
87 def options(cmd, keys, others):
3458c15ab2f0 wireproto: fix handling of '*' args for HTTP and SSH
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13720
diff changeset
88 opts = {}
3458c15ab2f0 wireproto: fix handling of '*' args for HTTP and SSH
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13720
diff changeset
89 for k in keys:
3458c15ab2f0 wireproto: fix handling of '*' args for HTTP and SSH
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13720
diff changeset
90 if k in others:
3458c15ab2f0 wireproto: fix handling of '*' args for HTTP and SSH
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13720
diff changeset
91 opts[k] = others[k]
3458c15ab2f0 wireproto: fix handling of '*' args for HTTP and SSH
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13720
diff changeset
92 del others[k]
3458c15ab2f0 wireproto: fix handling of '*' args for HTTP and SSH
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13720
diff changeset
93 if others:
37122
d4a2e0d5d042 procutil: bulk-replace util.std* to point to new module
Yuya Nishihara <yuya@tcha.org>
parents: 37087
diff changeset
94 procutil.stderr.write("warning: %s ignored unexpected arguments %s\n"
d4a2e0d5d042 procutil: bulk-replace util.std* to point to new module
Yuya Nishihara <yuya@tcha.org>
parents: 37087
diff changeset
95 % (cmd, ",".join(others)))
13721
3458c15ab2f0 wireproto: fix handling of '*' args for HTTP and SSH
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13720
diff changeset
96 return opts
3458c15ab2f0 wireproto: fix handling of '*' args for HTTP and SSH
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13720
diff changeset
97
27633
37d7cf569cf3 wireproto: support disabling bundle1 only if repo is generaldelta
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27246
diff changeset
98 def bundle1allowed(repo, action):
37d7cf569cf3 wireproto: support disabling bundle1 only if repo is generaldelta
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27246
diff changeset
99 """Whether a bundle1 operation is allowed from the server.
37d7cf569cf3 wireproto: support disabling bundle1 only if repo is generaldelta
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27246
diff changeset
100
37d7cf569cf3 wireproto: support disabling bundle1 only if repo is generaldelta
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27246
diff changeset
101 Priority is:
37d7cf569cf3 wireproto: support disabling bundle1 only if repo is generaldelta
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27246
diff changeset
102
37d7cf569cf3 wireproto: support disabling bundle1 only if repo is generaldelta
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27246
diff changeset
103 1. server.bundle1gd.<action> (if generaldelta active)
37d7cf569cf3 wireproto: support disabling bundle1 only if repo is generaldelta
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27246
diff changeset
104 2. server.bundle1.<action>
37d7cf569cf3 wireproto: support disabling bundle1 only if repo is generaldelta
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27246
diff changeset
105 3. server.bundle1gd (if generaldelta active)
37d7cf569cf3 wireproto: support disabling bundle1 only if repo is generaldelta
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27246
diff changeset
106 4. server.bundle1
37d7cf569cf3 wireproto: support disabling bundle1 only if repo is generaldelta
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27246
diff changeset
107 """
37d7cf569cf3 wireproto: support disabling bundle1 only if repo is generaldelta
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27246
diff changeset
108 ui = repo.ui
37d7cf569cf3 wireproto: support disabling bundle1 only if repo is generaldelta
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27246
diff changeset
109 gd = 'generaldelta' in repo.requirements
37d7cf569cf3 wireproto: support disabling bundle1 only if repo is generaldelta
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27246
diff changeset
110
37d7cf569cf3 wireproto: support disabling bundle1 only if repo is generaldelta
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27246
diff changeset
111 if gd:
34613
5e61cd5fb0fc configitems: register the 'server.bundle*' family of config
Boris Feld <boris.feld@octobus.net>
parents: 34329
diff changeset
112 v = ui.configbool('server', 'bundle1gd.%s' % action)
27633
37d7cf569cf3 wireproto: support disabling bundle1 only if repo is generaldelta
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27246
diff changeset
113 if v is not None:
37d7cf569cf3 wireproto: support disabling bundle1 only if repo is generaldelta
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27246
diff changeset
114 return v
37d7cf569cf3 wireproto: support disabling bundle1 only if repo is generaldelta
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27246
diff changeset
115
34613
5e61cd5fb0fc configitems: register the 'server.bundle*' family of config
Boris Feld <boris.feld@octobus.net>
parents: 34329
diff changeset
116 v = ui.configbool('server', 'bundle1.%s' % action)
27246
b288fb2724bf wireproto: config options to disable bundle1
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27243
diff changeset
117 if v is not None:
b288fb2724bf wireproto: config options to disable bundle1
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27243
diff changeset
118 return v
b288fb2724bf wireproto: config options to disable bundle1
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27243
diff changeset
119
27633
37d7cf569cf3 wireproto: support disabling bundle1 only if repo is generaldelta
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27246
diff changeset
120 if gd:
33220
40861b2254a5 configitems: register the 'server.bundle1gd' config
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 33219
diff changeset
121 v = ui.configbool('server', 'bundle1gd')
27633
37d7cf569cf3 wireproto: support disabling bundle1 only if repo is generaldelta
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27246
diff changeset
122 if v is not None:
37d7cf569cf3 wireproto: support disabling bundle1 only if repo is generaldelta
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27246
diff changeset
123 return v
37d7cf569cf3 wireproto: support disabling bundle1 only if repo is generaldelta
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27246
diff changeset
124
33219
ffb1d0f541f5 configitems: register the 'server.bundle1' config
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32898
diff changeset
125 return ui.configbool('server', 'bundle1')
27246
b288fb2724bf wireproto: config options to disable bundle1
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27243
diff changeset
126
36109
038bcb759b75 wireproto: remove unused proto argument from supportedcompengines (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36108
diff changeset
127 def supportedcompengines(ui, role):
30762
35b516f800e0 wireproto: advertise supported media types and compression formats
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30568
diff changeset
128 """Obtain the list of supported compression engines for a request."""
35b516f800e0 wireproto: advertise supported media types and compression formats
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30568
diff changeset
129 assert role in (util.CLIENTROLE, util.SERVERROLE)
35b516f800e0 wireproto: advertise supported media types and compression formats
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30568
diff changeset
130
35b516f800e0 wireproto: advertise supported media types and compression formats
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30568
diff changeset
131 compengines = util.compengines.supportedwireengines(role)
35b516f800e0 wireproto: advertise supported media types and compression formats
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30568
diff changeset
132
35b516f800e0 wireproto: advertise supported media types and compression formats
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30568
diff changeset
133 # Allow config to override default list and ordering.
35b516f800e0 wireproto: advertise supported media types and compression formats
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30568
diff changeset
134 if role == util.SERVERROLE:
35b516f800e0 wireproto: advertise supported media types and compression formats
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30568
diff changeset
135 configengines = ui.configlist('server', 'compressionengines')
35b516f800e0 wireproto: advertise supported media types and compression formats
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30568
diff changeset
136 config = 'server.compressionengines'
35b516f800e0 wireproto: advertise supported media types and compression formats
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30568
diff changeset
137 else:
35b516f800e0 wireproto: advertise supported media types and compression formats
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30568
diff changeset
138 # This is currently implemented mainly to facilitate testing. In most
35b516f800e0 wireproto: advertise supported media types and compression formats
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30568
diff changeset
139 # cases, the server should be in charge of choosing a compression engine
35b516f800e0 wireproto: advertise supported media types and compression formats
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30568
diff changeset
140 # because a server has the most to lose from a sub-optimal choice. (e.g.
35b516f800e0 wireproto: advertise supported media types and compression formats
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30568
diff changeset
141 # CPU DoS due to an expensive engine or a network DoS due to poor
35b516f800e0 wireproto: advertise supported media types and compression formats
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30568
diff changeset
142 # compression ratio).
35b516f800e0 wireproto: advertise supported media types and compression formats
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30568
diff changeset
143 configengines = ui.configlist('experimental',
35b516f800e0 wireproto: advertise supported media types and compression formats
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30568
diff changeset
144 'clientcompressionengines')
35b516f800e0 wireproto: advertise supported media types and compression formats
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30568
diff changeset
145 config = 'experimental.clientcompressionengines'
35b516f800e0 wireproto: advertise supported media types and compression formats
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30568
diff changeset
146
35b516f800e0 wireproto: advertise supported media types and compression formats
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30568
diff changeset
147 # No explicit config. Filter out the ones that aren't supposed to be
35b516f800e0 wireproto: advertise supported media types and compression formats
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30568
diff changeset
148 # advertised and return default ordering.
35b516f800e0 wireproto: advertise supported media types and compression formats
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30568
diff changeset
149 if not configengines:
35b516f800e0 wireproto: advertise supported media types and compression formats
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30568
diff changeset
150 attr = 'serverpriority' if role == util.SERVERROLE else 'clientpriority'
35b516f800e0 wireproto: advertise supported media types and compression formats
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30568
diff changeset
151 return [e for e in compengines
35b516f800e0 wireproto: advertise supported media types and compression formats
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30568
diff changeset
152 if getattr(e.wireprotosupport(), attr) > 0]
35b516f800e0 wireproto: advertise supported media types and compression formats
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30568
diff changeset
153
35b516f800e0 wireproto: advertise supported media types and compression formats
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30568
diff changeset
154 # If compression engines are listed in the config, assume there is a good
35b516f800e0 wireproto: advertise supported media types and compression formats
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30568
diff changeset
155 # reason for it (like server operators wanting to achieve specific
35b516f800e0 wireproto: advertise supported media types and compression formats
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30568
diff changeset
156 # performance characteristics). So fail fast if the config references
35b516f800e0 wireproto: advertise supported media types and compression formats
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30568
diff changeset
157 # unusable compression engines.
35b516f800e0 wireproto: advertise supported media types and compression formats
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30568
diff changeset
158 validnames = set(e.name() for e in compengines)
35b516f800e0 wireproto: advertise supported media types and compression formats
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30568
diff changeset
159 invalidnames = set(e for e in configengines if e not in validnames)
35b516f800e0 wireproto: advertise supported media types and compression formats
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30568
diff changeset
160 if invalidnames:
35b516f800e0 wireproto: advertise supported media types and compression formats
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30568
diff changeset
161 raise error.Abort(_('invalid compression engine defined in %s: %s') %
35b516f800e0 wireproto: advertise supported media types and compression formats
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30568
diff changeset
162 (config, ', '.join(sorted(invalidnames))))
35b516f800e0 wireproto: advertise supported media types and compression formats
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30568
diff changeset
163
35b516f800e0 wireproto: advertise supported media types and compression formats
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30568
diff changeset
164 compengines = [e for e in compengines if e.name() in configengines]
35b516f800e0 wireproto: advertise supported media types and compression formats
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30568
diff changeset
165 compengines = sorted(compengines,
35b516f800e0 wireproto: advertise supported media types and compression formats
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30568
diff changeset
166 key=lambda e: configengines.index(e.name()))
35b516f800e0 wireproto: advertise supported media types and compression formats
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30568
diff changeset
167
35b516f800e0 wireproto: advertise supported media types and compression formats
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30568
diff changeset
168 if not compengines:
35b516f800e0 wireproto: advertise supported media types and compression formats
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30568
diff changeset
169 raise error.Abort(_('%s config option does not specify any known '
35b516f800e0 wireproto: advertise supported media types and compression formats
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30568
diff changeset
170 'compression engines') % config,
35b516f800e0 wireproto: advertise supported media types and compression formats
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30568
diff changeset
171 hint=_('usable compression engines: %s') %
35b516f800e0 wireproto: advertise supported media types and compression formats
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30568
diff changeset
172 ', '.sorted(validnames))
35b516f800e0 wireproto: advertise supported media types and compression formats
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30568
diff changeset
173
35b516f800e0 wireproto: advertise supported media types and compression formats
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30568
diff changeset
174 return compengines
35b516f800e0 wireproto: advertise supported media types and compression formats
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30568
diff changeset
175
37295
45b39c69fae0 wireproto: separate commands tables for version 1 and 2 commands
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37293
diff changeset
176 # For version 1 transports.
37781
352932a11905 wireproto: move command registration types to wireprototypes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37780
diff changeset
177 commands = wireprototypes.commanddict()
20906
7a634b34fc91 wireproto: add decorator for wire protocol command
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20905
diff changeset
178
37295
45b39c69fae0 wireproto: separate commands tables for version 1 and 2 commands
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37293
diff changeset
179 # For version 2 transports.
37781
352932a11905 wireproto: move command registration types to wireprototypes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37780
diff changeset
180 commandsv2 = wireprototypes.commanddict()
37295
45b39c69fae0 wireproto: separate commands tables for version 1 and 2 commands
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37293
diff changeset
181
37780
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37779
diff changeset
182 def wireprotocommand(name, args=None, permission='push'):
36019
b4976912a6ef wireproto: improve docstring for @wireprotocommand
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36018
diff changeset
183 """Decorator to declare a wire protocol command.
b4976912a6ef wireproto: improve docstring for @wireprotocommand
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36018
diff changeset
184
b4976912a6ef wireproto: improve docstring for @wireprotocommand
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36018
diff changeset
185 ``name`` is the name of the wire protocol command being provided.
b4976912a6ef wireproto: improve docstring for @wireprotocommand
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36018
diff changeset
186
37535
69e46c1834ac wireproto: define and expose types of wire command arguments
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37534
diff changeset
187 ``args`` defines the named arguments accepted by the command. It is
37780
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37779
diff changeset
188 a space-delimited list of argument names. ``*`` denotes a special value
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37779
diff changeset
189 that says to accept all named arguments.
36808
0b18604db95e wireproto: declare permissions requirements in @wireprotocommand (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36768
diff changeset
190
0b18604db95e wireproto: declare permissions requirements in @wireprotocommand (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36768
diff changeset
191 ``permission`` defines the permission type needed to run this command.
0b18604db95e wireproto: declare permissions requirements in @wireprotocommand (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36768
diff changeset
192 Can be ``push`` or ``pull``. These roughly map to read-write and read-only,
0b18604db95e wireproto: declare permissions requirements in @wireprotocommand (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36768
diff changeset
193 respectively. Default is to assume command requires ``push`` permissions
0b18604db95e wireproto: declare permissions requirements in @wireprotocommand (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36768
diff changeset
194 because otherwise commands not declaring their permissions could modify
0b18604db95e wireproto: declare permissions requirements in @wireprotocommand (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36768
diff changeset
195 a repository that is supposed to be read-only.
36019
b4976912a6ef wireproto: improve docstring for @wireprotocommand
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36018
diff changeset
196 """
37780
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37779
diff changeset
197 transports = {k for k, v in wireprototypes.TRANSPORTS.items()
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37779
diff changeset
198 if v['version'] == 1}
36638
abc3b9801563 wireproto: allow wire protocol commands to declare transport support
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36564
diff changeset
199
37056
cd0ca979a8b8 wireproto: nominally don't expose "batch" to version 2 wire transports
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36994
diff changeset
200 # Because SSHv2 is a mirror of SSHv1, we allow "batch" commands through to
cd0ca979a8b8 wireproto: nominally don't expose "batch" to version 2 wire transports
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36994
diff changeset
201 # SSHv2.
cd0ca979a8b8 wireproto: nominally don't expose "batch" to version 2 wire transports
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36994
diff changeset
202 # TODO undo this hack when SSH is using the unified frame protocol.
cd0ca979a8b8 wireproto: nominally don't expose "batch" to version 2 wire transports
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36994
diff changeset
203 if name == b'batch':
cd0ca979a8b8 wireproto: nominally don't expose "batch" to version 2 wire transports
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36994
diff changeset
204 transports.add(wireprototypes.SSHV2)
cd0ca979a8b8 wireproto: nominally don't expose "batch" to version 2 wire transports
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36994
diff changeset
205
36808
0b18604db95e wireproto: declare permissions requirements in @wireprotocommand (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36768
diff changeset
206 if permission not in ('push', 'pull'):
36848
ef68493d652b wireproto: raise ProgrammingError instead of Abort
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36843
diff changeset
207 raise error.ProgrammingError('invalid wire protocol permission; '
ef68493d652b wireproto: raise ProgrammingError instead of Abort
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36843
diff changeset
208 'got %s; expected "push" or "pull"' %
ef68493d652b wireproto: raise ProgrammingError instead of Abort
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36843
diff changeset
209 permission)
36808
0b18604db95e wireproto: declare permissions requirements in @wireprotocommand (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36768
diff changeset
210
37780
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37779
diff changeset
211 if args is None:
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37779
diff changeset
212 args = ''
37535
69e46c1834ac wireproto: define and expose types of wire command arguments
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37534
diff changeset
213
37780
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37779
diff changeset
214 if not isinstance(args, bytes):
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37779
diff changeset
215 raise error.ProgrammingError('arguments for version 1 commands '
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37779
diff changeset
216 'must be declared as bytes')
37535
69e46c1834ac wireproto: define and expose types of wire command arguments
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37534
diff changeset
217
20906
7a634b34fc91 wireproto: add decorator for wire protocol command
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20905
diff changeset
218 def register(func):
37780
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37779
diff changeset
219 if name in commands:
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37779
diff changeset
220 raise error.ProgrammingError('%s command already registered '
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37779
diff changeset
221 'for version 1' % name)
37781
352932a11905 wireproto: move command registration types to wireprototypes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37780
diff changeset
222 commands[name] = wireprototypes.commandentry(
352932a11905 wireproto: move command registration types to wireprototypes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37780
diff changeset
223 func, args=args, transports=transports, permission=permission)
37295
45b39c69fae0 wireproto: separate commands tables for version 1 and 2 commands
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37293
diff changeset
224
20906
7a634b34fc91 wireproto: add decorator for wire protocol command
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20905
diff changeset
225 return func
7a634b34fc91 wireproto: add decorator for wire protocol command
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20905
diff changeset
226 return register
7a634b34fc91 wireproto: add decorator for wire protocol command
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20905
diff changeset
227
35848
ff4bc0ab6740 wireproto: check permissions when executing "batch" command (BC) (SEC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35847
diff changeset
228 # TODO define a more appropriate permissions type to use for this.
37779
379d54eae6eb wireproto: don't pass transportpolicy argument
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
229 @wireprotocommand('batch', 'cmds *', permission='pull')
14622
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14621
diff changeset
230 def batch(repo, proto, cmds, others):
37612
5e71dea79aae wireproto: move value encoding functions to wireprototypes (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37546
diff changeset
231 unescapearg = wireprototypes.unescapebatcharg
18382
f3b21beb9802 filtering: rename filters to their antonyms
Kevin Bullock <kbullock@ringworld.org>
parents: 18381
diff changeset
232 repo = repo.filtered("served")
14622
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14621
diff changeset
233 res = []
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14621
diff changeset
234 for pair in cmds.split(';'):
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14621
diff changeset
235 op, args = pair.split(' ', 1)
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14621
diff changeset
236 vals = {}
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14621
diff changeset
237 for a in args.split(','):
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14621
diff changeset
238 if a:
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14621
diff changeset
239 n, v = a.split('=')
29746
62e2e048d068 wireproto: unescape argument names in batch command (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29745
diff changeset
240 vals[unescapearg(n)] = unescapearg(v)
14622
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14621
diff changeset
241 func, spec = commands[op]
35848
ff4bc0ab6740 wireproto: check permissions when executing "batch" command (BC) (SEC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35847
diff changeset
242
36809
66de4555cefd wireproto: formalize permissions checking as part of protocol interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36808
diff changeset
243 # Validate that client has permissions to perform this command.
66de4555cefd wireproto: formalize permissions checking as part of protocol interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36808
diff changeset
244 perm = commands[op].permission
66de4555cefd wireproto: formalize permissions checking as part of protocol interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36808
diff changeset
245 assert perm in ('push', 'pull')
66de4555cefd wireproto: formalize permissions checking as part of protocol interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36808
diff changeset
246 proto.checkperm(perm)
35848
ff4bc0ab6740 wireproto: check permissions when executing "batch" command (BC) (SEC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35847
diff changeset
247
14622
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14621
diff changeset
248 if spec:
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14621
diff changeset
249 keys = spec.split()
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14621
diff changeset
250 data = {}
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14621
diff changeset
251 for k in keys:
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14621
diff changeset
252 if k == '*':
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14621
diff changeset
253 star = {}
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14621
diff changeset
254 for key in vals.keys():
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14621
diff changeset
255 if key not in keys:
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14621
diff changeset
256 star[key] = vals[key]
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14621
diff changeset
257 data['*'] = star
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14621
diff changeset
258 else:
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14621
diff changeset
259 data[k] = vals[k]
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14621
diff changeset
260 result = func(repo, proto, *[data[k] for k in keys])
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14621
diff changeset
261 else:
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14621
diff changeset
262 result = func(repo, proto)
37293
d5d665f6615a wireproto: stop aliasing wire protocol types (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37122
diff changeset
263 if isinstance(result, wireprototypes.ooberror):
15017
f4522df38c65 wireproto: add out-of-band error class to allow remote repo to report errors
Andrew Pritchard <andrewp@fogcreek.com>
parents: 14970
diff changeset
264 return result
36112
2f7290555c96 wireproto: introduce type for raw byte responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36111
diff changeset
265
2f7290555c96 wireproto: introduce type for raw byte responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36111
diff changeset
266 # For now, all batchable commands must return bytesresponse or
2f7290555c96 wireproto: introduce type for raw byte responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36111
diff changeset
267 # raw bytes (for backwards compatibility).
37293
d5d665f6615a wireproto: stop aliasing wire protocol types (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37122
diff changeset
268 assert isinstance(result, (wireprototypes.bytesresponse, bytes))
d5d665f6615a wireproto: stop aliasing wire protocol types (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37122
diff changeset
269 if isinstance(result, wireprototypes.bytesresponse):
36112
2f7290555c96 wireproto: introduce type for raw byte responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36111
diff changeset
270 result = result.data
37612
5e71dea79aae wireproto: move value encoding functions to wireprototypes (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37546
diff changeset
271 res.append(wireprototypes.escapebatcharg(result))
36112
2f7290555c96 wireproto: introduce type for raw byte responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36111
diff changeset
272
37293
d5d665f6615a wireproto: stop aliasing wire protocol types (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37122
diff changeset
273 return wireprototypes.bytesresponse(';'.join(res))
14622
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14621
diff changeset
274
37779
379d54eae6eb wireproto: don't pass transportpolicy argument
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
275 @wireprotocommand('between', 'pairs', permission='pull')
11583
944c23762c3c protocol: add proto to method prototypes
Matt Mackall <mpm@selenic.com>
parents: 11581
diff changeset
276 def between(repo, proto, pairs):
37612
5e71dea79aae wireproto: move value encoding functions to wireprototypes (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37546
diff changeset
277 pairs = [wireprototypes.decodelist(p, '-') for p in pairs.split(" ")]
11581
4530b3307fb9 protocol: introduce wireproto.py
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
278 r = []
4530b3307fb9 protocol: introduce wireproto.py
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
279 for b in repo.between(pairs):
37612
5e71dea79aae wireproto: move value encoding functions to wireprototypes (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37546
diff changeset
280 r.append(wireprototypes.encodelist(b) + "\n")
36112
2f7290555c96 wireproto: introduce type for raw byte responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36111
diff changeset
281
37293
d5d665f6615a wireproto: stop aliasing wire protocol types (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37122
diff changeset
282 return wireprototypes.bytesresponse(''.join(r))
11581
4530b3307fb9 protocol: introduce wireproto.py
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
283
37779
379d54eae6eb wireproto: don't pass transportpolicy argument
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
284 @wireprotocommand('branchmap', permission='pull')
11583
944c23762c3c protocol: add proto to method prototypes
Matt Mackall <mpm@selenic.com>
parents: 11581
diff changeset
285 def branchmap(repo, proto):
18281
898c575833c9 clfilter: drop extra filtering in wireprotocol
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 18280
diff changeset
286 branchmap = repo.branchmap()
11581
4530b3307fb9 protocol: introduce wireproto.py
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
287 heads = []
4530b3307fb9 protocol: introduce wireproto.py
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
288 for branch, nodes in branchmap.iteritems():
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28666
diff changeset
289 branchname = urlreq.quote(encoding.fromlocal(branch))
37612
5e71dea79aae wireproto: move value encoding functions to wireprototypes (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37546
diff changeset
290 branchnodes = wireprototypes.encodelist(nodes)
11597
9141d2c9e5a5 wireproto: refactor list of nodeid encoding / decoding
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 11594
diff changeset
291 heads.append('%s %s' % (branchname, branchnodes))
36112
2f7290555c96 wireproto: introduce type for raw byte responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36111
diff changeset
292
37293
d5d665f6615a wireproto: stop aliasing wire protocol types (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37122
diff changeset
293 return wireprototypes.bytesresponse('\n'.join(heads))
11581
4530b3307fb9 protocol: introduce wireproto.py
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
294
37779
379d54eae6eb wireproto: don't pass transportpolicy argument
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
295 @wireprotocommand('branches', 'nodes', permission='pull')
11583
944c23762c3c protocol: add proto to method prototypes
Matt Mackall <mpm@selenic.com>
parents: 11581
diff changeset
296 def branches(repo, proto, nodes):
37612
5e71dea79aae wireproto: move value encoding functions to wireprototypes (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37546
diff changeset
297 nodes = wireprototypes.decodelist(nodes)
11581
4530b3307fb9 protocol: introduce wireproto.py
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
298 r = []
4530b3307fb9 protocol: introduce wireproto.py
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
299 for b in repo.branches(nodes):
37612
5e71dea79aae wireproto: move value encoding functions to wireprototypes (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37546
diff changeset
300 r.append(wireprototypes.encodelist(b) + "\n")
36112
2f7290555c96 wireproto: introduce type for raw byte responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36111
diff changeset
301
37293
d5d665f6615a wireproto: stop aliasing wire protocol types (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37122
diff changeset
302 return wireprototypes.bytesresponse(''.join(r))
11581
4530b3307fb9 protocol: introduce wireproto.py
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
303
37779
379d54eae6eb wireproto: don't pass transportpolicy argument
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
304 @wireprotocommand('clonebundles', '', permission='pull')
26857
e5a1df51bb25 wireproto: move clonebundles command from extension (issue4931)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26690
diff changeset
305 def clonebundles(repo, proto):
e5a1df51bb25 wireproto: move clonebundles command from extension (issue4931)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26690
diff changeset
306 """Server command for returning info for available bundles to seed clones.
e5a1df51bb25 wireproto: move clonebundles command from extension (issue4931)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26690
diff changeset
307
e5a1df51bb25 wireproto: move clonebundles command from extension (issue4931)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26690
diff changeset
308 Clients will parse this response and determine what bundle to fetch.
e5a1df51bb25 wireproto: move clonebundles command from extension (issue4931)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26690
diff changeset
309
e5a1df51bb25 wireproto: move clonebundles command from extension (issue4931)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26690
diff changeset
310 Extensions may wrap this command to filter or dynamically emit data
e5a1df51bb25 wireproto: move clonebundles command from extension (issue4931)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26690
diff changeset
311 depending on the request. e.g. you could advertise URLs for the closest
e5a1df51bb25 wireproto: move clonebundles command from extension (issue4931)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26690
diff changeset
312 data center given the client's IP address.
e5a1df51bb25 wireproto: move clonebundles command from extension (issue4931)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26690
diff changeset
313 """
37293
d5d665f6615a wireproto: stop aliasing wire protocol types (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37122
diff changeset
314 return wireprototypes.bytesresponse(
d5d665f6615a wireproto: stop aliasing wire protocol types (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37122
diff changeset
315 repo.vfs.tryread('clonebundles.manifest'))
20774
cdc3ac896997 wireproto: extract capabilities list in outside the wireproto function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20671
diff changeset
316
36641
e89959970a08 wireproto: don't expose changegroupsubset capability if not available
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36640
diff changeset
317 wireprotocaps = ['lookup', 'branchmap', 'pushkey',
37056
cd0ca979a8b8 wireproto: nominally don't expose "batch" to version 2 wire transports
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36994
diff changeset
318 'known', 'getbundle', 'unbundlehash']
20775
982f13bef503 wireproto: move wireproto capabilities computation in a subfunction
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20774
diff changeset
319
982f13bef503 wireproto: move wireproto capabilities computation in a subfunction
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20774
diff changeset
320 def _capabilities(repo, proto):
982f13bef503 wireproto: move wireproto capabilities computation in a subfunction
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20774
diff changeset
321 """return a list of capabilities for a repo
982f13bef503 wireproto: move wireproto capabilities computation in a subfunction
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20774
diff changeset
322
982f13bef503 wireproto: move wireproto capabilities computation in a subfunction
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20774
diff changeset
323 This function exists to allow extensions to easily wrap capabilities
982f13bef503 wireproto: move wireproto capabilities computation in a subfunction
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20774
diff changeset
324 computation
982f13bef503 wireproto: move wireproto capabilities computation in a subfunction
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20774
diff changeset
325
982f13bef503 wireproto: move wireproto capabilities computation in a subfunction
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20774
diff changeset
326 - returns a lists: easy to alter
982f13bef503 wireproto: move wireproto capabilities computation in a subfunction
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20774
diff changeset
327 - change done here will be propagated to both `capabilities` and `hello`
21024
7731a2281cf0 spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents: 20969
diff changeset
328 command without any other action needed.
20775
982f13bef503 wireproto: move wireproto capabilities computation in a subfunction
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20774
diff changeset
329 """
20774
cdc3ac896997 wireproto: extract capabilities list in outside the wireproto function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20671
diff changeset
330 # copy to prevent modification of the global list
cdc3ac896997 wireproto: extract capabilities list in outside the wireproto function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20671
diff changeset
331 caps = list(wireprotocaps)
36641
e89959970a08 wireproto: don't expose changegroupsubset capability if not available
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36640
diff changeset
332
e89959970a08 wireproto: don't expose changegroupsubset capability if not available
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36640
diff changeset
333 # Command of same name as capability isn't exposed to version 1 of
e89959970a08 wireproto: don't expose changegroupsubset capability if not available
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36640
diff changeset
334 # transports. So conditionally add it.
e89959970a08 wireproto: don't expose changegroupsubset capability if not available
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36640
diff changeset
335 if commands.commandavailable('changegroupsubset', proto):
e89959970a08 wireproto: don't expose changegroupsubset capability if not available
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36640
diff changeset
336 caps.append('changegroupsubset')
e89959970a08 wireproto: don't expose changegroupsubset capability if not available
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36640
diff changeset
337
32764
33b7283a3828 streamclone: consider secret changesets (BC) (issue5589)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32331
diff changeset
338 if streamclone.allowservergeneration(repo):
33225
90a1b62bdc91 configitems: register the 'server.preferuncompressed' config
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 33224
diff changeset
339 if repo.ui.configbool('server', 'preferuncompressed'):
16361
6097ede2be4d protocol: Add the stream-preferred capability
Benoit Allard <benoit@aeteurope.nl>
parents: 15925
diff changeset
340 caps.append('stream-preferred')
12296
d7fff529d85d clone: only use stream when we understand the revlog format
Sune Foldager <cryo@cyanite.org>
parents: 12085
diff changeset
341 requiredformats = repo.requirements & repo.supportedformats
d7fff529d85d clone: only use stream when we understand the revlog format
Sune Foldager <cryo@cyanite.org>
parents: 12085
diff changeset
342 # if our local revlogs are just revlogv1, add 'stream' cap
32331
bd872f64a8ba cleanup: use set literals
Martin von Zweigbergk <martinvonz@google.com>
parents: 32300
diff changeset
343 if not requiredformats - {'revlogv1'}:
12296
d7fff529d85d clone: only use stream when we understand the revlog format
Sune Foldager <cryo@cyanite.org>
parents: 12085
diff changeset
344 caps.append('stream')
d7fff529d85d clone: only use stream when we understand the revlog format
Sune Foldager <cryo@cyanite.org>
parents: 12085
diff changeset
345 # otherwise, add 'streamreqs' detailing our local revlog format
d7fff529d85d clone: only use stream when we understand the revlog format
Sune Foldager <cryo@cyanite.org>
parents: 12085
diff changeset
346 else:
26911
d7e5e4da8394 stream: sort stream capability before serialisation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26857
diff changeset
347 caps.append('streamreqs=%s' % ','.join(sorted(requiredformats)))
33499
0407a51b9d8c codemod: register core configitems using a script
Jun Wu <quark@fb.com>
parents: 33225
diff changeset
348 if repo.ui.configbool('experimental', 'bundle2-advertise'):
35783
c97639ad6874 bundle2: specify what capabilities will be used for
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35782
diff changeset
349 capsblob = bundle2.encodecaps(bundle2.getrepocaps(repo, role='server'))
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28666
diff changeset
350 caps.append('bundle2=' + urlreq.quote(capsblob))
28666
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 28530
diff changeset
351 caps.append('unbundle=%s' % ','.join(bundle2.bundlepriority))
30568
e118233172fe wireproto: only advertise HTTP-specific capabilities to HTTP peers (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30482
diff changeset
352
36642
6e585bca962e wireproto: add transport specific capabilities in the transport
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36641
diff changeset
353 return proto.addcapabilities(repo, caps)
20775
982f13bef503 wireproto: move wireproto capabilities computation in a subfunction
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20774
diff changeset
354
21024
7731a2281cf0 spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents: 20969
diff changeset
355 # If you are writing an extension and consider wrapping this function. Wrap
20775
982f13bef503 wireproto: move wireproto capabilities computation in a subfunction
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20774
diff changeset
356 # `_capabilities` instead.
37779
379d54eae6eb wireproto: don't pass transportpolicy argument
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
357 @wireprotocommand('capabilities', permission='pull')
20775
982f13bef503 wireproto: move wireproto capabilities computation in a subfunction
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20774
diff changeset
358 def capabilities(repo, proto):
37413
33af46d639b4 wireproto: send server capabilities in canonical order
Joerg Sonnenberger <joerg@bec.de>
parents: 37412
diff changeset
359 caps = _capabilities(repo, proto)
33af46d639b4 wireproto: send server capabilities in canonical order
Joerg Sonnenberger <joerg@bec.de>
parents: 37412
diff changeset
360 return wireprototypes.bytesresponse(' '.join(sorted(caps)))
11594
67863f9d805f protocol: unify server-side capabilities functions
Matt Mackall <mpm@selenic.com>
parents: 11593
diff changeset
361
37779
379d54eae6eb wireproto: don't pass transportpolicy argument
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
362 @wireprotocommand('changegroup', 'roots', permission='pull')
11584
1af96b090116 protocol: unify changegroup commands
Matt Mackall <mpm@selenic.com>
parents: 11583
diff changeset
363 def changegroup(repo, proto, roots):
37612
5e71dea79aae wireproto: move value encoding functions to wireprototypes (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37546
diff changeset
364 nodes = wireprototypes.decodelist(roots)
34115
1632999d4bed changegroup: replace changegroup with makechangegroup
Durham Goode <durham@fb.com>
parents: 34112
diff changeset
365 outgoing = discovery.outgoing(repo, missingroots=nodes,
1632999d4bed changegroup: replace changegroup with makechangegroup
Durham Goode <durham@fb.com>
parents: 34112
diff changeset
366 missingheads=repo.heads())
1632999d4bed changegroup: replace changegroup with makechangegroup
Durham Goode <durham@fb.com>
parents: 34112
diff changeset
367 cg = changegroupmod.makechangegroup(repo, outgoing, '01', 'serve')
35705
8cdb671dbd0b wireproto: drop support for reader interface from streamres (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35495
diff changeset
368 gen = iter(lambda: cg.read(32768), '')
37293
d5d665f6615a wireproto: stop aliasing wire protocol types (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37122
diff changeset
369 return wireprototypes.streamres(gen=gen)
11584
1af96b090116 protocol: unify changegroup commands
Matt Mackall <mpm@selenic.com>
parents: 11583
diff changeset
370
36640
6906547c8476 wireproto: don't expose legacy commands to version 2 of wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36638
diff changeset
371 @wireprotocommand('changegroupsubset', 'bases heads',
36808
0b18604db95e wireproto: declare permissions requirements in @wireprotocommand (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36768
diff changeset
372 permission='pull')
11584
1af96b090116 protocol: unify changegroup commands
Matt Mackall <mpm@selenic.com>
parents: 11583
diff changeset
373 def changegroupsubset(repo, proto, bases, heads):
37612
5e71dea79aae wireproto: move value encoding functions to wireprototypes (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37546
diff changeset
374 bases = wireprototypes.decodelist(bases)
5e71dea79aae wireproto: move value encoding functions to wireprototypes (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37546
diff changeset
375 heads = wireprototypes.decodelist(heads)
34112
f7d41b85bbf6 changegroup: replace changegroupsubset with makechangegroup
Durham Goode <durham@fb.com>
parents: 34079
diff changeset
376 outgoing = discovery.outgoing(repo, missingroots=bases,
f7d41b85bbf6 changegroup: replace changegroupsubset with makechangegroup
Durham Goode <durham@fb.com>
parents: 34079
diff changeset
377 missingheads=heads)
f7d41b85bbf6 changegroup: replace changegroupsubset with makechangegroup
Durham Goode <durham@fb.com>
parents: 34079
diff changeset
378 cg = changegroupmod.makechangegroup(repo, outgoing, '01', 'serve')
35705
8cdb671dbd0b wireproto: drop support for reader interface from streamres (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35495
diff changeset
379 gen = iter(lambda: cg.read(32768), '')
37293
d5d665f6615a wireproto: stop aliasing wire protocol types (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37122
diff changeset
380 return wireprototypes.streamres(gen=gen)
11584
1af96b090116 protocol: unify changegroup commands
Matt Mackall <mpm@selenic.com>
parents: 11583
diff changeset
381
36808
0b18604db95e wireproto: declare permissions requirements in @wireprotocommand (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36768
diff changeset
382 @wireprotocommand('debugwireargs', 'one two *',
37779
379d54eae6eb wireproto: don't pass transportpolicy argument
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
383 permission='pull')
13721
3458c15ab2f0 wireproto: fix handling of '*' args for HTTP and SSH
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13720
diff changeset
384 def debugwireargs(repo, proto, one, two, others):
3458c15ab2f0 wireproto: fix handling of '*' args for HTTP and SSH
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13720
diff changeset
385 # only accept optional args from the known set
3458c15ab2f0 wireproto: fix handling of '*' args for HTTP and SSH
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13720
diff changeset
386 opts = options('debugwireargs', ['three', 'four'], others)
37293
d5d665f6615a wireproto: stop aliasing wire protocol types (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37122
diff changeset
387 return wireprototypes.bytesresponse(repo.debugwireargs(
d5d665f6615a wireproto: stop aliasing wire protocol types (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37122
diff changeset
388 one, two, **pycompat.strkwargs(opts)))
13720
9c4e04fe267e debug: add debugwireargs to test argument passing over the wire
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13450
diff changeset
389
37498
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
390 def find_pullbundle(repo, proto, opts, clheads, heads, common):
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
391 """Return a file object for the first matching pullbundle.
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
392
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
393 Pullbundles are specified in .hg/pullbundles.manifest similar to
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
394 clonebundles.
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
395 For each entry, the bundle specification is checked for compatibility:
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
396 - Client features vs the BUNDLESPEC.
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
397 - Revisions shared with the clients vs base revisions of the bundle.
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
398 A bundle can be applied only if all its base revisions are known by
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
399 the client.
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
400 - At least one leaf of the bundle's DAG is missing on the client.
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
401 - Every leaf of the bundle's DAG is part of node set the client wants.
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
402 E.g. do not send a bundle of all changes if the client wants only
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
403 one specific branch of many.
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
404 """
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
405 def decodehexstring(s):
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
406 return set([h.decode('hex') for h in s.split(';')])
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
407
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
408 manifest = repo.vfs.tryread('pullbundles.manifest')
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
409 if not manifest:
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
410 return None
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
411 res = exchange.parseclonebundlesmanifest(repo, manifest)
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
412 res = exchange.filterclonebundleentries(repo, res)
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
413 if not res:
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
414 return None
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
415 cl = repo.changelog
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
416 heads_anc = cl.ancestors([cl.rev(rev) for rev in heads], inclusive=True)
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
417 common_anc = cl.ancestors([cl.rev(rev) for rev in common], inclusive=True)
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
418 compformats = clientcompressionsupport(proto)
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
419 for entry in res:
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
420 if 'COMPRESSION' in entry and entry['COMPRESSION'] not in compformats:
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
421 continue
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
422 # No test yet for VERSION, since V2 is supported by any client
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
423 # that advertises partial pulls
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
424 if 'heads' in entry:
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
425 try:
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
426 bundle_heads = decodehexstring(entry['heads'])
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
427 except TypeError:
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
428 # Bad heads entry
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
429 continue
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
430 if bundle_heads.issubset(common):
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
431 continue # Nothing new
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
432 if all(cl.rev(rev) in common_anc for rev in bundle_heads):
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
433 continue # Still nothing new
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
434 if any(cl.rev(rev) not in heads_anc and
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
435 cl.rev(rev) not in common_anc for rev in bundle_heads):
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
436 continue
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
437 if 'bases' in entry:
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
438 try:
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
439 bundle_bases = decodehexstring(entry['bases'])
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
440 except TypeError:
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
441 # Bad bases entry
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
442 continue
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
443 if not all(cl.rev(rev) in common_anc for rev in bundle_bases):
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
444 continue
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
445 path = entry['URL']
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
446 repo.ui.debug('sending pullbundle "%s"\n' % path)
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
447 try:
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
448 return repo.vfs.open(path)
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
449 except IOError:
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
450 repo.ui.debug('pullbundle "%s" not accessible\n' % path)
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
451 continue
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
452 return None
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
453
37779
379d54eae6eb wireproto: don't pass transportpolicy argument
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
454 @wireprotocommand('getbundle', '*', permission='pull')
13741
b51bf961b3cb wireproto: add getbundle() function
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13726
diff changeset
455 def getbundle(repo, proto, others):
37613
96d735601ca1 wireproto: move gboptsmap to wireprototypes and rename (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37612
diff changeset
456 opts = options('getbundle', wireprototypes.GETBUNDLE_ARGUMENTS.keys(),
96d735601ca1 wireproto: move gboptsmap to wireprototypes and rename (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37612
diff changeset
457 others)
13741
b51bf961b3cb wireproto: add getbundle() function
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13726
diff changeset
458 for k, v in opts.iteritems():
37613
96d735601ca1 wireproto: move gboptsmap to wireprototypes and rename (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37612
diff changeset
459 keytype = wireprototypes.GETBUNDLE_ARGUMENTS[k]
21646
ce25f465e572 getbundle: declare type of parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21631
diff changeset
460 if keytype == 'nodes':
37612
5e71dea79aae wireproto: move value encoding functions to wireprototypes (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37546
diff changeset
461 opts[k] = wireprototypes.decodelist(v)
21646
ce25f465e572 getbundle: declare type of parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21631
diff changeset
462 elif keytype == 'csv':
25403
30ab130af221 wireprotocol: distinguish list and set in getbundle argument
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25339
diff changeset
463 opts[k] = list(v.split(','))
30ab130af221 wireprotocol: distinguish list and set in getbundle argument
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25339
diff changeset
464 elif keytype == 'scsv':
19201
309c439cdbaa bundle-ng: add bundlecaps argument to getbundle() command
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 19176
diff changeset
465 opts[k] = set(v.split(','))
21988
12cd3827b860 wireproto: add a ``boolean`` type for getbundle parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21728
diff changeset
466 elif keytype == 'boolean':
26686
3e7f675628ad wireproto: properly parse false boolean args (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26587
diff changeset
467 # Client should serialize False as '0', which is a non-empty string
3e7f675628ad wireproto: properly parse false boolean args (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26587
diff changeset
468 # so it evaluates as a True bool.
3e7f675628ad wireproto: properly parse false boolean args (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26587
diff changeset
469 if v == '0':
3e7f675628ad wireproto: properly parse false boolean args (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26587
diff changeset
470 opts[k] = False
3e7f675628ad wireproto: properly parse false boolean args (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26587
diff changeset
471 else:
3e7f675628ad wireproto: properly parse false boolean args (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26587
diff changeset
472 opts[k] = bool(v)
21646
ce25f465e572 getbundle: declare type of parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21631
diff changeset
473 elif keytype != 'plain':
ce25f465e572 getbundle: declare type of parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21631
diff changeset
474 raise KeyError('unknown getbundle option type %s'
ce25f465e572 getbundle: declare type of parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21631
diff changeset
475 % keytype)
27246
b288fb2724bf wireproto: config options to disable bundle1
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27243
diff changeset
476
27633
37d7cf569cf3 wireproto: support disabling bundle1 only if repo is generaldelta
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27246
diff changeset
477 if not bundle1allowed(repo, 'pull'):
27246
b288fb2724bf wireproto: config options to disable bundle1
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27243
diff changeset
478 if not exchange.bundle2requested(opts.get('bundlecaps')):
36261
2e07dc514073 wireprotoserver: add version to HTTP protocol name (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36259
diff changeset
479 if proto.name == 'http-v1':
37293
d5d665f6615a wireproto: stop aliasing wire protocol types (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37122
diff changeset
480 return wireprototypes.ooberror(bundle2required)
30874
3d4afc2fdcd7 bundle1: fix bundle1-denied reporting for pull over ssh
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 30872
diff changeset
481 raise error.Abort(bundle2requiredmain,
3d4afc2fdcd7 bundle1: fix bundle1-denied reporting for pull over ssh
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 30872
diff changeset
482 hint=bundle2requiredhint)
27246
b288fb2724bf wireproto: config options to disable bundle1
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27243
diff changeset
483
35787
a84dbc87dae9 exchange: send bundle2 stream clones uncompressed
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35785
diff changeset
484 prefercompressed = True
35782
9d249f3de730 wireproto: don't compress errors from getbundle()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35760
diff changeset
485
30876
f3807a135e43 wireproto: properly report server Abort during 'getbundle'
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 30874
diff changeset
486 try:
37498
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
487 clheads = set(repo.changelog.heads())
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
488 heads = set(opts.get('heads', set()))
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
489 common = set(opts.get('common', set()))
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
490 common.discard(nullid)
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
491 if (repo.ui.configbool('server', 'pullbundle') and
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
492 'partial-pull' in proto.getprotocaps()):
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
493 # Check if a pre-built bundle covers this request.
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
494 bundle = find_pullbundle(repo, proto, opts, clheads, heads, common)
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
495 if bundle:
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
496 return wireprototypes.streamres(gen=util.filechunkiter(bundle),
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
497 prefer_uncompressed=True)
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
498
33223
d227451ee280 configitems: register the 'server.disablefullbundle' config
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 33220
diff changeset
499 if repo.ui.configbool('server', 'disablefullbundle'):
32300
d0d9a4fca59b clone: add a server-side option to disable full getbundles (pull-based clones)
Siddharth Agarwal <sid0@fb.com>
parents: 31460
diff changeset
500 # Check to see if this is a full clone.
35760
133a678673cb clone: allow bundle2's stream clone with 'server.disablefullbundle'
Boris Feld <boris.feld@octobus.net>
parents: 35759
diff changeset
501 changegroup = opts.get('cg', True)
133a678673cb clone: allow bundle2's stream clone with 'server.disablefullbundle'
Boris Feld <boris.feld@octobus.net>
parents: 35759
diff changeset
502 if changegroup and not common and clheads == heads:
32300
d0d9a4fca59b clone: add a server-side option to disable full getbundles (pull-based clones)
Siddharth Agarwal <sid0@fb.com>
parents: 31460
diff changeset
503 raise error.Abort(
d0d9a4fca59b clone: add a server-side option to disable full getbundles (pull-based clones)
Siddharth Agarwal <sid0@fb.com>
parents: 31460
diff changeset
504 _('server has pull-based clones disabled'),
d0d9a4fca59b clone: add a server-side option to disable full getbundles (pull-based clones)
Siddharth Agarwal <sid0@fb.com>
parents: 31460
diff changeset
505 hint=_('remove --pull if specified or upgrade Mercurial'))
d0d9a4fca59b clone: add a server-side option to disable full getbundles (pull-based clones)
Siddharth Agarwal <sid0@fb.com>
parents: 31460
diff changeset
506
35785
ba15580e53d5 exchange: return bundle info from getbundlechunks() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35783
diff changeset
507 info, chunks = exchange.getbundlechunks(repo, 'serve',
ba15580e53d5 exchange: return bundle info from getbundlechunks() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35783
diff changeset
508 **pycompat.strkwargs(opts))
35787
a84dbc87dae9 exchange: send bundle2 stream clones uncompressed
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35785
diff changeset
509 prefercompressed = info.get('prefercompressed', True)
30876
f3807a135e43 wireproto: properly report server Abort during 'getbundle'
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 30874
diff changeset
510 except error.Abort as exc:
f3807a135e43 wireproto: properly report server Abort during 'getbundle'
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 30874
diff changeset
511 # cleanly forward Abort error to the client
f3807a135e43 wireproto: properly report server Abort during 'getbundle'
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 30874
diff changeset
512 if not exchange.bundle2requested(opts.get('bundlecaps')):
36261
2e07dc514073 wireprotoserver: add version to HTTP protocol name (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36259
diff changeset
513 if proto.name == 'http-v1':
37293
d5d665f6615a wireproto: stop aliasing wire protocol types (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37122
diff changeset
514 return wireprototypes.ooberror(pycompat.bytestr(exc) + '\n')
30876
f3807a135e43 wireproto: properly report server Abort during 'getbundle'
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 30874
diff changeset
515 raise # cannot do better for bundle1 + ssh
f3807a135e43 wireproto: properly report server Abort during 'getbundle'
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 30874
diff changeset
516 # bundle2 request expect a bundle2 reply
f3807a135e43 wireproto: properly report server Abort during 'getbundle'
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 30874
diff changeset
517 bundler = bundle2.bundle20(repo.ui)
36291
af0a19d8812b py3: get bytes-repr of network errors portably
Augie Fackler <augie@google.com>
parents: 36261
diff changeset
518 manargs = [('message', pycompat.bytestr(exc))]
30876
f3807a135e43 wireproto: properly report server Abort during 'getbundle'
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 30874
diff changeset
519 advargs = []
f3807a135e43 wireproto: properly report server Abort during 'getbundle'
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 30874
diff changeset
520 if exc.hint is not None:
f3807a135e43 wireproto: properly report server Abort during 'getbundle'
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 30874
diff changeset
521 advargs.append(('hint', exc.hint))
f3807a135e43 wireproto: properly report server Abort during 'getbundle'
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 30874
diff changeset
522 bundler.addpart(bundle2.bundlepart('error:abort',
f3807a135e43 wireproto: properly report server Abort during 'getbundle'
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 30874
diff changeset
523 manargs, advargs))
35782
9d249f3de730 wireproto: don't compress errors from getbundle()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35760
diff changeset
524 chunks = bundler.getchunks()
35787
a84dbc87dae9 exchange: send bundle2 stream clones uncompressed
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35785
diff changeset
525 prefercompressed = False
35782
9d249f3de730 wireproto: don't compress errors from getbundle()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35760
diff changeset
526
37293
d5d665f6615a wireproto: stop aliasing wire protocol types (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37122
diff changeset
527 return wireprototypes.streamres(
d5d665f6615a wireproto: stop aliasing wire protocol types (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37122
diff changeset
528 gen=chunks, prefer_uncompressed=not prefercompressed)
13741
b51bf961b3cb wireproto: add getbundle() function
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13726
diff changeset
529
37779
379d54eae6eb wireproto: don't pass transportpolicy argument
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
530 @wireprotocommand('heads', permission='pull')
11583
944c23762c3c protocol: add proto to method prototypes
Matt Mackall <mpm@selenic.com>
parents: 11581
diff changeset
531 def heads(repo, proto):
18281
898c575833c9 clfilter: drop extra filtering in wireprotocol
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 18280
diff changeset
532 h = repo.heads()
37612
5e71dea79aae wireproto: move value encoding functions to wireprototypes (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37546
diff changeset
533 return wireprototypes.bytesresponse(wireprototypes.encodelist(h) + '\n')
11581
4530b3307fb9 protocol: introduce wireproto.py
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
534
37779
379d54eae6eb wireproto: don't pass transportpolicy argument
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
535 @wireprotocommand('hello', permission='pull')
11594
67863f9d805f protocol: unify server-side capabilities functions
Matt Mackall <mpm@selenic.com>
parents: 11593
diff changeset
536 def hello(repo, proto):
36259
62bca1c50e96 wireproto: improve docstring for "hello"
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36112
diff changeset
537 """Called as part of SSH handshake to obtain server info.
62bca1c50e96 wireproto: improve docstring for "hello"
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36112
diff changeset
538
62bca1c50e96 wireproto: improve docstring for "hello"
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36112
diff changeset
539 Returns a list of lines describing interesting things about the
62bca1c50e96 wireproto: improve docstring for "hello"
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36112
diff changeset
540 server, in an RFC822-like format.
11594
67863f9d805f protocol: unify server-side capabilities functions
Matt Mackall <mpm@selenic.com>
parents: 11593
diff changeset
541
36259
62bca1c50e96 wireproto: improve docstring for "hello"
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36112
diff changeset
542 Currently, the only one defined is ``capabilities``, which consists of a
62bca1c50e96 wireproto: improve docstring for "hello"
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36112
diff changeset
543 line of space separated tokens describing server abilities:
62bca1c50e96 wireproto: improve docstring for "hello"
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36112
diff changeset
544
62bca1c50e96 wireproto: improve docstring for "hello"
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36112
diff changeset
545 capabilities: <token0> <token1> <token2>
62bca1c50e96 wireproto: improve docstring for "hello"
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36112
diff changeset
546 """
36112
2f7290555c96 wireproto: introduce type for raw byte responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36111
diff changeset
547 caps = capabilities(repo, proto).data
37293
d5d665f6615a wireproto: stop aliasing wire protocol types (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37122
diff changeset
548 return wireprototypes.bytesresponse('capabilities: %s\n' % caps)
11594
67863f9d805f protocol: unify server-side capabilities functions
Matt Mackall <mpm@selenic.com>
parents: 11593
diff changeset
549
37779
379d54eae6eb wireproto: don't pass transportpolicy argument
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
550 @wireprotocommand('listkeys', 'namespace', permission='pull')
11583
944c23762c3c protocol: add proto to method prototypes
Matt Mackall <mpm@selenic.com>
parents: 11581
diff changeset
551 def listkeys(repo, proto, namespace):
36558
33c6f8f0388d wireproto: sort response to listkeys
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36461
diff changeset
552 d = sorted(repo.listkeys(encoding.tolocal(namespace)).items())
37293
d5d665f6615a wireproto: stop aliasing wire protocol types (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37122
diff changeset
553 return wireprototypes.bytesresponse(pushkeymod.encodekeys(d))
11581
4530b3307fb9 protocol: introduce wireproto.py
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
554
37779
379d54eae6eb wireproto: don't pass transportpolicy argument
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
555 @wireprotocommand('lookup', 'key', permission='pull')
11583
944c23762c3c protocol: add proto to method prototypes
Matt Mackall <mpm@selenic.com>
parents: 11581
diff changeset
556 def lookup(repo, proto, key):
11581
4530b3307fb9 protocol: introduce wireproto.py
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
557 try:
15925
f9fc46698352 wireproto: refuse to lookup secret csets
Matt Mackall <mpm@selenic.com>
parents: 15713
diff changeset
558 k = encoding.tolocal(key)
37353
ac666c5c2e0c wireproto: use repo.lookup() for lookup command
Martin von Zweigbergk <martinvonz@google.com>
parents: 37320
diff changeset
559 n = repo.lookup(k)
ac666c5c2e0c wireproto: use repo.lookup() for lookup command
Martin von Zweigbergk <martinvonz@google.com>
parents: 37320
diff changeset
560 r = hex(n)
11581
4530b3307fb9 protocol: introduce wireproto.py
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
561 success = 1
25660
328739ea70c3 global: mass rewrite to use modern exception syntax
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25493
diff changeset
562 except Exception as inst:
37087
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module
Yuya Nishihara <yuya@tcha.org>
parents: 37056
diff changeset
563 r = stringutil.forcebytestr(inst)
11581
4530b3307fb9 protocol: introduce wireproto.py
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
564 success = 0
37293
d5d665f6615a wireproto: stop aliasing wire protocol types (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37122
diff changeset
565 return wireprototypes.bytesresponse('%d %s\n' % (success, r))
11581
4530b3307fb9 protocol: introduce wireproto.py
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
566
37779
379d54eae6eb wireproto: don't pass transportpolicy argument
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
567 @wireprotocommand('known', 'nodes *', permission='pull')
14436
5adb52524779 wireproto: enable optional args for known() for future extensibility
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14419
diff changeset
568 def known(repo, proto, nodes, others):
37612
5e71dea79aae wireproto: move value encoding functions to wireprototypes (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37546
diff changeset
569 v = ''.join(b and '1' or '0'
5e71dea79aae wireproto: move value encoding functions to wireprototypes (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37546
diff changeset
570 for b in repo.known(wireprototypes.decodelist(nodes)))
37293
d5d665f6615a wireproto: stop aliasing wire protocol types (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37122
diff changeset
571 return wireprototypes.bytesresponse(v)
13723
e615765fdcc7 wireproto: add known([id]) function
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13722
diff changeset
572
37779
379d54eae6eb wireproto: don't pass transportpolicy argument
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
573 @wireprotocommand('protocaps', 'caps', permission='pull')
37393
afcfdf53e4b5 wireproto: provide accessors for client capabilities
Joerg Sonnenberger <joerg@bec.de>
parents: 37353
diff changeset
574 def protocaps(repo, proto, caps):
afcfdf53e4b5 wireproto: provide accessors for client capabilities
Joerg Sonnenberger <joerg@bec.de>
parents: 37353
diff changeset
575 if proto.name == wireprototypes.SSHV1:
afcfdf53e4b5 wireproto: provide accessors for client capabilities
Joerg Sonnenberger <joerg@bec.de>
parents: 37353
diff changeset
576 proto._protocaps = set(caps.split(' '))
afcfdf53e4b5 wireproto: provide accessors for client capabilities
Joerg Sonnenberger <joerg@bec.de>
parents: 37353
diff changeset
577 return wireprototypes.bytesresponse('OK')
afcfdf53e4b5 wireproto: provide accessors for client capabilities
Joerg Sonnenberger <joerg@bec.de>
parents: 37353
diff changeset
578
37779
379d54eae6eb wireproto: don't pass transportpolicy argument
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
579 @wireprotocommand('pushkey', 'namespace key old new', permission='push')
11583
944c23762c3c protocol: add proto to method prototypes
Matt Mackall <mpm@selenic.com>
parents: 11581
diff changeset
580 def pushkey(repo, proto, namespace, key, old, new):
13050
3790452d499b pushkey: use UTF-8
Matt Mackall <mpm@selenic.com>
parents: 13049
diff changeset
581 # compatibility with pre-1.8 clients which were accidentally
3790452d499b pushkey: use UTF-8
Matt Mackall <mpm@selenic.com>
parents: 13049
diff changeset
582 # sending raw binary nodes rather than utf-8-encoded hex
37087
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module
Yuya Nishihara <yuya@tcha.org>
parents: 37056
diff changeset
583 if len(new) == 20 and stringutil.escapestr(new) != new:
13050
3790452d499b pushkey: use UTF-8
Matt Mackall <mpm@selenic.com>
parents: 13049
diff changeset
584 # looks like it could be a binary node
3790452d499b pushkey: use UTF-8
Matt Mackall <mpm@selenic.com>
parents: 13049
diff changeset
585 try:
14064
e4bfb9c337f3 remove unused imports and variables
Alexander Solovyov <alexander@solovyov.net>
parents: 14048
diff changeset
586 new.decode('utf-8')
13050
3790452d499b pushkey: use UTF-8
Matt Mackall <mpm@selenic.com>
parents: 13049
diff changeset
587 new = encoding.tolocal(new) # but cleanly decodes as UTF-8
3790452d499b pushkey: use UTF-8
Matt Mackall <mpm@selenic.com>
parents: 13049
diff changeset
588 except UnicodeDecodeError:
3790452d499b pushkey: use UTF-8
Matt Mackall <mpm@selenic.com>
parents: 13049
diff changeset
589 pass # binary, leave unmodified
3790452d499b pushkey: use UTF-8
Matt Mackall <mpm@selenic.com>
parents: 13049
diff changeset
590 else:
3790452d499b pushkey: use UTF-8
Matt Mackall <mpm@selenic.com>
parents: 13049
diff changeset
591 new = encoding.tolocal(new) # normal path
3790452d499b pushkey: use UTF-8
Matt Mackall <mpm@selenic.com>
parents: 13049
diff changeset
592
36104
2ad145fbde54 wireprotoserver: add context manager mechanism for redirecting stdio
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36021
diff changeset
593 with proto.mayberedirectstdio() as output:
36018
ae79cf6f9c82 wireproto: remove unnecessary exception trapping
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35900
diff changeset
594 r = repo.pushkey(encoding.tolocal(namespace), encoding.tolocal(key),
ae79cf6f9c82 wireproto: remove unnecessary exception trapping
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35900
diff changeset
595 encoding.tolocal(old), new) or False
17793
8474be4412ca wireproto: fix pushkey hook failure and output on remote http repo
Wagner Bruna <wbruna@softwareexpress.com.br>
parents: 17603
diff changeset
596
36104
2ad145fbde54 wireprotoserver: add context manager mechanism for redirecting stdio
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36021
diff changeset
597 output = output.getvalue() if output else ''
37293
d5d665f6615a wireproto: stop aliasing wire protocol types (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37122
diff changeset
598 return wireprototypes.bytesresponse('%d\n%s' % (int(r), output))
11581
4530b3307fb9 protocol: introduce wireproto.py
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
599
37779
379d54eae6eb wireproto: don't pass transportpolicy argument
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
600 @wireprotocommand('stream_out', permission='pull')
11585
5d907fbb9703 protocol: unify stream_out command
Matt Mackall <mpm@selenic.com>
parents: 11584
diff changeset
601 def stream(repo, proto):
11627
04f76a954842 protocol: move the streamclone implementation into wireproto
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 11625
diff changeset
602 '''If the server supports streaming clone, it advertises the "stream"
04f76a954842 protocol: move the streamclone implementation into wireproto
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 11625
diff changeset
603 capability with a value representing the version and flags of the repo
04f76a954842 protocol: move the streamclone implementation into wireproto
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 11625
diff changeset
604 it is serving. Client checks to see if it understands the format.
04f76a954842 protocol: move the streamclone implementation into wireproto
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 11625
diff changeset
605 '''
37293
d5d665f6615a wireproto: stop aliasing wire protocol types (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37122
diff changeset
606 return wireprototypes.streamreslegacy(
d5d665f6615a wireproto: stop aliasing wire protocol types (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37122
diff changeset
607 streamclone.generatev1wireproto(repo))
11585
5d907fbb9703 protocol: unify stream_out command
Matt Mackall <mpm@selenic.com>
parents: 11584
diff changeset
608
37779
379d54eae6eb wireproto: don't pass transportpolicy argument
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
609 @wireprotocommand('unbundle', 'heads', permission='push')
11593
d054cc5c7737 protocol: unify unbundle on the server side
Matt Mackall <mpm@selenic.com>
parents: 11592
diff changeset
610 def unbundle(repo, proto, heads):
37612
5e71dea79aae wireproto: move value encoding functions to wireprototypes (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37546
diff changeset
611 their_heads = wireprototypes.decodelist(heads)
11593
d054cc5c7737 protocol: unify unbundle on the server side
Matt Mackall <mpm@selenic.com>
parents: 11592
diff changeset
612
36105
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36104
diff changeset
613 with proto.mayberedirectstdio() as output:
11593
d054cc5c7737 protocol: unify unbundle on the server side
Matt Mackall <mpm@selenic.com>
parents: 11592
diff changeset
614 try:
36105
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36104
diff changeset
615 exchange.check_heads(repo, their_heads, 'preparing changes')
37414
2d965bfeb8f6 wireproto: allow direct stream processing for unbundle
Joerg Sonnenberger <joerg@bec.de>
parents: 37413
diff changeset
616 cleanup = lambda: None
2d965bfeb8f6 wireproto: allow direct stream processing for unbundle
Joerg Sonnenberger <joerg@bec.de>
parents: 37413
diff changeset
617 try:
2d965bfeb8f6 wireproto: allow direct stream processing for unbundle
Joerg Sonnenberger <joerg@bec.de>
parents: 37413
diff changeset
618 payload = proto.getpayload()
2d965bfeb8f6 wireproto: allow direct stream processing for unbundle
Joerg Sonnenberger <joerg@bec.de>
parents: 37413
diff changeset
619 if repo.ui.configbool('server', 'streamunbundle'):
2d965bfeb8f6 wireproto: allow direct stream processing for unbundle
Joerg Sonnenberger <joerg@bec.de>
parents: 37413
diff changeset
620 def cleanup():
2d965bfeb8f6 wireproto: allow direct stream processing for unbundle
Joerg Sonnenberger <joerg@bec.de>
parents: 37413
diff changeset
621 # Ensure that the full payload is consumed, so
2d965bfeb8f6 wireproto: allow direct stream processing for unbundle
Joerg Sonnenberger <joerg@bec.de>
parents: 37413
diff changeset
622 # that the connection doesn't contain trailing garbage.
2d965bfeb8f6 wireproto: allow direct stream processing for unbundle
Joerg Sonnenberger <joerg@bec.de>
parents: 37413
diff changeset
623 for p in payload:
2d965bfeb8f6 wireproto: allow direct stream processing for unbundle
Joerg Sonnenberger <joerg@bec.de>
parents: 37413
diff changeset
624 pass
2d965bfeb8f6 wireproto: allow direct stream processing for unbundle
Joerg Sonnenberger <joerg@bec.de>
parents: 37413
diff changeset
625 fp = util.chunkbuffer(payload)
2d965bfeb8f6 wireproto: allow direct stream processing for unbundle
Joerg Sonnenberger <joerg@bec.de>
parents: 37413
diff changeset
626 else:
2d965bfeb8f6 wireproto: allow direct stream processing for unbundle
Joerg Sonnenberger <joerg@bec.de>
parents: 37413
diff changeset
627 # write bundle data to temporary file as it can be big
2d965bfeb8f6 wireproto: allow direct stream processing for unbundle
Joerg Sonnenberger <joerg@bec.de>
parents: 37413
diff changeset
628 fp, tempname = None, None
2d965bfeb8f6 wireproto: allow direct stream processing for unbundle
Joerg Sonnenberger <joerg@bec.de>
parents: 37413
diff changeset
629 def cleanup():
2d965bfeb8f6 wireproto: allow direct stream processing for unbundle
Joerg Sonnenberger <joerg@bec.de>
parents: 37413
diff changeset
630 if fp:
2d965bfeb8f6 wireproto: allow direct stream processing for unbundle
Joerg Sonnenberger <joerg@bec.de>
parents: 37413
diff changeset
631 fp.close()
2d965bfeb8f6 wireproto: allow direct stream processing for unbundle
Joerg Sonnenberger <joerg@bec.de>
parents: 37413
diff changeset
632 if tempname:
2d965bfeb8f6 wireproto: allow direct stream processing for unbundle
Joerg Sonnenberger <joerg@bec.de>
parents: 37413
diff changeset
633 os.unlink(tempname)
2d965bfeb8f6 wireproto: allow direct stream processing for unbundle
Joerg Sonnenberger <joerg@bec.de>
parents: 37413
diff changeset
634 fd, tempname = tempfile.mkstemp(prefix='hg-unbundle-')
2d965bfeb8f6 wireproto: allow direct stream processing for unbundle
Joerg Sonnenberger <joerg@bec.de>
parents: 37413
diff changeset
635 repo.ui.debug('redirecting incoming bundle to %s\n' %
2d965bfeb8f6 wireproto: allow direct stream processing for unbundle
Joerg Sonnenberger <joerg@bec.de>
parents: 37413
diff changeset
636 tempname)
2d965bfeb8f6 wireproto: allow direct stream processing for unbundle
Joerg Sonnenberger <joerg@bec.de>
parents: 37413
diff changeset
637 fp = os.fdopen(fd, pycompat.sysstr('wb+'))
2d965bfeb8f6 wireproto: allow direct stream processing for unbundle
Joerg Sonnenberger <joerg@bec.de>
parents: 37413
diff changeset
638 r = 0
2d965bfeb8f6 wireproto: allow direct stream processing for unbundle
Joerg Sonnenberger <joerg@bec.de>
parents: 37413
diff changeset
639 for p in payload:
2d965bfeb8f6 wireproto: allow direct stream processing for unbundle
Joerg Sonnenberger <joerg@bec.de>
parents: 37413
diff changeset
640 fp.write(p)
2d965bfeb8f6 wireproto: allow direct stream processing for unbundle
Joerg Sonnenberger <joerg@bec.de>
parents: 37413
diff changeset
641 fp.seek(0)
27246
b288fb2724bf wireproto: config options to disable bundle1
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27243
diff changeset
642
36105
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36104
diff changeset
643 gen = exchange.readbundle(repo.ui, fp, None)
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36104
diff changeset
644 if (isinstance(gen, changegroupmod.cg1unpacker)
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36104
diff changeset
645 and not bundle1allowed(repo, 'push')):
36261
2e07dc514073 wireprotoserver: add version to HTTP protocol name (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36259
diff changeset
646 if proto.name == 'http-v1':
36105
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36104
diff changeset
647 # need to special case http because stderr do not get to
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36104
diff changeset
648 # the http client on failed push so we need to abuse
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36104
diff changeset
649 # some other error type to make sure the message get to
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36104
diff changeset
650 # the user.
37293
d5d665f6615a wireproto: stop aliasing wire protocol types (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37122
diff changeset
651 return wireprototypes.ooberror(bundle2required)
36105
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36104
diff changeset
652 raise error.Abort(bundle2requiredmain,
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36104
diff changeset
653 hint=bundle2requiredhint)
24796
61ff209fc01d bundle2: refactor error bundle creation for the wireprotocol
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24696
diff changeset
654
36105
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36104
diff changeset
655 r = exchange.unbundle(repo, gen, their_heads, 'serve',
36107
957e773614d0 wireprotoserver: rename _client to client (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36105
diff changeset
656 proto.client())
36105
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36104
diff changeset
657 if util.safehasattr(r, 'addpart'):
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36104
diff changeset
658 # The return looks streamable, we are in the bundle2 case
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36104
diff changeset
659 # and should return a stream.
37293
d5d665f6615a wireproto: stop aliasing wire protocol types (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37122
diff changeset
660 return wireprototypes.streamreslegacy(gen=r.getchunks())
d5d665f6615a wireproto: stop aliasing wire protocol types (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37122
diff changeset
661 return wireprototypes.pushres(
d5d665f6615a wireproto: stop aliasing wire protocol types (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37122
diff changeset
662 r, output.getvalue() if output else '')
36105
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36104
diff changeset
663
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36104
diff changeset
664 finally:
37414
2d965bfeb8f6 wireproto: allow direct stream processing for unbundle
Joerg Sonnenberger <joerg@bec.de>
parents: 37413
diff changeset
665 cleanup()
36105
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36104
diff changeset
666
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36104
diff changeset
667 except (error.BundleValueError, error.Abort, error.PushRaced) as exc:
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36104
diff changeset
668 # handle non-bundle2 case first
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36104
diff changeset
669 if not getattr(exc, 'duringunbundle2', False):
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36104
diff changeset
670 try:
25493
d8e7b0781ad7 bundle2: convey PushkeyFailed error over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25403
diff changeset
671 raise
36105
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36104
diff changeset
672 except error.Abort:
37122
d4a2e0d5d042 procutil: bulk-replace util.std* to point to new module
Yuya Nishihara <yuya@tcha.org>
parents: 37087
diff changeset
673 # The old code we moved used procutil.stderr directly.
36105
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36104
diff changeset
674 # We did not change it to minimise code change.
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36104
diff changeset
675 # This need to be moved to something proper.
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36104
diff changeset
676 # Feel free to do it.
37122
d4a2e0d5d042 procutil: bulk-replace util.std* to point to new module
Yuya Nishihara <yuya@tcha.org>
parents: 37087
diff changeset
677 procutil.stderr.write("abort: %s\n" % exc)
36105
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36104
diff changeset
678 if exc.hint is not None:
37122
d4a2e0d5d042 procutil: bulk-replace util.std* to point to new module
Yuya Nishihara <yuya@tcha.org>
parents: 37087
diff changeset
679 procutil.stderr.write("(%s)\n" % exc.hint)
d4a2e0d5d042 procutil: bulk-replace util.std* to point to new module
Yuya Nishihara <yuya@tcha.org>
parents: 37087
diff changeset
680 procutil.stderr.flush()
37293
d5d665f6615a wireproto: stop aliasing wire protocol types (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37122
diff changeset
681 return wireprototypes.pushres(
d5d665f6615a wireproto: stop aliasing wire protocol types (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37122
diff changeset
682 0, output.getvalue() if output else '')
36105
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36104
diff changeset
683 except error.PushRaced:
37293
d5d665f6615a wireproto: stop aliasing wire protocol types (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37122
diff changeset
684 return wireprototypes.pusherr(
d5d665f6615a wireproto: stop aliasing wire protocol types (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37122
diff changeset
685 pycompat.bytestr(exc),
d5d665f6615a wireproto: stop aliasing wire protocol types (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37122
diff changeset
686 output.getvalue() if output else '')
36105
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36104
diff changeset
687
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36104
diff changeset
688 bundler = bundle2.bundle20(repo.ui)
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36104
diff changeset
689 for out in getattr(exc, '_bundle2salvagedoutput', ()):
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36104
diff changeset
690 bundler.addpart(out)
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36104
diff changeset
691 try:
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36104
diff changeset
692 try:
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36104
diff changeset
693 raise
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36104
diff changeset
694 except error.PushkeyFailed as exc:
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36104
diff changeset
695 # check client caps
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36104
diff changeset
696 remotecaps = getattr(exc, '_replycaps', None)
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36104
diff changeset
697 if (remotecaps is not None
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36104
diff changeset
698 and 'pushkey' not in remotecaps.get('error', ())):
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36104
diff changeset
699 # no support remote side, fallback to Abort handler.
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36104
diff changeset
700 raise
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36104
diff changeset
701 part = bundler.newpart('error:pushkey')
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36104
diff changeset
702 part.addparam('in-reply-to', exc.partid)
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36104
diff changeset
703 if exc.namespace is not None:
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36104
diff changeset
704 part.addparam('namespace', exc.namespace,
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36104
diff changeset
705 mandatory=False)
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36104
diff changeset
706 if exc.key is not None:
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36104
diff changeset
707 part.addparam('key', exc.key, mandatory=False)
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36104
diff changeset
708 if exc.new is not None:
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36104
diff changeset
709 part.addparam('new', exc.new, mandatory=False)
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36104
diff changeset
710 if exc.old is not None:
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36104
diff changeset
711 part.addparam('old', exc.old, mandatory=False)
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36104
diff changeset
712 if exc.ret is not None:
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36104
diff changeset
713 part.addparam('ret', exc.ret, mandatory=False)
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36104
diff changeset
714 except error.BundleValueError as exc:
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36104
diff changeset
715 errpart = bundler.newpart('error:unsupportedcontent')
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36104
diff changeset
716 if exc.parttype is not None:
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36104
diff changeset
717 errpart.addparam('parttype', exc.parttype)
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36104
diff changeset
718 if exc.params:
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36104
diff changeset
719 errpart.addparam('params', '\0'.join(exc.params))
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36104
diff changeset
720 except error.Abort as exc:
37087
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module
Yuya Nishihara <yuya@tcha.org>
parents: 37056
diff changeset
721 manargs = [('message', stringutil.forcebytestr(exc))]
36105
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36104
diff changeset
722 advargs = []
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36104
diff changeset
723 if exc.hint is not None:
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36104
diff changeset
724 advargs.append(('hint', exc.hint))
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36104
diff changeset
725 bundler.addpart(bundle2.bundlepart('error:abort',
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36104
diff changeset
726 manargs, advargs))
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36104
diff changeset
727 except error.PushRaced as exc:
36347
be9c497e0bfd wireproto: fix lingering str(exception) with util.forcebytestr(exception)
Augie Fackler <augie@google.com>
parents: 36291
diff changeset
728 bundler.newpart('error:pushraced',
37087
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module
Yuya Nishihara <yuya@tcha.org>
parents: 37056
diff changeset
729 [('message', stringutil.forcebytestr(exc))])
37293
d5d665f6615a wireproto: stop aliasing wire protocol types (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37122
diff changeset
730 return wireprototypes.streamreslegacy(gen=bundler.getchunks())