annotate mercurial/bundle2.py @ 43845:1ffbd03c8d75

filemerge: drop a default argument to appease pytype The function slices and takes the length of this argument without internally setting it if not provided. There was no bug here because both callers passed the argument. Differential Revision: https://phab.mercurial-scm.org/D7464
author Matt Harbison <matt_harbison@yahoo.com>
date Wed, 20 Nov 2019 13:01:56 -0500
parents 888bd39ed555
children c443b9ba6f63
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
20801
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
1 # bundle2.py - generic container format to transmit arbitrary data.
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
2 #
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
3 # Copyright 2013 Facebook, Inc.
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
4 #
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
5 # This software may be used and distributed according to the terms of the
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
6 # GNU General Public License version 2 or any later version.
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
7 """Handling of the new bundle2 format
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
8
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
9 The goal of bundle2 is to act as an atomically packet to transmit a set of
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
10 payloads in an application agnostic way. It consist in a sequence of "parts"
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
11 that will be handed to and processed by the application layer.
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
12
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
13
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
14 General format architecture
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
15 ===========================
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
16
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
17 The format is architectured as follow
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
18
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
19 - magic string
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
20 - stream level parameters
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
21 - payload parts (any number)
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
22 - end of stream marker.
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
23
20856
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
24 the Binary format
20801
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
25 ============================
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
26
21024
7731a2281cf0 spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents: 21020
diff changeset
27 All numbers are unsigned and big-endian.
20801
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
28
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
29 stream level parameters
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
30 ------------------------
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
31
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
32 Binary format is as follow
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
33
23009
90f86ad3d4ff bundle2: change header size and make them signed (new format)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23008
diff changeset
34 :params size: int32
20801
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
35
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
36 The total number of Bytes used by the parameters
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
37
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
38 :params value: arbitrary number of Bytes
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
39
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
40 A blob of `params size` containing the serialized version of all stream level
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
41 parameters.
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
42
21024
7731a2281cf0 spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents: 21020
diff changeset
43 The blob contains a space separated list of parameters. Parameters with value
20811
9785c3f8f598 bundle2: urlquote stream parameter name and value
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20810
diff changeset
44 are stored in the form `<name>=<value>`. Both name and value are urlquoted.
20804
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
45
20813
8c74b3ce5b70 bundle2: refuse empty parameter name
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20812
diff changeset
46 Empty name are obviously forbidden.
8c74b3ce5b70 bundle2: refuse empty parameter name
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20812
diff changeset
47
20844
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
48 Name MUST start with a letter. If this first letter is lower case, the
21024
7731a2281cf0 spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents: 21020
diff changeset
49 parameter is advisory and can be safely ignored. However when the first
20844
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
50 letter is capital, the parameter is mandatory and the bundling process MUST
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
51 stop if he is not able to proceed it.
20814
8532f5e1b9df bundle2: force the first char of parameter to be an letter.
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20813
diff changeset
52
20808
4c9130c7a29f bundle2: clarify stream parameter design in the documentation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20805
diff changeset
53 Stream parameters use a simple textual format for two main reasons:
20804
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
54
21024
7731a2281cf0 spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents: 21020
diff changeset
55 - Stream level parameters should remain simple and we want to discourage any
20808
4c9130c7a29f bundle2: clarify stream parameter design in the documentation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20805
diff changeset
56 crazy usage.
21024
7731a2281cf0 spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents: 21020
diff changeset
57 - Textual data allow easy human inspection of a bundle2 header in case of
20808
4c9130c7a29f bundle2: clarify stream parameter design in the documentation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20805
diff changeset
58 troubles.
4c9130c7a29f bundle2: clarify stream parameter design in the documentation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20805
diff changeset
59
4c9130c7a29f bundle2: clarify stream parameter design in the documentation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20805
diff changeset
60 Any Applicative level options MUST go into a bundle2 part instead.
20801
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
61
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
62 Payload part
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
63 ------------------------
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
64
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
65 Binary format is as follow
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
66
23009
90f86ad3d4ff bundle2: change header size and make them signed (new format)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23008
diff changeset
67 :header size: int32
20801
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
68
25507
5bee483705e8 bundle2: clarify in docstring that header size is for a single header
Martin von Zweigbergk <martinvonz@google.com>
parents: 25506
diff changeset
69 The total number of Bytes used by the part header. When the header is empty
20801
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
70 (size = 0) this is interpreted as the end of stream marker.
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
71
20856
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
72 :header:
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
73
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
74 The header defines how to interpret the part. It contains two piece of
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
75 data: the part type, and the part parameters.
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
76
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
77 The part type is used to route an application level handler, that can
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
78 interpret payload.
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
79
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
80 Part parameters are passed to the application level handler. They are
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
81 meant to convey information that will help the application level object to
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
82 interpret the part payload.
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
83
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
84 The binary format of the header is has follow
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
85
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
86 :typesize: (one byte)
20877
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
87
23916
a3f7c781786b bundle2: fix parttype enforcement
Matt Mackall <mpm@selenic.com>
parents: 23868
diff changeset
88 :parttype: alphanumerical part name (restricted to [a-zA-Z0-9_:-]*)
20877
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
89
20995
e995d104c87f bundle2: add an integer id to part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20952
diff changeset
90 :partid: A 32bits integer (unique in the bundle) that can be used to refer
e995d104c87f bundle2: add an integer id to part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20952
diff changeset
91 to this part.
e995d104c87f bundle2: add an integer id to part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20952
diff changeset
92
20877
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
93 :parameters:
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
94
21024
7731a2281cf0 spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents: 21020
diff changeset
95 Part's parameter may have arbitrary content, the binary structure is::
20877
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
96
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
97 <mandatory-count><advisory-count><param-sizes><param-data>
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
98
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
99 :mandatory-count: 1 byte, number of mandatory parameters
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
100
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
101 :advisory-count: 1 byte, number of advisory parameters
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
102
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
103 :param-sizes:
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
104
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
105 N couple of bytes, where N is the total number of parameters. Each
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
106 couple contains (<size-of-key>, <size-of-value) for one parameter.
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
107
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
108 :param-data:
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
109
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
110 A blob of bytes from which each parameter key and value can be
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
111 retrieved using the list of size couples stored in the previous
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
112 field.
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
113
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
114 Mandatory parameters comes first, then the advisory ones.
20856
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
115
21607
054fa5176fa7 bundle2: forbid duplicate parameter keys
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21606
diff changeset
116 Each parameter's key MUST be unique within the part.
054fa5176fa7 bundle2: forbid duplicate parameter keys
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21606
diff changeset
117
20856
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
118 :payload:
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
119
20876
ddd56f3eb786 bundle2: support for bundling and unbundling payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20864
diff changeset
120 payload is a series of `<chunksize><chunkdata>`.
ddd56f3eb786 bundle2: support for bundling and unbundling payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20864
diff changeset
121
23009
90f86ad3d4ff bundle2: change header size and make them signed (new format)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23008
diff changeset
122 `chunksize` is an int32, `chunkdata` are plain bytes (as much as
20876
ddd56f3eb786 bundle2: support for bundling and unbundling payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20864
diff changeset
123 `chunksize` says)` The payload part is concluded by a zero size chunk.
ddd56f3eb786 bundle2: support for bundling and unbundling payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20864
diff changeset
124
ddd56f3eb786 bundle2: support for bundling and unbundling payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20864
diff changeset
125 The current implementation always produces either zero or one chunk.
21024
7731a2281cf0 spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents: 21020
diff changeset
126 This is an implementation limitation that will ultimately be lifted.
20891
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20890
diff changeset
127
23009
90f86ad3d4ff bundle2: change header size and make them signed (new format)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23008
diff changeset
128 `chunksize` can be negative to trigger special case processing. No such
90f86ad3d4ff bundle2: change header size and make them signed (new format)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23008
diff changeset
129 processing is in place yet.
90f86ad3d4ff bundle2: change header size and make them signed (new format)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23008
diff changeset
130
20891
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20890
diff changeset
131 Bundle processing
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20890
diff changeset
132 ============================
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20890
diff changeset
133
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20890
diff changeset
134 Each part is processed in order using a "part handler". Handler are registered
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20890
diff changeset
135 for a certain part type.
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20890
diff changeset
136
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20890
diff changeset
137 The matching of a part to its handler is case insensitive. The case of the
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20890
diff changeset
138 part type is used to know if a part is mandatory or advisory. If the Part type
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20890
diff changeset
139 contains any uppercase char it is considered mandatory. When no handler is
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20890
diff changeset
140 known for a Mandatory part, the process is aborted and an exception is raised.
20892
6fe95448596d bundle2: read the whole bundle from stream on abort
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20891
diff changeset
141 If the part is advisory and no handler is known, the part is ignored. When the
6fe95448596d bundle2: read the whole bundle from stream on abort
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20891
diff changeset
142 process is aborted, the full bundle is still read from the stream to keep the
6fe95448596d bundle2: read the whole bundle from stream on abort
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20891
diff changeset
143 channel usable. But none of the part read from an abort are processed. In the
6fe95448596d bundle2: read the whole bundle from stream on abort
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20891
diff changeset
144 future, dropping the stream may become an option for channel we do not care to
6fe95448596d bundle2: read the whole bundle from stream on abort
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20891
diff changeset
145 preserve.
20801
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
146 """
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
147
33672
da7c285ec6da bundle2: use modern Python division
Augie Fackler <augie@google.com>
parents: 33665
diff changeset
148 from __future__ import absolute_import, division
25919
8221fefaea08 bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
149
36964
79b73be4dda5 rev-branch-cache: add a function to generate a part
Boris Feld <boris.feld@octobus.net>
parents: 36963
diff changeset
150 import collections
24026
3daef83a1873 bundle2.unpackermixin: control for underlying file descriptor
Eric Sumner <ericsumner@fb.com>
parents: 23916
diff changeset
151 import errno
35046
241d9caca11e bundle2: use os.SEEK_* constants
Gregory Szorc <gregory.szorc@gmail.com>
parents: 34820
diff changeset
152 import os
25919
8221fefaea08 bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
153 import re
20814
8532f5e1b9df bundle2: force the first char of parameter to be an letter.
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20813
diff changeset
154 import string
25919
8221fefaea08 bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
155 import struct
8221fefaea08 bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
156 import sys
20804
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
157
25919
8221fefaea08 bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
158 from .i18n import _
8221fefaea08 bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
159 from . import (
35258
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35120
diff changeset
160 bookmarks,
25919
8221fefaea08 bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
161 changegroup,
36963
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part
Boris Feld <boris.feld@octobus.net>
parents: 36428
diff changeset
162 encoding,
25919
8221fefaea08 bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
163 error,
34820
a95067b1dca6 phase: introduce a new 'check:phases' part
Boris Feld <boris.feld@octobus.net>
parents: 34637
diff changeset
164 node as nodemod,
25919
8221fefaea08 bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
165 obsolete,
33031
e8c8d81eb864 bundle: add config option to include phases
Martin von Zweigbergk <martinvonz@google.com>
parents: 33030
diff changeset
166 phases,
25919
8221fefaea08 bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
167 pushkey,
30030
0f6d6fdd3c2a pycompat: provide 'ispy3' constant
Yuya Nishihara <yuya@tcha.org>
parents: 29937
diff changeset
168 pycompat,
35758
b996ddf5963d bundle2: add a 'stream' part handler for stream cloning
Boris Feld <boris.feld@octobus.net>
parents: 35655
diff changeset
169 streamclone,
25919
8221fefaea08 bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
170 tags,
8221fefaea08 bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
171 url,
8221fefaea08 bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
172 util,
8221fefaea08 bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
173 )
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
174 from .utils import stringutil
20802
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
175
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28672
diff changeset
176 urlerr = util.urlerr
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28672
diff changeset
177 urlreq = util.urlreq
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28672
diff changeset
178
20804
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
179 _pack = struct.pack
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
180 _unpack = struct.unpack
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
181
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
182 _fstreamparamsize = b'>i'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
183 _fpartheadersize = b'>i'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
184 _fparttypesize = b'>B'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
185 _fpartid = b'>I'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
186 _fpayloadsize = b'>i'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
187 _fpartparamcount = b'>BB'
20877
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
188
35793
4fb2bb61597c bundle2: increase payload part chunk size to 32kb
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35792
diff changeset
189 preferedchunksize = 32768
21001
c93bb6a08fa1 bundle2: support chunk iterator as part data
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21000
diff changeset
190
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
191 _parttypeforbidden = re.compile(b'[^a-zA-Z0-9_:-]')
23868
405eacbecc52 bundle2: enforce parttype as alphanumerical
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23590
diff changeset
192
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
193
25313
8f2c362bcb58 bundle2: introduce a specific function for bundling debug message
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25181
diff changeset
194 def outdebug(ui, message):
8f2c362bcb58 bundle2: introduce a specific function for bundling debug message
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25181
diff changeset
195 """debug regarding output stream (bundling)"""
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
196 if ui.configbool(b'devel', b'bundle2.debug'):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
197 ui.debug(b'bundle2-output: %s\n' % message)
25313
8f2c362bcb58 bundle2: introduce a specific function for bundling debug message
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25181
diff changeset
198
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
199
25318
64dd8f25fcf4 bundle2: introduce a specific function for debug messages while unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25317
diff changeset
200 def indebug(ui, message):
64dd8f25fcf4 bundle2: introduce a specific function for debug messages while unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25317
diff changeset
201 """debug on input stream (unbundling)"""
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
202 if ui.configbool(b'devel', b'bundle2.debug'):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
203 ui.debug(b'bundle2-input: %s\n' % message)
25318
64dd8f25fcf4 bundle2: introduce a specific function for debug messages while unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25317
diff changeset
204
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
205
23868
405eacbecc52 bundle2: enforce parttype as alphanumerical
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23590
diff changeset
206 def validateparttype(parttype):
405eacbecc52 bundle2: enforce parttype as alphanumerical
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23590
diff changeset
207 """raise ValueError if a parttype contains invalid character"""
23916
a3f7c781786b bundle2: fix parttype enforcement
Matt Mackall <mpm@selenic.com>
parents: 23868
diff changeset
208 if _parttypeforbidden.search(parttype):
23868
405eacbecc52 bundle2: enforce parttype as alphanumerical
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23590
diff changeset
209 raise ValueError(parttype)
405eacbecc52 bundle2: enforce parttype as alphanumerical
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23590
diff changeset
210
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
211
20877
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
212 def _makefpartparamsizes(nbparams):
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
213 """return a struct format to read part parameter sizes
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
214
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
215 The number parameters is variable so we need to build that format
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
216 dynamically.
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
217 """
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
218 return b'>' + (b'BB' * nbparams)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
219
20804
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
220
20890
ec7fc110faee bundle2: introduce a `parthandler` decorator
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20889
diff changeset
221 parthandlermapping = {}
20889
deed5edb72de bundle2: first version of a bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20887
diff changeset
222
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
223
21623
5b26d82e4e2a bundle2: make it possible to declare params handled by a part handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21622
diff changeset
224 def parthandler(parttype, params=()):
20890
ec7fc110faee bundle2: introduce a `parthandler` decorator
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20889
diff changeset
225 """decorator that register a function as a bundle2 part handler
ec7fc110faee bundle2: introduce a `parthandler` decorator
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20889
diff changeset
226
ec7fc110faee bundle2: introduce a `parthandler` decorator
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20889
diff changeset
227 eg::
ec7fc110faee bundle2: introduce a `parthandler` decorator
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20889
diff changeset
228
21624
d61066d787c8 bundle2: declare supported parameters for all handlers
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21623
diff changeset
229 @parthandler('myparttype', ('mandatory', 'param', 'handled'))
20890
ec7fc110faee bundle2: introduce a `parthandler` decorator
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20889
diff changeset
230 def myparttypehandler(...):
ec7fc110faee bundle2: introduce a `parthandler` decorator
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20889
diff changeset
231 '''process a part of type "my part".'''
ec7fc110faee bundle2: introduce a `parthandler` decorator
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20889
diff changeset
232 ...
ec7fc110faee bundle2: introduce a `parthandler` decorator
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20889
diff changeset
233 """
23868
405eacbecc52 bundle2: enforce parttype as alphanumerical
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23590
diff changeset
234 validateparttype(parttype)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
235
20890
ec7fc110faee bundle2: introduce a `parthandler` decorator
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20889
diff changeset
236 def _decorator(func):
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
237 lparttype = parttype.lower() # enforce lower case matching.
20891
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20890
diff changeset
238 assert lparttype not in parthandlermapping
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20890
diff changeset
239 parthandlermapping[lparttype] = func
21623
5b26d82e4e2a bundle2: make it possible to declare params handled by a part handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21622
diff changeset
240 func.params = frozenset(params)
20890
ec7fc110faee bundle2: introduce a `parthandler` decorator
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20889
diff changeset
241 return func
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
242
20890
ec7fc110faee bundle2: introduce a `parthandler` decorator
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20889
diff changeset
243 return _decorator
20889
deed5edb72de bundle2: first version of a bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20887
diff changeset
244
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
245
20949
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
246 class unbundlerecords(object):
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
247 """keep record of what happens during and unbundle
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
248
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
249 New records are added using `records.add('cat', obj)`. Where 'cat' is a
21024
7731a2281cf0 spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents: 21020
diff changeset
250 category of record and obj is an arbitrary object.
20949
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
251
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
252 `records['cat']` will return all entries of this category 'cat'.
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
253
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
254 Iterating on the object itself will yield `('category', obj)` tuples
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
255 for all entries.
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
256
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
257 All iterations happens in chronological order.
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
258 """
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
259
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
260 def __init__(self):
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
261 self._categories = {}
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
262 self._sequences = []
20996
ed3c5e18a047 bundle2: add reply awareness to unbundlerecords
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20995
diff changeset
263 self._replies = {}
20949
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
264
20996
ed3c5e18a047 bundle2: add reply awareness to unbundlerecords
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20995
diff changeset
265 def add(self, category, entry, inreplyto=None):
20949
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
266 """add a new record of a given category.
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
267
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
268 The entry can then be retrieved in the list returned by
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
269 self['category']."""
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
270 self._categories.setdefault(category, []).append(entry)
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
271 self._sequences.append((category, entry))
20996
ed3c5e18a047 bundle2: add reply awareness to unbundlerecords
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20995
diff changeset
272 if inreplyto is not None:
ed3c5e18a047 bundle2: add reply awareness to unbundlerecords
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20995
diff changeset
273 self.getreplies(inreplyto).add(category, entry)
ed3c5e18a047 bundle2: add reply awareness to unbundlerecords
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20995
diff changeset
274
ed3c5e18a047 bundle2: add reply awareness to unbundlerecords
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20995
diff changeset
275 def getreplies(self, partid):
23139
e53f6b72a0e4 spelling: fixes from proofreading of spell checker issues
Mads Kiilerich <madski@unity3d.com>
parents: 23067
diff changeset
276 """get the records that are replies to a specific part"""
20996
ed3c5e18a047 bundle2: add reply awareness to unbundlerecords
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20995
diff changeset
277 return self._replies.setdefault(partid, unbundlerecords())
20949
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
278
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
279 def __getitem__(self, cat):
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
280 return tuple(self._categories.get(cat, ()))
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
281
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
282 def __iter__(self):
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
283 return iter(self._sequences)
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
284
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
285 def __len__(self):
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
286 return len(self._sequences)
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
287
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
288 def __nonzero__(self):
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
289 return bool(self._sequences)
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
290
31476
413b44003462 py3: add __bool__ to every class defining __nonzero__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30965
diff changeset
291 __bool__ = __nonzero__
413b44003462 py3: add __bool__ to every class defining __nonzero__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30965
diff changeset
292
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
293
20948
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
294 class bundleoperation(object):
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
295 """an object that represents a single bundling process
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
296
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
297 Its purpose is to carry unbundle-related objects and states.
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
298
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
299 A new object should be created at the beginning of each bundle processing.
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
300 The object is to be returned by the processing function.
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
301
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
302 The object has very little content now it will ultimately contain:
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
303 * an access to the repo the bundle is applied to,
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
304 * a ui object,
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
305 * a way to retrieve a transaction to add changes to the repo,
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
306 * a way to record the result of processing each part,
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
307 * a way to construct a bundle response when applicable.
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
308 """
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
309
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
310 def __init__(self, repo, transactiongetter, captureoutput=True, source=b''):
20948
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
311 self.repo = repo
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
312 self.ui = repo.ui
20949
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
313 self.records = unbundlerecords()
20997
d7df4b7378ae bundle2: produce a bundle2 reply
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20996
diff changeset
314 self.reply = None
24878
e530cde6d115 bundle2: disable ouput capture unless we use http (issue4613 issue4615)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24851
diff changeset
315 self.captureoutput = captureoutput
33664
f3407d56a6e8 bundle2: add the capability to store hookargs on bundle operation object
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33541
diff changeset
316 self.hookargs = {}
33665
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33664
diff changeset
317 self._gettransaction = transactiongetter
35265
1f30cbac34b6 bundle2: add a 'modes' dictionary to the bundle operation
Boris Feld <boris.feld@octobus.net>
parents: 35261
diff changeset
318 # carries value that can modify part behavior
1f30cbac34b6 bundle2: add a 'modes' dictionary to the bundle operation
Boris Feld <boris.feld@octobus.net>
parents: 35261
diff changeset
319 self.modes = {}
37235
7e906d8a825f bundle2: add 'source' atrribute to bundleoperation class
Pulkit Goyal <7895pulkit@gmail.com>
parents: 37167
diff changeset
320 self.source = source
33665
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33664
diff changeset
321
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33664
diff changeset
322 def gettransaction(self):
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33664
diff changeset
323 transaction = self._gettransaction()
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33664
diff changeset
324
33810
b3a41f4d837b bundle2: relax the condition to update transaction.hookargs
Yuya Nishihara <yuya@tcha.org>
parents: 33809
diff changeset
325 if self.hookargs:
33665
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33664
diff changeset
326 # the ones added to the transaction supercede those added
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33664
diff changeset
327 # to the operation.
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33664
diff changeset
328 self.hookargs.update(transaction.hookargs)
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33664
diff changeset
329 transaction.hookargs = self.hookargs
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33664
diff changeset
330
33810
b3a41f4d837b bundle2: relax the condition to update transaction.hookargs
Yuya Nishihara <yuya@tcha.org>
parents: 33809
diff changeset
331 # mark the hookargs as flushed. further attempts to add to
b3a41f4d837b bundle2: relax the condition to update transaction.hookargs
Yuya Nishihara <yuya@tcha.org>
parents: 33809
diff changeset
332 # hookargs will result in an abort.
b3a41f4d837b bundle2: relax the condition to update transaction.hookargs
Yuya Nishihara <yuya@tcha.org>
parents: 33809
diff changeset
333 self.hookargs = None
33665
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33664
diff changeset
334
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33664
diff changeset
335 return transaction
33664
f3407d56a6e8 bundle2: add the capability to store hookargs on bundle operation object
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33541
diff changeset
336
f3407d56a6e8 bundle2: add the capability to store hookargs on bundle operation object
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33541
diff changeset
337 def addhookargs(self, hookargs):
33665
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33664
diff changeset
338 if self.hookargs is None:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
339 raise error.ProgrammingError(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
340 b'attempted to add hookargs to '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
341 b'operation after transaction started'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
342 )
33664
f3407d56a6e8 bundle2: add the capability to store hookargs on bundle operation object
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33541
diff changeset
343 self.hookargs.update(hookargs)
20948
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
344
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
345
20952
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20950
diff changeset
346 class TransactionUnavailable(RuntimeError):
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20950
diff changeset
347 pass
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20950
diff changeset
348
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
349
20952
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20950
diff changeset
350 def _notransaction():
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20950
diff changeset
351 """default method to get a transaction while processing a bundle
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20950
diff changeset
352
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20950
diff changeset
353 Raise an exception to highlight the fact that no transaction was expected
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20950
diff changeset
354 to be created"""
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20950
diff changeset
355 raise TransactionUnavailable()
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20950
diff changeset
356
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
357
37238
684a6a261f30 bundle2: make source a mandatory argument for bundle2.applybundle() (API)
Pulkit Goyal <7895pulkit@gmail.com>
parents: 37237
diff changeset
358 def applybundle(repo, unbundler, tr, source, url=None, **kwargs):
26790
28a6c2d72097 bundle2: introduce an "applybundle" function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26698
diff changeset
359 # transform me into unbundler.apply() as soon as the freeze is lifted
33043
18c2489ac96d bundle: make applybundle() delegate v1 bundles to applybundle1()
Martin von Zweigbergk <martinvonz@google.com>
parents: 33040
diff changeset
360 if isinstance(unbundler, unbundle20):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
361 tr.hookargs[b'bundle2'] = b'1'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
362 if source is not None and b'source' not in tr.hookargs:
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
363 tr.hookargs[b'source'] = source
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
364 if url is not None and b'url' not in tr.hookargs:
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
365 tr.hookargs[b'url'] = url
37236
9041c91561fc bunlde2: add 'source' as an optional argument to processbundle()
Pulkit Goyal <7895pulkit@gmail.com>
parents: 37235
diff changeset
366 return processbundle(repo, unbundler, lambda: tr, source=source)
33043
18c2489ac96d bundle: make applybundle() delegate v1 bundles to applybundle1()
Martin von Zweigbergk <martinvonz@google.com>
parents: 33040
diff changeset
367 else:
33044
1d2b68952950 bundle: inline applybundle1()
Martin von Zweigbergk <martinvonz@google.com>
parents: 33043
diff changeset
368 # the transactiongetter won't be used, but we might as well set it
37237
f7d3915d5b3a bundleoperation: pass the source argument from all the users
Pulkit Goyal <7895pulkit@gmail.com>
parents: 37236
diff changeset
369 op = bundleoperation(repo, lambda: tr, source=source)
33044
1d2b68952950 bundle: inline applybundle1()
Martin von Zweigbergk <martinvonz@google.com>
parents: 33043
diff changeset
370 _processchangegroup(op, unbundler, tr, source, url, **kwargs)
1d2b68952950 bundle: inline applybundle1()
Martin von Zweigbergk <martinvonz@google.com>
parents: 33043
diff changeset
371 return op
26790
28a6c2d72097 bundle2: introduce an "applybundle" function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26698
diff changeset
372
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
373
34150
e9e0e1143fc5 bundle2: move part iterator a separate class
Durham Goode <durham@fb.com>
parents: 34101
diff changeset
374 class partiterator(object):
34152
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
375 def __init__(self, repo, op, unbundler):
34151
550343626bb2 bundle2: move part counter to partiterator
Durham Goode <durham@fb.com>
parents: 34150
diff changeset
376 self.repo = repo
34152
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
377 self.op = op
34150
e9e0e1143fc5 bundle2: move part iterator a separate class
Durham Goode <durham@fb.com>
parents: 34101
diff changeset
378 self.unbundler = unbundler
34151
550343626bb2 bundle2: move part counter to partiterator
Durham Goode <durham@fb.com>
parents: 34150
diff changeset
379 self.iterator = None
550343626bb2 bundle2: move part counter to partiterator
Durham Goode <durham@fb.com>
parents: 34150
diff changeset
380 self.count = 0
34258
e71890f27767 bundle2: move processpart stream maintenance into part iterator
Durham Goode <durham@fb.com>
parents: 34249
diff changeset
381 self.current = None
34150
e9e0e1143fc5 bundle2: move part iterator a separate class
Durham Goode <durham@fb.com>
parents: 34101
diff changeset
382
e9e0e1143fc5 bundle2: move part iterator a separate class
Durham Goode <durham@fb.com>
parents: 34101
diff changeset
383 def __enter__(self):
34151
550343626bb2 bundle2: move part counter to partiterator
Durham Goode <durham@fb.com>
parents: 34150
diff changeset
384 def func():
42931
181ee2118a96 bundle2: fix an off-by-one in debug message of number of parts
Martin von Zweigbergk <martinvonz@google.com>
parents: 42894
diff changeset
385 itr = enumerate(self.unbundler.iterparts(), 1)
34151
550343626bb2 bundle2: move part counter to partiterator
Durham Goode <durham@fb.com>
parents: 34150
diff changeset
386 for count, p in itr:
550343626bb2 bundle2: move part counter to partiterator
Durham Goode <durham@fb.com>
parents: 34150
diff changeset
387 self.count = count
34258
e71890f27767 bundle2: move processpart stream maintenance into part iterator
Durham Goode <durham@fb.com>
parents: 34249
diff changeset
388 self.current = p
34151
550343626bb2 bundle2: move part counter to partiterator
Durham Goode <durham@fb.com>
parents: 34150
diff changeset
389 yield p
35114
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35113
diff changeset
390 p.consume()
34258
e71890f27767 bundle2: move processpart stream maintenance into part iterator
Durham Goode <durham@fb.com>
parents: 34249
diff changeset
391 self.current = None
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
392
34151
550343626bb2 bundle2: move part counter to partiterator
Durham Goode <durham@fb.com>
parents: 34150
diff changeset
393 self.iterator = func()
550343626bb2 bundle2: move part counter to partiterator
Durham Goode <durham@fb.com>
parents: 34150
diff changeset
394 return self.iterator
34150
e9e0e1143fc5 bundle2: move part iterator a separate class
Durham Goode <durham@fb.com>
parents: 34101
diff changeset
395
34152
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
396 def __exit__(self, type, exc, tb):
34151
550343626bb2 bundle2: move part counter to partiterator
Durham Goode <durham@fb.com>
parents: 34150
diff changeset
397 if not self.iterator:
550343626bb2 bundle2: move part counter to partiterator
Durham Goode <durham@fb.com>
parents: 34150
diff changeset
398 return
550343626bb2 bundle2: move part counter to partiterator
Durham Goode <durham@fb.com>
parents: 34150
diff changeset
399
34637
5f79f5f8487a bundle2: immediate exit for ctrl+c (issue5692)
Durham Goode <durham@fb.com>
parents: 34323
diff changeset
400 # Only gracefully abort in a normal exception situation. User aborts
5f79f5f8487a bundle2: immediate exit for ctrl+c (issue5692)
Durham Goode <durham@fb.com>
parents: 34323
diff changeset
401 # like Ctrl+C throw a KeyboardInterrupt which is not a base Exception,
5f79f5f8487a bundle2: immediate exit for ctrl+c (issue5692)
Durham Goode <durham@fb.com>
parents: 34323
diff changeset
402 # and should not gracefully cleanup.
5f79f5f8487a bundle2: immediate exit for ctrl+c (issue5692)
Durham Goode <durham@fb.com>
parents: 34323
diff changeset
403 if isinstance(exc, Exception):
34152
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
404 # Any exceptions seeking to the end of the bundle at this point are
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
405 # almost certainly related to the underlying stream being bad.
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
406 # And, chances are that the exception we're handling is related to
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
407 # getting in that bad state. So, we swallow the seeking error and
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
408 # re-raise the original error.
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
409 seekerror = False
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
410 try:
34637
5f79f5f8487a bundle2: immediate exit for ctrl+c (issue5692)
Durham Goode <durham@fb.com>
parents: 34323
diff changeset
411 if self.current:
5f79f5f8487a bundle2: immediate exit for ctrl+c (issue5692)
Durham Goode <durham@fb.com>
parents: 34323
diff changeset
412 # consume the part content to not corrupt the stream.
35114
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35113
diff changeset
413 self.current.consume()
34637
5f79f5f8487a bundle2: immediate exit for ctrl+c (issue5692)
Durham Goode <durham@fb.com>
parents: 34323
diff changeset
414
34152
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
415 for part in self.iterator:
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
416 # consume the bundle content
35114
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35113
diff changeset
417 part.consume()
34152
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
418 except Exception:
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
419 seekerror = True
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
420
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
421 # Small hack to let caller code distinguish exceptions from bundle2
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
422 # processing from processing the old format. This is mostly needed
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
423 # to handle different return codes to unbundle according to the type
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
424 # of bundle. We should probably clean up or drop this return code
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
425 # craziness in a future version.
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
426 exc.duringunbundle2 = True
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
427 salvaged = []
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
428 replycaps = None
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
429 if self.op.reply is not None:
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
430 salvaged = self.op.reply.salvageoutput()
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
431 replycaps = self.op.reply.capabilities
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
432 exc._replycaps = replycaps
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
433 exc._bundle2salvagedoutput = salvaged
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
434
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
435 # Re-raising from a variable loses the original stack. So only use
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
436 # that form if we need to.
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
437 if seekerror:
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
438 raise exc
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
439
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
440 self.repo.ui.debug(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
441 b'bundle2-input-bundle: %i parts total\n' % self.count
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
442 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
443
34150
e9e0e1143fc5 bundle2: move part iterator a separate class
Durham Goode <durham@fb.com>
parents: 34101
diff changeset
444
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
445 def processbundle(repo, unbundler, transactiongetter=None, op=None, source=b''):
20889
deed5edb72de bundle2: first version of a bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20887
diff changeset
446 """This function process a bundle, apply effect to/from a repo
deed5edb72de bundle2: first version of a bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20887
diff changeset
447
20947
c33d7bf53812 bundle2: feed a unbundle20 to the `processbundle` function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20892
diff changeset
448 It iterates over each part then searches for and uses the proper handling
c33d7bf53812 bundle2: feed a unbundle20 to the `processbundle` function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20892
diff changeset
449 code to process the part. Parts are processed in order.
20889
deed5edb72de bundle2: first version of a bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20887
diff changeset
450
20891
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20890
diff changeset
451 Unknown Mandatory part will abort the process.
24851
df0ce98c882f bundle2: also save output when error happens during part processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24849
diff changeset
452
df0ce98c882f bundle2: also save output when error happens during part processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24849
diff changeset
453 It is temporarily possible to provide a prebuilt bundleoperation to the
df0ce98c882f bundle2: also save output when error happens during part processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24849
diff changeset
454 function. This is used to ensure output is properly propagated in case of
df0ce98c882f bundle2: also save output when error happens during part processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24849
diff changeset
455 an error during the unbundling. This output capturing part will likely be
df0ce98c882f bundle2: also save output when error happens during part processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24849
diff changeset
456 reworked and this ability will probably go away in the process.
20889
deed5edb72de bundle2: first version of a bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20887
diff changeset
457 """
24851
df0ce98c882f bundle2: also save output when error happens during part processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24849
diff changeset
458 if op is None:
df0ce98c882f bundle2: also save output when error happens during part processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24849
diff changeset
459 if transactiongetter is None:
df0ce98c882f bundle2: also save output when error happens during part processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24849
diff changeset
460 transactiongetter = _notransaction
37237
f7d3915d5b3a bundleoperation: pass the source argument from all the users
Pulkit Goyal <7895pulkit@gmail.com>
parents: 37236
diff changeset
461 op = bundleoperation(repo, transactiongetter, source=source)
20889
deed5edb72de bundle2: first version of a bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20887
diff changeset
462 # todo:
deed5edb72de bundle2: first version of a bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20887
diff changeset
463 # - replace this is a init function soon.
deed5edb72de bundle2: first version of a bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20887
diff changeset
464 # - exception catching
deed5edb72de bundle2: first version of a bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20887
diff changeset
465 unbundler.params
25331
53ca47ef645d bundle2: add generic debug output regarding processed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25324
diff changeset
466 if repo.ui.debugflag:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
467 msg = [b'bundle2-input-bundle:']
25331
53ca47ef645d bundle2: add generic debug output regarding processed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25324
diff changeset
468 if unbundler.params:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
469 msg.append(b' %i params' % len(unbundler.params))
33770
119e1c6be1ce bundle2: fix transaction availability detection
Boris Feld <boris.feld@octobus.net>
parents: 33711
diff changeset
470 if op._gettransaction is None or op._gettransaction is _notransaction:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
471 msg.append(b' no-transaction')
25331
53ca47ef645d bundle2: add generic debug output regarding processed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25324
diff changeset
472 else:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
473 msg.append(b' with-transaction')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
474 msg.append(b'\n')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
475 repo.ui.debug(b''.join(msg))
34150
e9e0e1143fc5 bundle2: move part iterator a separate class
Durham Goode <durham@fb.com>
parents: 34101
diff changeset
476
34261
f010097c885c bundle2: move part processing to a separate function
Durham Goode <durham@fb.com>
parents: 34260
diff changeset
477 processparts(repo, op, unbundler)
f010097c885c bundle2: move part processing to a separate function
Durham Goode <durham@fb.com>
parents: 34260
diff changeset
478
f010097c885c bundle2: move part processing to a separate function
Durham Goode <durham@fb.com>
parents: 34260
diff changeset
479 return op
f010097c885c bundle2: move part processing to a separate function
Durham Goode <durham@fb.com>
parents: 34260
diff changeset
480
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
481
34261
f010097c885c bundle2: move part processing to a separate function
Durham Goode <durham@fb.com>
parents: 34260
diff changeset
482 def processparts(repo, op, unbundler):
34152
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
483 with partiterator(repo, op, unbundler) as parts:
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
484 for part in parts:
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
485 _processpart(op, part)
25332
0cdd1c811705 bundle2: add generic debug output at the end of bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25331
diff changeset
486
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
487
33038
f0efd2bffe1e bundle: extract _processchangegroup() method
Martin von Zweigbergk <martinvonz@google.com>
parents: 33037
diff changeset
488 def _processchangegroup(op, cg, tr, source, url, **kwargs):
33461
bb72031f0ea8 changegroup: stop returning and recording added nodes in 'cg.apply'
Boris Feld <boris.feld@octobus.net>
parents: 33460
diff changeset
489 ret = cg.apply(op.repo, tr, source, url, **kwargs)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
490 op.records.add(b'changegroup', {b'return': ret,})
33038
f0efd2bffe1e bundle: extract _processchangegroup() method
Martin von Zweigbergk <martinvonz@google.com>
parents: 33037
diff changeset
491 return ret
f0efd2bffe1e bundle: extract _processchangegroup() method
Martin von Zweigbergk <martinvonz@google.com>
parents: 33037
diff changeset
492
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
493
34259
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34258
diff changeset
494 def _gethandler(op, part):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
495 status = b'unknown' # used by debug output
34259
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34258
diff changeset
496 try:
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34258
diff changeset
497 handler = parthandlermapping.get(part.type)
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34258
diff changeset
498 if handler is None:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
499 status = b'unsupported-type'
34259
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34258
diff changeset
500 raise error.BundleUnknownFeatureError(parttype=part.type)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
501 indebug(op.ui, b'found a handler for part %s' % part.type)
34259
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34258
diff changeset
502 unknownparams = part.mandatorykeys - handler.params
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34258
diff changeset
503 if unknownparams:
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34258
diff changeset
504 unknownparams = list(unknownparams)
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34258
diff changeset
505 unknownparams.sort()
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
506 status = b'unsupported-params (%s)' % b', '.join(unknownparams)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
507 raise error.BundleUnknownFeatureError(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
508 parttype=part.type, params=unknownparams
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
509 )
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
510 status = b'supported'
34259
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34258
diff changeset
511 except error.BundleUnknownFeatureError as exc:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
512 if part.mandatory: # mandatory parts
34259
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34258
diff changeset
513 raise
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
514 indebug(op.ui, b'ignoring unsupported advisory part %s' % exc)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
515 return # skip to part processing
34259
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34258
diff changeset
516 finally:
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34258
diff changeset
517 if op.ui.debugflag:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
518 msg = [b'bundle2-input-part: "%s"' % part.type]
34259
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34258
diff changeset
519 if not part.mandatory:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
520 msg.append(b' (advisory)')
34259
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34258
diff changeset
521 nbmp = len(part.mandatorykeys)
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34258
diff changeset
522 nbap = len(part.params) - nbmp
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34258
diff changeset
523 if nbmp or nbap:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
524 msg.append(b' (params:')
34259
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34258
diff changeset
525 if nbmp:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
526 msg.append(b' %i mandatory' % nbmp)
34259
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34258
diff changeset
527 if nbap:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
528 msg.append(b' %i advisory' % nbmp)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
529 msg.append(b')')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
530 msg.append(b' %s\n' % status)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
531 op.ui.debug(b''.join(msg))
34259
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34258
diff changeset
532
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34258
diff changeset
533 return handler
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34258
diff changeset
534
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
535
23008
d3137827016a bundle2: extract processing of part into its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23001
diff changeset
536 def _processpart(op, part):
d3137827016a bundle2: extract processing of part into its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23001
diff changeset
537 """process a single part from a bundle
d3137827016a bundle2: extract processing of part into its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23001
diff changeset
538
d3137827016a bundle2: extract processing of part into its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23001
diff changeset
539 The part is guaranteed to have been fully consumed when the function exits
d3137827016a bundle2: extract processing of part into its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23001
diff changeset
540 (even if an exception is raised)."""
34260
cc7b37c90616 bundle2: remove unnecessary try finally
Durham Goode <durham@fb.com>
parents: 34259
diff changeset
541 handler = _gethandler(op, part)
cc7b37c90616 bundle2: remove unnecessary try finally
Durham Goode <durham@fb.com>
parents: 34259
diff changeset
542 if handler is None:
cc7b37c90616 bundle2: remove unnecessary try finally
Durham Goode <durham@fb.com>
parents: 34259
diff changeset
543 return
23008
d3137827016a bundle2: extract processing of part into its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23001
diff changeset
544
34260
cc7b37c90616 bundle2: remove unnecessary try finally
Durham Goode <durham@fb.com>
parents: 34259
diff changeset
545 # handler is called outside the above try block so that we don't
cc7b37c90616 bundle2: remove unnecessary try finally
Durham Goode <durham@fb.com>
parents: 34259
diff changeset
546 # risk catching KeyErrors from anything other than the
cc7b37c90616 bundle2: remove unnecessary try finally
Durham Goode <durham@fb.com>
parents: 34259
diff changeset
547 # parthandlermapping lookup (any KeyError raised by handler()
cc7b37c90616 bundle2: remove unnecessary try finally
Durham Goode <durham@fb.com>
parents: 34259
diff changeset
548 # itself represents a defect of a different variety).
cc7b37c90616 bundle2: remove unnecessary try finally
Durham Goode <durham@fb.com>
parents: 34259
diff changeset
549 output = None
cc7b37c90616 bundle2: remove unnecessary try finally
Durham Goode <durham@fb.com>
parents: 34259
diff changeset
550 if op.captureoutput and op.reply is not None:
cc7b37c90616 bundle2: remove unnecessary try finally
Durham Goode <durham@fb.com>
parents: 34259
diff changeset
551 op.ui.pushbuffer(error=True, subproc=True)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
552 output = b''
34260
cc7b37c90616 bundle2: remove unnecessary try finally
Durham Goode <durham@fb.com>
parents: 34259
diff changeset
553 try:
cc7b37c90616 bundle2: remove unnecessary try finally
Durham Goode <durham@fb.com>
parents: 34259
diff changeset
554 handler(op, part)
23008
d3137827016a bundle2: extract processing of part into its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23001
diff changeset
555 finally:
34260
cc7b37c90616 bundle2: remove unnecessary try finally
Durham Goode <durham@fb.com>
parents: 34259
diff changeset
556 if output is not None:
cc7b37c90616 bundle2: remove unnecessary try finally
Durham Goode <durham@fb.com>
parents: 34259
diff changeset
557 output = op.ui.popbuffer()
cc7b37c90616 bundle2: remove unnecessary try finally
Durham Goode <durham@fb.com>
parents: 34259
diff changeset
558 if output:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
559 outpart = op.reply.newpart(b'output', data=output, mandatory=False)
34260
cc7b37c90616 bundle2: remove unnecessary try finally
Durham Goode <durham@fb.com>
parents: 34259
diff changeset
560 outpart.addparam(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
561 b'in-reply-to', pycompat.bytestr(part.id), mandatory=False
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
562 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
563
23008
d3137827016a bundle2: extract processing of part into its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23001
diff changeset
564
21138
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
565 def decodecaps(blob):
23139
e53f6b72a0e4 spelling: fixes from proofreading of spell checker issues
Mads Kiilerich <madski@unity3d.com>
parents: 23067
diff changeset
566 """decode a bundle2 caps bytes blob into a dictionary
21138
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
567
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
568 The blob is a list of capabilities (one per line)
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
569 Capabilities may have values using a line of the form::
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
570
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
571 capability=value1,value2,value3
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
572
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
573 The values are always a list."""
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
574 caps = {}
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
575 for line in blob.splitlines():
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
576 if not line:
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
577 continue
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
578 if b'=' not in line:
21138
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
579 key, vals = line, ()
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
580 else:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
581 key, vals = line.split(b'=', 1)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
582 vals = vals.split(b',')
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28672
diff changeset
583 key = urlreq.unquote(key)
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28672
diff changeset
584 vals = [urlreq.unquote(v) for v in vals]
21138
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
585 caps[key] = vals
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
586 return caps
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
587
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
588
21139
2b8c82f7f11d bundle2: capabilities encoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21138
diff changeset
589 def encodecaps(caps):
2b8c82f7f11d bundle2: capabilities encoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21138
diff changeset
590 """encode a bundle2 caps dictionary into a bytes blob"""
2b8c82f7f11d bundle2: capabilities encoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21138
diff changeset
591 chunks = []
2b8c82f7f11d bundle2: capabilities encoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21138
diff changeset
592 for ca in sorted(caps):
2b8c82f7f11d bundle2: capabilities encoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21138
diff changeset
593 vals = caps[ca]
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28672
diff changeset
594 ca = urlreq.quote(ca)
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28672
diff changeset
595 vals = [urlreq.quote(v) for v in vals]
21139
2b8c82f7f11d bundle2: capabilities encoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21138
diff changeset
596 if vals:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
597 ca = b"%s=%s" % (ca, b','.join(vals))
21139
2b8c82f7f11d bundle2: capabilities encoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21138
diff changeset
598 chunks.append(ca)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
599 return b'\n'.join(chunks)
21139
2b8c82f7f11d bundle2: capabilities encoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21138
diff changeset
600
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
601
28666
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
602 bundletypes = {
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
603 b"": (b"", b'UN'), # only when using unbundle on ssh and old http servers
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
604 # since the unification ssh accepts a header but there
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
605 # is no capability signaling it.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
606 b"HG20": (), # special-cased below
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
607 b"HG10UN": (b"HG10UN", b'UN'),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
608 b"HG10BZ": (b"HG10", b'BZ'),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
609 b"HG10GZ": (b"HG10GZ", b'GZ'),
28666
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
610 }
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
611
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
612 # hgweb uses this list to communicate its preferred type
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
613 bundlepriority = [b'HG10GZ', b'HG10BZ', b'HG10UN']
28666
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
614
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
615
20801
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
616 class bundle20(object):
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
617 """represent an outgoing bundle2 container
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
618
21599
57cd844d7a5b bundle2: have ``newpart`` automatically add the part to the bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21598
diff changeset
619 Use the `addparam` method to add stream level parameter. and `newpart` to
20856
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
620 populate it. Then call `getchunks` to retrieve all the binary chunks of
21024
7731a2281cf0 spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents: 21020
diff changeset
621 data that compose the bundle2 container."""
20801
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
622
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
623 _magicstring = b'HG20'
24640
685639f9430d bundle20: move magic string into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24071
diff changeset
624
21134
2f8c4fa237f5 bundle2: adds a capabilities attribute on bundler20
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21133
diff changeset
625 def __init__(self, ui, capabilities=()):
20842
938718d72624 bundle2: print debug information during bundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20814
diff changeset
626 self.ui = ui
20801
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
627 self._params = []
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
628 self._parts = []
21136
b6fd496e5c72 bundle2: support for capabilities with values
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21135
diff changeset
629 self.capabilities = dict(capabilities)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
630 self._compengine = util.compengines.forbundletype(b'UN')
30757
511a4bf52754 bundle2: allow compression options to be passed to compressor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30439
diff changeset
631 self._compopts = None
35787
a84dbc87dae9 exchange: send bundle2 stream clones uncompressed
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35783
diff changeset
632 # If compression is being handled by a consumer of the raw
a84dbc87dae9 exchange: send bundle2 stream clones uncompressed
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35783
diff changeset
633 # data (e.g. the wire protocol), unsetting this flag tells
a84dbc87dae9 exchange: send bundle2 stream clones uncompressed
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35783
diff changeset
634 # consumers that the bundle is best left uncompressed.
a84dbc87dae9 exchange: send bundle2 stream clones uncompressed
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35783
diff changeset
635 self.prefercompressed = True
26404
795f02a24b87 bundle2: allow compressed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26396
diff changeset
636
30757
511a4bf52754 bundle2: allow compression options to be passed to compressor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30439
diff changeset
637 def setcompression(self, alg, compopts=None):
26404
795f02a24b87 bundle2: allow compressed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26396
diff changeset
638 """setup core part compression to <alg>"""
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
639 if alg in (None, b'UN'):
26404
795f02a24b87 bundle2: allow compressed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26396
diff changeset
640 return
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
641 assert not any(n.lower() == b'compression' for n, v in self._params)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
642 self.addparam(b'Compression', alg)
30351
f81002f736d7 bundle2: use new compression engine API for compression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30177
diff changeset
643 self._compengine = util.compengines.forbundletype(alg)
30757
511a4bf52754 bundle2: allow compression options to be passed to compressor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30439
diff changeset
644 self._compopts = compopts
20801
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
645
21900
b8bd97085ec9 bundle2: add a ``bundle20.nbparts`` property
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21660
diff changeset
646 @property
b8bd97085ec9 bundle2: add a ``bundle20.nbparts`` property
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21660
diff changeset
647 def nbparts(self):
b8bd97085ec9 bundle2: add a ``bundle20.nbparts`` property
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21660
diff changeset
648 """total number of parts added to the bundler"""
b8bd97085ec9 bundle2: add a ``bundle20.nbparts`` property
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21660
diff changeset
649 return len(self._parts)
b8bd97085ec9 bundle2: add a ``bundle20.nbparts`` property
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21660
diff changeset
650
21597
1daad9dcdba2 bundle2: small doc update on the bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21186
diff changeset
651 # methods used to defines the bundle2 content
20804
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
652 def addparam(self, name, value=None):
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
653 """add a stream level parameter"""
20813
8c74b3ce5b70 bundle2: refuse empty parameter name
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20812
diff changeset
654 if not name:
38609
d474b3b44d4f bundle2: use ProgrammingError to report bad use of addparam()
Yuya Nishihara <yuya@tcha.org>
parents: 37842
diff changeset
655 raise error.ProgrammingError(b'empty parameter name')
43488
ea25b4673231 bundle2: more pytype suppressions around bytestring
Augie Fackler <augie@google.com>
parents: 43131
diff changeset
656 if name[0:1] not in pycompat.bytestr(
ea25b4673231 bundle2: more pytype suppressions around bytestring
Augie Fackler <augie@google.com>
parents: 43131
diff changeset
657 string.ascii_letters # pytype: disable=wrong-arg-types
ea25b4673231 bundle2: more pytype suppressions around bytestring
Augie Fackler <augie@google.com>
parents: 43131
diff changeset
658 ):
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
659 raise error.ProgrammingError(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
660 b'non letter first character: %s' % name
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
661 )
20804
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
662 self._params.append((name, value))
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
663
20856
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
664 def addpart(self, part):
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
665 """add a new part to the bundle2 container
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
666
21024
7731a2281cf0 spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents: 21020
diff changeset
667 Parts contains the actual applicative payload."""
20995
e995d104c87f bundle2: add an integer id to part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20952
diff changeset
668 assert part.id is None
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
669 part.id = len(self._parts) # very cheap counter
20856
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
670 self._parts.append(part)
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
671
21598
1b0dbb91de5b bundle2: add a ``newpart`` method to ``bundle20``
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21597
diff changeset
672 def newpart(self, typeid, *args, **kwargs):
21602
cc33ae50bab3 bundle2: warn about error during initialization in ``newpart`` docstring
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21601
diff changeset
673 """create a new part and add it to the containers
cc33ae50bab3 bundle2: warn about error during initialization in ``newpart`` docstring
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21601
diff changeset
674
cc33ae50bab3 bundle2: warn about error during initialization in ``newpart`` docstring
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21601
diff changeset
675 As the part is directly added to the containers. For now, this means
cc33ae50bab3 bundle2: warn about error during initialization in ``newpart`` docstring
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21601
diff changeset
676 that any failure to properly initialize the part after calling
cc33ae50bab3 bundle2: warn about error during initialization in ``newpart`` docstring
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21601
diff changeset
677 ``newpart`` should result in a failure of the whole bundling process.
cc33ae50bab3 bundle2: warn about error during initialization in ``newpart`` docstring
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21601
diff changeset
678
cc33ae50bab3 bundle2: warn about error during initialization in ``newpart`` docstring
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21601
diff changeset
679 You can still fall back to manually create and add if you need better
cc33ae50bab3 bundle2: warn about error during initialization in ``newpart`` docstring
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21601
diff changeset
680 control."""
21598
1b0dbb91de5b bundle2: add a ``newpart`` method to ``bundle20``
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21597
diff changeset
681 part = bundlepart(typeid, *args, **kwargs)
21599
57cd844d7a5b bundle2: have ``newpart`` automatically add the part to the bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21598
diff changeset
682 self.addpart(part)
21598
1b0dbb91de5b bundle2: add a ``newpart`` method to ``bundle20``
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21597
diff changeset
683 return part
1b0dbb91de5b bundle2: add a ``newpart`` method to ``bundle20``
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21597
diff changeset
684
21597
1daad9dcdba2 bundle2: small doc update on the bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21186
diff changeset
685 # methods used to generate the bundle2 stream
20801
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
686 def getchunks(self):
25322
1f7c0403f8be bundle2: add generic debug output regarding generated bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25321
diff changeset
687 if self.ui.debugflag:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
688 msg = [b'bundle2-output-bundle: "%s",' % self._magicstring]
25322
1f7c0403f8be bundle2: add generic debug output regarding generated bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25321
diff changeset
689 if self._params:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
690 msg.append(b' (%i params)' % len(self._params))
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
691 msg.append(b' %i parts total\n' % len(self._parts))
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
692 self.ui.debug(b''.join(msg))
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
693 outdebug(self.ui, b'start emission of %s stream' % self._magicstring)
24640
685639f9430d bundle20: move magic string into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24071
diff changeset
694 yield self._magicstring
20804
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
695 param = self._paramchunk()
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
696 outdebug(self.ui, b'bundle parameter: %s' % param)
20804
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
697 yield _pack(_fstreamparamsize, len(param))
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
698 if param:
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
699 yield param
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
700 for chunk in self._compengine.compressstream(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
701 self._getcorechunk(), self._compopts
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
702 ):
30357
5925bda42dbd bundle2: use compressstream compression engine API
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30353
diff changeset
703 yield chunk
20802
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
704
20804
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
705 def _paramchunk(self):
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
706 """return a encoded version of all stream parameters"""
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
707 blocks = []
20809
b93bb639451a bundle2: support for bundling parameter value
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20808
diff changeset
708 for par, value in self._params:
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28672
diff changeset
709 par = urlreq.quote(par)
20809
b93bb639451a bundle2: support for bundling parameter value
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20808
diff changeset
710 if value is not None:
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28672
diff changeset
711 value = urlreq.quote(value)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
712 par = b'%s=%s' % (par, value)
20809
b93bb639451a bundle2: support for bundling parameter value
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20808
diff changeset
713 blocks.append(par)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
714 return b' '.join(blocks)
20804
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
715
26396
d90c30801cdf bundle20: extract core payload generation in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26395
diff changeset
716 def _getcorechunk(self):
d90c30801cdf bundle20: extract core payload generation in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26395
diff changeset
717 """yield chunk for the core part of the bundle
d90c30801cdf bundle20: extract core payload generation in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26395
diff changeset
718
d90c30801cdf bundle20: extract core payload generation in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26395
diff changeset
719 (all but headers and parameters)"""
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
720 outdebug(self.ui, b'start of parts')
26396
d90c30801cdf bundle20: extract core payload generation in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26395
diff changeset
721 for part in self._parts:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
722 outdebug(self.ui, b'bundle part: "%s"' % part.type)
26396
d90c30801cdf bundle20: extract core payload generation in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26395
diff changeset
723 for chunk in part.getchunks(ui=self.ui):
d90c30801cdf bundle20: extract core payload generation in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26395
diff changeset
724 yield chunk
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
725 outdebug(self.ui, b'end of bundle')
26396
d90c30801cdf bundle20: extract core payload generation in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26395
diff changeset
726 yield _pack(_fpartheadersize, 0)
d90c30801cdf bundle20: extract core payload generation in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26395
diff changeset
727
24794
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24793
diff changeset
728 def salvageoutput(self):
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24793
diff changeset
729 """return a list with a copy of all output parts in the bundle
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24793
diff changeset
730
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24793
diff changeset
731 This is meant to be used during error handling to make sure we preserve
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24793
diff changeset
732 server output"""
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24793
diff changeset
733 salvaged = []
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24793
diff changeset
734 for part in self._parts:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
735 if part.type.startswith(b'output'):
24794
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24793
diff changeset
736 salvaged.append(part.copy())
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24793
diff changeset
737 return salvaged
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24793
diff changeset
738
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24793
diff changeset
739
21013
a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21005
diff changeset
740 class unpackermixin(object):
a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21005
diff changeset
741 """A mixin to extract bytes and struct data from a stream"""
20802
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
742
21013
a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21005
diff changeset
743 def __init__(self, fp):
20802
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
744 self._fp = fp
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
745
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
746 def _unpack(self, format):
31862
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31861
diff changeset
747 """unpack this struct format from the stream
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31861
diff changeset
748
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31861
diff changeset
749 This method is meant for internal usage by the bundle2 protocol only.
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31861
diff changeset
750 They directly manipulate the low level stream including bundle2 level
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31861
diff changeset
751 instruction.
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31861
diff changeset
752
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31861
diff changeset
753 Do not use it to implement higher-level logic or methods."""
20802
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
754 data = self._readexact(struct.calcsize(format))
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
755 return _unpack(format, data)
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
756
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
757 def _readexact(self, size):
31862
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31861
diff changeset
758 """read exactly <size> bytes from the stream
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31861
diff changeset
759
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31861
diff changeset
760 This method is meant for internal usage by the bundle2 protocol only.
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31861
diff changeset
761 They directly manipulate the low level stream including bundle2 level
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31861
diff changeset
762 instruction.
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31861
diff changeset
763
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31861
diff changeset
764 Do not use it to implement higher-level logic or methods."""
20802
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
765 return changegroup.readexactly(self._fp, size)
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
766
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
767
25640
39f0064a3079 bundle2.getunbundler: rename "header" to "magicstring"
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25518
diff changeset
768 def getunbundler(ui, fp, magicstring=None):
39f0064a3079 bundle2.getunbundler: rename "header" to "magicstring"
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25518
diff changeset
769 """return a valid unbundler object for a given magicstring"""
39f0064a3079 bundle2.getunbundler: rename "header" to "magicstring"
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25518
diff changeset
770 if magicstring is None:
39f0064a3079 bundle2.getunbundler: rename "header" to "magicstring"
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25518
diff changeset
771 magicstring = changegroup.readexactly(fp, 4)
39f0064a3079 bundle2.getunbundler: rename "header" to "magicstring"
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25518
diff changeset
772 magic, version = magicstring[0:2], magicstring[2:4]
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
773 if magic != b'HG':
33127
0aae80d14fed bundle2: add some debugging information to the not-a-bundle error
Siddharth Agarwal <sid0@fb.com>
parents: 33044
diff changeset
774 ui.debug(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
775 b"error: invalid magic: %r (version %r), should be 'HG'\n"
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
776 % (magic, version)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
777 )
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
778 raise error.Abort(_(b'not a Mercurial bundle'))
24648
5cac3accdaa1 unbundle20: allow generic dispatch between unbundlers
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24642
diff changeset
779 unbundlerclass = formatmap.get(version)
5cac3accdaa1 unbundle20: allow generic dispatch between unbundlers
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24642
diff changeset
780 if unbundlerclass is None:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
781 raise error.Abort(_(b'unknown bundle version %s') % version)
24648
5cac3accdaa1 unbundle20: allow generic dispatch between unbundlers
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24642
diff changeset
782 unbundler = unbundlerclass(ui, fp)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
783 indebug(ui, b'start processing of %s stream' % magicstring)
24642
54e5c239c2d9 unbundle20: move header parsing into the 'getunbundler' function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24641
diff changeset
784 return unbundler
24641
60fecc5b14a4 unbundle20: retrieve unbundler instances through a factory function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24640
diff changeset
785
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
786
21013
a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21005
diff changeset
787 class unbundle20(unpackermixin):
a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21005
diff changeset
788 """interpret a bundle2 stream
a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21005
diff changeset
789
21129
07bcbf326c8d bundle2: use an official iterparts method to unbundle parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21066
diff changeset
790 This class is fed with a binary stream and yields parts through its
07bcbf326c8d bundle2: use an official iterparts method to unbundle parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21066
diff changeset
791 `iterparts` methods."""
21013
a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21005
diff changeset
792
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
793 _magicstring = b'HG20'
26542
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
794
24642
54e5c239c2d9 unbundle20: move header parsing into the 'getunbundler' function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24641
diff changeset
795 def __init__(self, ui, fp):
21066
5ecfe76d0d96 bundle2: make header reading optional
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21062
diff changeset
796 """If header is specified, we do not read it out of the stream."""
21013
a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21005
diff changeset
797 self.ui = ui
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
798 self._compengine = util.compengines.forbundletype(b'UN')
26802
42f705f2c02d bundle2: make unbundle.compressed return True when compressed
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26795
diff changeset
799 self._compressed = None
21013
a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21005
diff changeset
800 super(unbundle20, self).__init__(fp)
a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21005
diff changeset
801
20802
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
802 @util.propertycache
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
803 def params(self):
21024
7731a2281cf0 spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents: 21020
diff changeset
804 """dictionary of stream level parameters"""
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
805 indebug(self.ui, b'reading bundle2 stream parameters')
20805
c5aaeca0cfbf bundle2: support for unbundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20804
diff changeset
806 params = {}
c5aaeca0cfbf bundle2: support for unbundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20804
diff changeset
807 paramssize = self._unpack(_fstreamparamsize)[0]
23011
006a81d07e57 bundle2: detect and disallow a negative chunk size
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23010
diff changeset
808 if paramssize < 0:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
809 raise error.BundleValueError(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
810 b'negative bundle param size: %i' % paramssize
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
811 )
20805
c5aaeca0cfbf bundle2: support for unbundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20804
diff changeset
812 if paramssize:
26541
d40029b4296e bundle2: split parameter retrieval and processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26404
diff changeset
813 params = self._readexact(paramssize)
d40029b4296e bundle2: split parameter retrieval and processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26404
diff changeset
814 params = self._processallparams(params)
20805
c5aaeca0cfbf bundle2: support for unbundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20804
diff changeset
815 return params
20802
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
816
26541
d40029b4296e bundle2: split parameter retrieval and processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26404
diff changeset
817 def _processallparams(self, paramsblock):
d40029b4296e bundle2: split parameter retrieval and processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26404
diff changeset
818 """"""
29591
6215b5537ba5 bundle2: use a sorted dict for holding parameters
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29294
diff changeset
819 params = util.sortdict()
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
820 for p in paramsblock.split(b' '):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
821 p = p.split(b'=', 1)
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28672
diff changeset
822 p = [urlreq.unquote(i) for i in p]
26541
d40029b4296e bundle2: split parameter retrieval and processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26404
diff changeset
823 if len(p) < 2:
d40029b4296e bundle2: split parameter retrieval and processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26404
diff changeset
824 p.append(None)
d40029b4296e bundle2: split parameter retrieval and processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26404
diff changeset
825 self._processparam(*p)
d40029b4296e bundle2: split parameter retrieval and processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26404
diff changeset
826 params[p[0]] = p[1]
d40029b4296e bundle2: split parameter retrieval and processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26404
diff changeset
827 return params
d40029b4296e bundle2: split parameter retrieval and processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26404
diff changeset
828
20844
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
829 def _processparam(self, name, value):
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
830 """process a parameter, applying its effect if needed
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
831
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
832 Parameter starting with a lower case letter are advisory and will be
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
833 ignored when unknown. Those starting with an upper case letter are
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
834 mandatory and will this function will raise a KeyError when unknown.
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
835
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
836 Note: no option are currently supported. Any input will be either
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
837 ignored or failing.
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
838 """
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
839 if not name:
43506
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43495
diff changeset
840 raise ValueError('empty parameter name')
43488
ea25b4673231 bundle2: more pytype suppressions around bytestring
Augie Fackler <augie@google.com>
parents: 43131
diff changeset
841 if name[0:1] not in pycompat.bytestr(
ea25b4673231 bundle2: more pytype suppressions around bytestring
Augie Fackler <augie@google.com>
parents: 43131
diff changeset
842 string.ascii_letters # pytype: disable=wrong-arg-types
ea25b4673231 bundle2: more pytype suppressions around bytestring
Augie Fackler <augie@google.com>
parents: 43131
diff changeset
843 ):
43506
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43495
diff changeset
844 raise ValueError('non letter first character: %s' % name)
26395
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26393
diff changeset
845 try:
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26393
diff changeset
846 handler = b2streamparamsmap[name.lower()]
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26393
diff changeset
847 except KeyError:
34277
a48b3db0baba bundle2: stop using %r to quote part names
Augie Fackler <augie@google.com>
parents: 34261
diff changeset
848 if name[0:1].islower():
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
849 indebug(self.ui, b"ignoring unknown parameter %s" % name)
26395
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26393
diff changeset
850 else:
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26393
diff changeset
851 raise error.BundleUnknownFeatureError(params=(name,))
20844
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
852 else:
26395
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26393
diff changeset
853 handler(self, name, value)
20844
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
854
26542
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
855 def _forwardchunks(self):
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
856 """utility to transfer a bundle2 as binary
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
857
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
858 This is made necessary by the fact the 'getbundle' command over 'ssh'
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
859 have no way to know then the reply end, relying on the bundle to be
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
860 interpreted to know its end. This is terrible and we are sorry, but we
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
861 needed to move forward to get general delta enabled.
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
862 """
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
863 yield self._magicstring
43421
be384a2052aa py3: don't use bytes with vars() or __dict__
Martin von Zweigbergk <martinvonz@google.com>
parents: 43131
diff changeset
864 assert 'params' not in vars(self)
26542
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
865 paramssize = self._unpack(_fstreamparamsize)[0]
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
866 if paramssize < 0:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
867 raise error.BundleValueError(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
868 b'negative bundle param size: %i' % paramssize
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
869 )
26542
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
870 if paramssize:
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
871 params = self._readexact(paramssize)
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
872 self._processallparams(params)
42143
29569f2db929 bundle2: handle compression in _forwardchunks
Joerg Sonnenberger <joerg@bec.de>
parents: 42057
diff changeset
873 # The payload itself is decompressed below, so drop
29569f2db929 bundle2: handle compression in _forwardchunks
Joerg Sonnenberger <joerg@bec.de>
parents: 42057
diff changeset
874 # the compression parameter passed down to compensate.
29569f2db929 bundle2: handle compression in _forwardchunks
Joerg Sonnenberger <joerg@bec.de>
parents: 42057
diff changeset
875 outparams = []
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
876 for p in params.split(b' '):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
877 k, v = p.split(b'=', 1)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
878 if k.lower() != b'compression':
42143
29569f2db929 bundle2: handle compression in _forwardchunks
Joerg Sonnenberger <joerg@bec.de>
parents: 42057
diff changeset
879 outparams.append(p)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
880 outparams = b' '.join(outparams)
42143
29569f2db929 bundle2: handle compression in _forwardchunks
Joerg Sonnenberger <joerg@bec.de>
parents: 42057
diff changeset
881 yield _pack(_fstreamparamsize, len(outparams))
29569f2db929 bundle2: handle compression in _forwardchunks
Joerg Sonnenberger <joerg@bec.de>
parents: 42057
diff changeset
882 yield outparams
29569f2db929 bundle2: handle compression in _forwardchunks
Joerg Sonnenberger <joerg@bec.de>
parents: 42057
diff changeset
883 else:
29569f2db929 bundle2: handle compression in _forwardchunks
Joerg Sonnenberger <joerg@bec.de>
parents: 42057
diff changeset
884 yield _pack(_fstreamparamsize, paramssize)
26542
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
885 # From there, payload might need to be decompressed
30353
d045b4091197 bundle2: use compression engines API to obtain decompressor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30351
diff changeset
886 self._fp = self._compengine.decompressorreader(self._fp)
26542
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
887 emptycount = 0
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
888 while emptycount < 2:
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
889 # so we can brainlessly loop
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
890 assert _fpartheadersize == _fpayloadsize
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
891 size = self._unpack(_fpartheadersize)[0]
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
892 yield _pack(_fpartheadersize, size)
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
893 if size:
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
894 emptycount = 0
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
895 else:
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
896 emptycount += 1
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
897 continue
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
898 if size == flaginterrupt:
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
899 continue
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
900 elif size < 0:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
901 raise error.BundleValueError(b'negative chunk size: %i')
26542
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
902 yield self._readexact(size)
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
903
35116
da91e7309daf bundle2: don't use seekable bundle2 parts by default (issue5691)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35115
diff changeset
904 def iterparts(self, seekable=False):
20802
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
905 """yield all parts contained in the stream"""
35116
da91e7309daf bundle2: don't use seekable bundle2 parts by default (issue5691)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35115
diff changeset
906 cls = seekableunbundlepart if seekable else unbundlepart
20802
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
907 # make sure param have been loaded
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
908 self.params
26404
795f02a24b87 bundle2: allow compressed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26396
diff changeset
909 # From there, payload need to be decompressed
30353
d045b4091197 bundle2: use compression engines API to obtain decompressor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30351
diff changeset
910 self._fp = self._compengine.decompressorreader(self._fp)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
911 indebug(self.ui, b'start extraction of bundle2 parts')
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
912 headerblock = self._readpartheader()
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
913 while headerblock is not None:
35116
da91e7309daf bundle2: don't use seekable bundle2 parts by default (issue5691)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35115
diff changeset
914 part = cls(self.ui, headerblock, self._fp)
20802
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
915 yield part
35114
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35113
diff changeset
916 # Ensure part is fully consumed so we can start reading the next
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35113
diff changeset
917 # part.
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35113
diff changeset
918 part.consume()
35115
2b72bc88043f bundle2: only seek to beginning of part in bundlerepo
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35114
diff changeset
919
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
920 headerblock = self._readpartheader()
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
921 indebug(self.ui, b'end of bundle2 stream')
20802
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
922
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
923 def _readpartheader(self):
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
924 """reads a part header size and return the bytes blob
20864
9a75d2559cff bundle2: support unbundling empty part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20856
diff changeset
925
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
926 returns None if empty"""
20864
9a75d2559cff bundle2: support unbundling empty part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20856
diff changeset
927 headersize = self._unpack(_fpartheadersize)[0]
23011
006a81d07e57 bundle2: detect and disallow a negative chunk size
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23010
diff changeset
928 if headersize < 0:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
929 raise error.BundleValueError(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
930 b'negative part header size: %i' % headersize
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
931 )
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
932 indebug(self.ui, b'part header size: %i' % headersize)
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
933 if headersize:
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
934 return self._readexact(headersize)
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
935 return None
20864
9a75d2559cff bundle2: support unbundling empty part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20856
diff changeset
936
24071
184a2f6f40da bundle2.unbundle20: add compressed() method
Eric Sumner <ericsumner@fb.com>
parents: 24070
diff changeset
937 def compressed(self):
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
938 self.params # load params
26802
42f705f2c02d bundle2: make unbundle.compressed return True when compressed
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26795
diff changeset
939 return self._compressed
20802
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
940
31863
cd7aaf344d83 bundle2: move the 'close' method off the unpackermixin
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31862
diff changeset
941 def close(self):
cd7aaf344d83 bundle2: move the 'close' method off the unpackermixin
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31862
diff changeset
942 """close underlying file"""
43115
4aa72cdf616f py3: delete b'' prefix from safehasattr arguments
Martin von Zweigbergk <martinvonz@google.com>
parents: 43080
diff changeset
943 if util.safehasattr(self._fp, 'close'):
31863
cd7aaf344d83 bundle2: move the 'close' method off the unpackermixin
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31862
diff changeset
944 return self._fp.close()
cd7aaf344d83 bundle2: move the 'close' method off the unpackermixin
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31862
diff changeset
945
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
946
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
947 formatmap = {b'20': unbundle20}
24648
5cac3accdaa1 unbundle20: allow generic dispatch between unbundlers
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24642
diff changeset
948
26395
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26393
diff changeset
949 b2streamparamsmap = {}
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26393
diff changeset
950
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
951
26395
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26393
diff changeset
952 def b2streamparamhandler(name):
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26393
diff changeset
953 """register a handler for a stream level parameter"""
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
954
26395
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26393
diff changeset
955 def decorator(func):
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26393
diff changeset
956 assert name not in formatmap
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26393
diff changeset
957 b2streamparamsmap[name] = func
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26393
diff changeset
958 return func
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
959
26395
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26393
diff changeset
960 return decorator
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26393
diff changeset
961
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
962
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
963 @b2streamparamhandler(b'compression')
26404
795f02a24b87 bundle2: allow compressed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26396
diff changeset
964 def processcompression(unbundler, param, value):
795f02a24b87 bundle2: allow compressed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26396
diff changeset
965 """read compression parameter and install payload decompression"""
30353
d045b4091197 bundle2: use compression engines API to obtain decompressor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30351
diff changeset
966 if value not in util.compengines.supportedbundletypes:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
967 raise error.BundleUnknownFeatureError(params=(param,), values=(value,))
30353
d045b4091197 bundle2: use compression engines API to obtain decompressor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30351
diff changeset
968 unbundler._compengine = util.compengines.forbundletype(value)
26802
42f705f2c02d bundle2: make unbundle.compressed return True when compressed
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26795
diff changeset
969 if value is not None:
42f705f2c02d bundle2: make unbundle.compressed return True when compressed
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26795
diff changeset
970 unbundler._compressed = True
26404
795f02a24b87 bundle2: allow compressed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26396
diff changeset
971
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
972
21005
3d38ebb586fe bundle2: rename part to bundlepart
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21004
diff changeset
973 class bundlepart(object):
20856
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
974 """A bundle2 part contains application level payload
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
975
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
976 The part `type` is used to route the part to the application level
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
977 handler.
21604
c399bf961cb9 bundle2: the ability to set ``data`` attribute of the part is now official
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21603
diff changeset
978
c399bf961cb9 bundle2: the ability to set ``data`` attribute of the part is now official
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21603
diff changeset
979 The part payload is contained in ``part.data``. It could be raw bytes or a
21605
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
980 generator of byte chunks.
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
981
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
982 You can add parameters to the part using the ``addparam`` method.
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
983 Parameters can be either mandatory (default) or advisory. Remote side
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
984 should be able to safely ignore the advisory ones.
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
985
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
986 Both data and parameters cannot be modified after the generation has begun.
20856
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
987 """
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
988
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
989 def __init__(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
990 self,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
991 parttype,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
992 mandatoryparams=(),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
993 advisoryparams=(),
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
994 data=b'',
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
995 mandatory=True,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
996 ):
23868
405eacbecc52 bundle2: enforce parttype as alphanumerical
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23590
diff changeset
997 validateparttype(parttype)
20995
e995d104c87f bundle2: add an integer id to part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20952
diff changeset
998 self.id = None
20856
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
999 self.type = parttype
21604
c399bf961cb9 bundle2: the ability to set ``data`` attribute of the part is now official
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21603
diff changeset
1000 self._data = data
21605
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
1001 self._mandatoryparams = list(mandatoryparams)
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
1002 self._advisoryparams = list(advisoryparams)
21607
054fa5176fa7 bundle2: forbid duplicate parameter keys
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21606
diff changeset
1003 # checking for duplicated entries
054fa5176fa7 bundle2: forbid duplicate parameter keys
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21606
diff changeset
1004 self._seenparams = set()
054fa5176fa7 bundle2: forbid duplicate parameter keys
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21606
diff changeset
1005 for pname, __ in self._mandatoryparams + self._advisoryparams:
054fa5176fa7 bundle2: forbid duplicate parameter keys
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21606
diff changeset
1006 if pname in self._seenparams:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1007 raise error.ProgrammingError(b'duplicated params: %s' % pname)
21607
054fa5176fa7 bundle2: forbid duplicate parameter keys
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21606
diff changeset
1008 self._seenparams.add(pname)
21601
7ff01befc7ec bundle2: track life cycle of parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21600
diff changeset
1009 # status of the part's generation:
7ff01befc7ec bundle2: track life cycle of parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21600
diff changeset
1010 # - None: not started,
7ff01befc7ec bundle2: track life cycle of parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21600
diff changeset
1011 # - False: currently generated,
7ff01befc7ec bundle2: track life cycle of parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21600
diff changeset
1012 # - True: generation done.
7ff01befc7ec bundle2: track life cycle of parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21600
diff changeset
1013 self._generated = None
23590
4440c7cc3728 bundle2.bundlepart: make mandatory part flag explicit in API
Eric Sumner <ericsumner@fb.com>
parents: 23586
diff changeset
1014 self.mandatory = mandatory
20856
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
1015
30872
1f51b4658f21 bundle2: implement a basic __repr__ for bundle2 part
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 30871
diff changeset
1016 def __repr__(self):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1017 cls = b"%s.%s" % (self.__class__.__module__, self.__class__.__name__)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1018 return b'<%s object at %x; id: %s; type: %s; mandatory: %s>' % (
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1019 cls,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1020 id(self),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1021 self.id,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1022 self.type,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1023 self.mandatory,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1024 )
30872
1f51b4658f21 bundle2: implement a basic __repr__ for bundle2 part
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 30871
diff changeset
1025
24793
2ec894582ea2 bundle2: add a 'copy' method on parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24743
diff changeset
1026 def copy(self):
2ec894582ea2 bundle2: add a 'copy' method on parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24743
diff changeset
1027 """return a copy of the part
2ec894582ea2 bundle2: add a 'copy' method on parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24743
diff changeset
1028
2ec894582ea2 bundle2: add a 'copy' method on parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24743
diff changeset
1029 The new part have the very same content but no partid assigned yet.
2ec894582ea2 bundle2: add a 'copy' method on parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24743
diff changeset
1030 Parts with generated data cannot be copied."""
43115
4aa72cdf616f py3: delete b'' prefix from safehasattr arguments
Martin von Zweigbergk <martinvonz@google.com>
parents: 43080
diff changeset
1031 assert not util.safehasattr(self.data, 'next')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1032 return self.__class__(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1033 self.type,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1034 self._mandatoryparams,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1035 self._advisoryparams,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1036 self._data,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1037 self.mandatory,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1038 )
24793
2ec894582ea2 bundle2: add a 'copy' method on parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24743
diff changeset
1039
21604
c399bf961cb9 bundle2: the ability to set ``data`` attribute of the part is now official
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21603
diff changeset
1040 # methods used to defines the part content
27879
52a4ad62b006 cleanup: use modern @property/@foo.setter property specification
Augie Fackler <augie@google.com>
parents: 27751
diff changeset
1041 @property
52a4ad62b006 cleanup: use modern @property/@foo.setter property specification
Augie Fackler <augie@google.com>
parents: 27751
diff changeset
1042 def data(self):
52a4ad62b006 cleanup: use modern @property/@foo.setter property specification
Augie Fackler <augie@google.com>
parents: 27751
diff changeset
1043 return self._data
52a4ad62b006 cleanup: use modern @property/@foo.setter property specification
Augie Fackler <augie@google.com>
parents: 27751
diff changeset
1044
52a4ad62b006 cleanup: use modern @property/@foo.setter property specification
Augie Fackler <augie@google.com>
parents: 27751
diff changeset
1045 @data.setter
52a4ad62b006 cleanup: use modern @property/@foo.setter property specification
Augie Fackler <augie@google.com>
parents: 27751
diff changeset
1046 def data(self, data):
21604
c399bf961cb9 bundle2: the ability to set ``data`` attribute of the part is now official
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21603
diff changeset
1047 if self._generated is not None:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1048 raise error.ReadOnlyPartError(b'part is being generated')
21604
c399bf961cb9 bundle2: the ability to set ``data`` attribute of the part is now official
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21603
diff changeset
1049 self._data = data
c399bf961cb9 bundle2: the ability to set ``data`` attribute of the part is now official
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21603
diff changeset
1050
21605
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
1051 @property
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
1052 def mandatoryparams(self):
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
1053 # make it an immutable tuple to force people through ``addparam``
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
1054 return tuple(self._mandatoryparams)
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
1055
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
1056 @property
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
1057 def advisoryparams(self):
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
1058 # make it an immutable tuple to force people through ``addparam``
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
1059 return tuple(self._advisoryparams)
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
1060
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1061 def addparam(self, name, value=b'', mandatory=True):
31861
6d055cd6669f bundle2: add documention to 'part.addparams'
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31647
diff changeset
1062 """add a parameter to the part
6d055cd6669f bundle2: add documention to 'part.addparams'
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31647
diff changeset
1063
6d055cd6669f bundle2: add documention to 'part.addparams'
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31647
diff changeset
1064 If 'mandatory' is set to True, the remote handler must claim support
6d055cd6669f bundle2: add documention to 'part.addparams'
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31647
diff changeset
1065 for this parameter or the unbundling will be aborted.
6d055cd6669f bundle2: add documention to 'part.addparams'
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31647
diff changeset
1066
6d055cd6669f bundle2: add documention to 'part.addparams'
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31647
diff changeset
1067 The 'name' and 'value' cannot exceed 255 bytes each.
6d055cd6669f bundle2: add documention to 'part.addparams'
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31647
diff changeset
1068 """
21605
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
1069 if self._generated is not None:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1070 raise error.ReadOnlyPartError(b'part is being generated')
21607
054fa5176fa7 bundle2: forbid duplicate parameter keys
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21606
diff changeset
1071 if name in self._seenparams:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1072 raise ValueError(b'duplicated params: %s' % name)
21607
054fa5176fa7 bundle2: forbid duplicate parameter keys
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21606
diff changeset
1073 self._seenparams.add(name)
21605
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
1074 params = self._advisoryparams
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
1075 if mandatory:
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
1076 params = self._mandatoryparams
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
1077 params.append((name, value))
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
1078
21601
7ff01befc7ec bundle2: track life cycle of parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21600
diff changeset
1079 # methods used to generates the bundle2 stream
25321
b44ee346211f bundle2: add debug output for part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25320
diff changeset
1080 def getchunks(self, ui):
21601
7ff01befc7ec bundle2: track life cycle of parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21600
diff changeset
1081 if self._generated is not None:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1082 raise error.ProgrammingError(b'part can only be consumed once')
21601
7ff01befc7ec bundle2: track life cycle of parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21600
diff changeset
1083 self._generated = False
25323
21a25fb81d2c bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25322
diff changeset
1084
21a25fb81d2c bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25322
diff changeset
1085 if ui.debugflag:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1086 msg = [b'bundle2-output-part: "%s"' % self.type]
25323
21a25fb81d2c bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25322
diff changeset
1087 if not self.mandatory:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1088 msg.append(b' (advisory)')
25323
21a25fb81d2c bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25322
diff changeset
1089 nbmp = len(self.mandatoryparams)
21a25fb81d2c bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25322
diff changeset
1090 nbap = len(self.advisoryparams)
21a25fb81d2c bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25322
diff changeset
1091 if nbmp or nbap:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1092 msg.append(b' (params:')
25323
21a25fb81d2c bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25322
diff changeset
1093 if nbmp:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1094 msg.append(b' %i mandatory' % nbmp)
25323
21a25fb81d2c bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25322
diff changeset
1095 if nbap:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1096 msg.append(b' %i advisory' % nbmp)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1097 msg.append(b')')
25323
21a25fb81d2c bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25322
diff changeset
1098 if not self.data:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1099 msg.append(b' empty payload')
43115
4aa72cdf616f py3: delete b'' prefix from safehasattr arguments
Martin von Zweigbergk <martinvonz@google.com>
parents: 43080
diff changeset
1100 elif util.safehasattr(self.data, 'next') or util.safehasattr(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1101 self.data, b'__next__'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1102 ):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1103 msg.append(b' streamed payload')
25323
21a25fb81d2c bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25322
diff changeset
1104 else:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1105 msg.append(b' %i bytes payload' % len(self.data))
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1106 msg.append(b'\n')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1107 ui.debug(b''.join(msg))
25323
21a25fb81d2c bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25322
diff changeset
1108
20877
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1109 #### header
23590
4440c7cc3728 bundle2.bundlepart: make mandatory part flag explicit in API
Eric Sumner <ericsumner@fb.com>
parents: 23586
diff changeset
1110 if self.mandatory:
4440c7cc3728 bundle2.bundlepart: make mandatory part flag explicit in API
Eric Sumner <ericsumner@fb.com>
parents: 23586
diff changeset
1111 parttype = self.type.upper()
4440c7cc3728 bundle2.bundlepart: make mandatory part flag explicit in API
Eric Sumner <ericsumner@fb.com>
parents: 23586
diff changeset
1112 else:
4440c7cc3728 bundle2.bundlepart: make mandatory part flag explicit in API
Eric Sumner <ericsumner@fb.com>
parents: 23586
diff changeset
1113 parttype = self.type.lower()
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1114 outdebug(ui, b'part %s: "%s"' % (pycompat.bytestr(self.id), parttype))
20877
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1115 ## parttype
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1116 header = [
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1117 _pack(_fparttypesize, len(parttype)),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1118 parttype,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1119 _pack(_fpartid, self.id),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1120 ]
20877
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1121 ## parameters
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1122 # count
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1123 manpar = self.mandatoryparams
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1124 advpar = self.advisoryparams
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1125 header.append(_pack(_fpartparamcount, len(manpar), len(advpar)))
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1126 # size
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1127 parsizes = []
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1128 for key, value in manpar:
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1129 parsizes.append(len(key))
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1130 parsizes.append(len(value))
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1131 for key, value in advpar:
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1132 parsizes.append(len(key))
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1133 parsizes.append(len(value))
33672
da7c285ec6da bundle2: use modern Python division
Augie Fackler <augie@google.com>
parents: 33665
diff changeset
1134 paramsizes = _pack(_makefpartparamsizes(len(parsizes) // 2), *parsizes)
20877
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1135 header.append(paramsizes)
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1136 # key, value
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1137 for key, value in manpar:
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1138 header.append(key)
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1139 header.append(value)
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1140 for key, value in advpar:
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1141 header.append(key)
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1142 header.append(value)
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1143 ## finalize header
34249
ab379eed2e31 bundle2: raise a more helpful error if building a bundle part header fails
Augie Fackler <raf@durin42.com>
parents: 34221
diff changeset
1144 try:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1145 headerchunk = b''.join(header)
34249
ab379eed2e31 bundle2: raise a more helpful error if building a bundle part header fails
Augie Fackler <raf@durin42.com>
parents: 34221
diff changeset
1146 except TypeError:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1147 raise TypeError(
43506
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43495
diff changeset
1148 'Found a non-bytes trying to '
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43495
diff changeset
1149 'build bundle part header: %r' % header
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1150 )
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1151 outdebug(ui, b'header chunk size: %i' % len(headerchunk))
20856
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
1152 yield _pack(_fpartheadersize, len(headerchunk))
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
1153 yield headerchunk
20877
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1154 ## payload
23067
420a051616ce bundle2: transmit exception during part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23066
diff changeset
1155 try:
420a051616ce bundle2: transmit exception during part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23066
diff changeset
1156 for chunk in self._payloadchunks():
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1157 outdebug(ui, b'payload chunk size: %i' % len(chunk))
23067
420a051616ce bundle2: transmit exception during part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23066
diff changeset
1158 yield _pack(_fpayloadsize, len(chunk))
420a051616ce bundle2: transmit exception during part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23066
diff changeset
1159 yield chunk
26144
4bc3707f3e67 bundle2: don't try to recover from a GeneratorExit (issue4785)
Augie Fackler <augie@google.com>
parents: 25919
diff changeset
1160 except GeneratorExit:
4bc3707f3e67 bundle2: don't try to recover from a GeneratorExit (issue4785)
Augie Fackler <augie@google.com>
parents: 25919
diff changeset
1161 # GeneratorExit means that nobody is listening for our
4bc3707f3e67 bundle2: don't try to recover from a GeneratorExit (issue4785)
Augie Fackler <augie@google.com>
parents: 25919
diff changeset
1162 # results anyway, so just bail quickly rather than trying
4bc3707f3e67 bundle2: don't try to recover from a GeneratorExit (issue4785)
Augie Fackler <augie@google.com>
parents: 25919
diff changeset
1163 # to produce an error part.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1164 ui.debug(b'bundle2-generatorexit\n')
26144
4bc3707f3e67 bundle2: don't try to recover from a GeneratorExit (issue4785)
Augie Fackler <augie@google.com>
parents: 25919
diff changeset
1165 raise
25660
328739ea70c3 global: mass rewrite to use modern exception syntax
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25641
diff changeset
1166 except BaseException as exc:
37084
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module
Yuya Nishihara <yuya@tcha.org>
parents: 37005
diff changeset
1167 bexc = stringutil.forcebytestr(exc)
23067
420a051616ce bundle2: transmit exception during part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23066
diff changeset
1168 # backup exception data for later
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1169 ui.debug(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1170 b'bundle2-input-stream-interrupt: encoding exception %s' % bexc
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1171 )
32186
76f9a0009b4b pycompat: extract helper to raise exception with traceback
Yuya Nishihara <yuya@tcha.org>
parents: 32024
diff changeset
1172 tb = sys.exc_info()[2]
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1173 msg = b'unexpected error: %s' % bexc
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1174 interpart = bundlepart(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1175 b'error:abort', [(b'message', msg)], mandatory=False
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1176 )
23067
420a051616ce bundle2: transmit exception during part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23066
diff changeset
1177 interpart.id = 0
420a051616ce bundle2: transmit exception during part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23066
diff changeset
1178 yield _pack(_fpayloadsize, -1)
25321
b44ee346211f bundle2: add debug output for part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25320
diff changeset
1179 for chunk in interpart.getchunks(ui=ui):
23067
420a051616ce bundle2: transmit exception during part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23066
diff changeset
1180 yield chunk
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1181 outdebug(ui, b'closing payload chunk')
23067
420a051616ce bundle2: transmit exception during part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23066
diff changeset
1182 # abort current part payload
420a051616ce bundle2: transmit exception during part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23066
diff changeset
1183 yield _pack(_fpayloadsize, 0)
32186
76f9a0009b4b pycompat: extract helper to raise exception with traceback
Yuya Nishihara <yuya@tcha.org>
parents: 32024
diff changeset
1184 pycompat.raisewithtb(exc, tb)
21000
4cae06ae1562 bundle2: extract a _payloadchunks method for part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20998
diff changeset
1185 # end of payload
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1186 outdebug(ui, b'closing payload chunk')
21000
4cae06ae1562 bundle2: extract a _payloadchunks method for part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20998
diff changeset
1187 yield _pack(_fpayloadsize, 0)
21601
7ff01befc7ec bundle2: track life cycle of parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21600
diff changeset
1188 self._generated = True
21000
4cae06ae1562 bundle2: extract a _payloadchunks method for part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20998
diff changeset
1189
4cae06ae1562 bundle2: extract a _payloadchunks method for part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20998
diff changeset
1190 def _payloadchunks(self):
4cae06ae1562 bundle2: extract a _payloadchunks method for part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20998
diff changeset
1191 """yield chunks of a the part payload
4cae06ae1562 bundle2: extract a _payloadchunks method for part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20998
diff changeset
1192
4cae06ae1562 bundle2: extract a _payloadchunks method for part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20998
diff changeset
1193 Exists to handle the different methods to provide data to a part."""
20876
ddd56f3eb786 bundle2: support for bundling and unbundling payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20864
diff changeset
1194 # we only support fixed size data now.
ddd56f3eb786 bundle2: support for bundling and unbundling payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20864
diff changeset
1195 # This will be improved in the future.
43115
4aa72cdf616f py3: delete b'' prefix from safehasattr arguments
Martin von Zweigbergk <martinvonz@google.com>
parents: 43080
diff changeset
1196 if util.safehasattr(self.data, 'next') or util.safehasattr(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1197 self.data, b'__next__'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1198 ):
21001
c93bb6a08fa1 bundle2: support chunk iterator as part data
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21000
diff changeset
1199 buff = util.chunkbuffer(self.data)
c93bb6a08fa1 bundle2: support chunk iterator as part data
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21000
diff changeset
1200 chunk = buff.read(preferedchunksize)
c93bb6a08fa1 bundle2: support chunk iterator as part data
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21000
diff changeset
1201 while chunk:
c93bb6a08fa1 bundle2: support chunk iterator as part data
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21000
diff changeset
1202 yield chunk
c93bb6a08fa1 bundle2: support chunk iterator as part data
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21000
diff changeset
1203 chunk = buff.read(preferedchunksize)
c93bb6a08fa1 bundle2: support chunk iterator as part data
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21000
diff changeset
1204 elif len(self.data):
20876
ddd56f3eb786 bundle2: support for bundling and unbundling payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20864
diff changeset
1205 yield self.data
20802
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
1206
23066
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1207
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1208 flaginterrupt = -1
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1209
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1210
23066
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1211 class interrupthandler(unpackermixin):
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1212 """read one part and process it with restricted capability
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1213
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1214 This allows to transmit exception raised on the producer size during part
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1215 iteration while the consumer is reading a part.
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1216
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1217 Part processed in this manner only have access to a ui object,"""
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1218
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1219 def __init__(self, ui, fp):
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1220 super(interrupthandler, self).__init__(fp)
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1221 self.ui = ui
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1222
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1223 def _readpartheader(self):
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1224 """reads a part header size and return the bytes blob
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1225
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1226 returns None if empty"""
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1227 headersize = self._unpack(_fpartheadersize)[0]
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1228 if headersize < 0:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1229 raise error.BundleValueError(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1230 b'negative part header size: %i' % headersize
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1231 )
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1232 indebug(self.ui, b'part header size: %i\n' % headersize)
23066
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1233 if headersize:
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1234 return self._readexact(headersize)
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1235 return None
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1236
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1237 def __call__(self):
25335
8f7137a85a0e bundle2: add generic debug output regarding processed interruption
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25334
diff changeset
1238
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1239 self.ui.debug(
43117
8ff1ecfadcd1 cleanup: join string literals that are already on one line
Martin von Zweigbergk <martinvonz@google.com>
parents: 43115
diff changeset
1240 b'bundle2-input-stream-interrupt: opening out of band context\n'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1241 )
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1242 indebug(self.ui, b'bundle2 stream interruption, looking for a part.')
23066
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1243 headerblock = self._readpartheader()
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1244 if headerblock is None:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1245 indebug(self.ui, b'no part found during interruption.')
23066
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1246 return
35116
da91e7309daf bundle2: don't use seekable bundle2 parts by default (issue5691)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35115
diff changeset
1247 part = unbundlepart(self.ui, headerblock, self._fp)
23066
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1248 op = interruptoperation(self.ui)
34258
e71890f27767 bundle2: move processpart stream maintenance into part iterator
Durham Goode <durham@fb.com>
parents: 34249
diff changeset
1249 hardabort = False
e71890f27767 bundle2: move processpart stream maintenance into part iterator
Durham Goode <durham@fb.com>
parents: 34249
diff changeset
1250 try:
e71890f27767 bundle2: move processpart stream maintenance into part iterator
Durham Goode <durham@fb.com>
parents: 34249
diff changeset
1251 _processpart(op, part)
e71890f27767 bundle2: move processpart stream maintenance into part iterator
Durham Goode <durham@fb.com>
parents: 34249
diff changeset
1252 except (SystemExit, KeyboardInterrupt):
e71890f27767 bundle2: move processpart stream maintenance into part iterator
Durham Goode <durham@fb.com>
parents: 34249
diff changeset
1253 hardabort = True
e71890f27767 bundle2: move processpart stream maintenance into part iterator
Durham Goode <durham@fb.com>
parents: 34249
diff changeset
1254 raise
e71890f27767 bundle2: move processpart stream maintenance into part iterator
Durham Goode <durham@fb.com>
parents: 34249
diff changeset
1255 finally:
e71890f27767 bundle2: move processpart stream maintenance into part iterator
Durham Goode <durham@fb.com>
parents: 34249
diff changeset
1256 if not hardabort:
35114
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35113
diff changeset
1257 part.consume()
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1258 self.ui.debug(
43117
8ff1ecfadcd1 cleanup: join string literals that are already on one line
Martin von Zweigbergk <martinvonz@google.com>
parents: 43115
diff changeset
1259 b'bundle2-input-stream-interrupt: closing out of band context\n'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1260 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1261
23066
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1262
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1263 class interruptoperation(object):
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1264 """A limited operation to be use by part handler during interruption
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1265
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1266 It only have access to an ui object.
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1267 """
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1268
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1269 def __init__(self, ui):
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1270 self.ui = ui
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1271 self.reply = None
24878
e530cde6d115 bundle2: disable ouput capture unless we use http (issue4613 issue4615)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24851
diff changeset
1272 self.captureoutput = False
23066
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1273
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1274 @property
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1275 def repo(self):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1276 raise error.ProgrammingError(b'no repo access from stream interruption')
23066
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1277
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1278 def gettransaction(self):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1279 raise TransactionUnavailable(b'no repo access from stream interruption')
23066
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1280
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1281
35113
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35112
diff changeset
1282 def decodepayloadchunks(ui, fh):
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35112
diff changeset
1283 """Reads bundle2 part payload data into chunks.
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35112
diff changeset
1284
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35112
diff changeset
1285 Part payload data consists of framed chunks. This function takes
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35112
diff changeset
1286 a file handle and emits those chunks.
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35112
diff changeset
1287 """
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1288 dolog = ui.configbool(b'devel', b'bundle2.debug')
35117
589eed45a452 bundle2: inline debug logging
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35116
diff changeset
1289 debug = ui.debug
589eed45a452 bundle2: inline debug logging
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35116
diff changeset
1290
35119
764e3ad1cf54 bundle2: inline struct operations
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35118
diff changeset
1291 headerstruct = struct.Struct(_fpayloadsize)
764e3ad1cf54 bundle2: inline struct operations
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35118
diff changeset
1292 headersize = headerstruct.size
764e3ad1cf54 bundle2: inline struct operations
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35118
diff changeset
1293 unpack = headerstruct.unpack
764e3ad1cf54 bundle2: inline struct operations
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35118
diff changeset
1294
35113
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35112
diff changeset
1295 readexactly = changegroup.readexactly
35118
1fb0846ad792 bundle2: inline changegroup.readexactly()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35117
diff changeset
1296 read = fh.read
35113
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35112
diff changeset
1297
35119
764e3ad1cf54 bundle2: inline struct operations
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35118
diff changeset
1298 chunksize = unpack(readexactly(fh, headersize))[0]
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1299 indebug(ui, b'payload chunk size: %i' % chunksize)
35113
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35112
diff changeset
1300
35118
1fb0846ad792 bundle2: inline changegroup.readexactly()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35117
diff changeset
1301 # changegroup.readexactly() is inlined below for performance.
35113
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35112
diff changeset
1302 while chunksize:
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35112
diff changeset
1303 if chunksize >= 0:
35118
1fb0846ad792 bundle2: inline changegroup.readexactly()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35117
diff changeset
1304 s = read(chunksize)
1fb0846ad792 bundle2: inline changegroup.readexactly()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35117
diff changeset
1305 if len(s) < chunksize:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1306 raise error.Abort(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1307 _(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1308 b'stream ended unexpectedly '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1309 b' (got %d bytes, expected %d)'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1310 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1311 % (len(s), chunksize)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1312 )
35118
1fb0846ad792 bundle2: inline changegroup.readexactly()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35117
diff changeset
1313
1fb0846ad792 bundle2: inline changegroup.readexactly()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35117
diff changeset
1314 yield s
35113
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35112
diff changeset
1315 elif chunksize == flaginterrupt:
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35112
diff changeset
1316 # Interrupt "signal" detected. The regular stream is interrupted
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35112
diff changeset
1317 # and a bundle2 part follows. Consume it.
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35112
diff changeset
1318 interrupthandler(ui, fh)()
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35112
diff changeset
1319 else:
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35112
diff changeset
1320 raise error.BundleValueError(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1321 b'negative payload chunk size: %s' % chunksize
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1322 )
35113
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35112
diff changeset
1323
35118
1fb0846ad792 bundle2: inline changegroup.readexactly()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35117
diff changeset
1324 s = read(headersize)
1fb0846ad792 bundle2: inline changegroup.readexactly()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35117
diff changeset
1325 if len(s) < headersize:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1326 raise error.Abort(
43117
8ff1ecfadcd1 cleanup: join string literals that are already on one line
Martin von Zweigbergk <martinvonz@google.com>
parents: 43115
diff changeset
1327 _(b'stream ended unexpectedly (got %d bytes, expected %d)')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1328 % (len(s), chunksize)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1329 )
35118
1fb0846ad792 bundle2: inline changegroup.readexactly()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35117
diff changeset
1330
35119
764e3ad1cf54 bundle2: inline struct operations
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35118
diff changeset
1331 chunksize = unpack(s)[0]
35117
589eed45a452 bundle2: inline debug logging
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35116
diff changeset
1332
589eed45a452 bundle2: inline debug logging
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35116
diff changeset
1333 # indebug() inlined for performance.
589eed45a452 bundle2: inline debug logging
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35116
diff changeset
1334 if dolog:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1335 debug(b'bundle2-input: payload chunk size: %i\n' % chunksize)
35113
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35112
diff changeset
1336
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1337
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1338 class unbundlepart(unpackermixin):
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1339 """a bundle part read from a bundle"""
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1340
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1341 def __init__(self, ui, header, fp):
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1342 super(unbundlepart, self).__init__(fp)
43115
4aa72cdf616f py3: delete b'' prefix from safehasattr arguments
Martin von Zweigbergk <martinvonz@google.com>
parents: 43080
diff changeset
1343 self._seekable = util.safehasattr(fp, 'seek') and util.safehasattr(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1344 fp, b'tell'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1345 )
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1346 self.ui = ui
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1347 # unbundle state attr
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1348 self._headerdata = header
21015
14dd49260246 bundle2: move the fromheader closure into the class itself
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21014
diff changeset
1349 self._headeroffset = 0
21019
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
1350 self._initialized = False
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
1351 self.consumed = False
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1352 # part data
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1353 self.id = None
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1354 self.type = None
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1355 self.mandatoryparams = None
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1356 self.advisoryparams = None
21610
d6056805f8f4 bundle2: introduce a ``params`` dictionary on unbundled parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21609
diff changeset
1357 self.params = None
21612
f221eb0531d9 bundle2: expose mandatory params in a mandatorykeys attribute
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21611
diff changeset
1358 self.mandatorykeys = ()
21019
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
1359 self._readheader()
23585
94b25d71dd0f bundle2.unbundlepart: decouple mandatory from parttype
Eric Sumner <ericsumner@fb.com>
parents: 23439
diff changeset
1360 self._mandatory = None
24036
c7601086338a bundle2.unbundlepart: tell() implementation
Eric Sumner <ericsumner@fb.com>
parents: 24035
diff changeset
1361 self._pos = 0
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1362
21015
14dd49260246 bundle2: move the fromheader closure into the class itself
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21014
diff changeset
1363 def _fromheader(self, size):
14dd49260246 bundle2: move the fromheader closure into the class itself
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21014
diff changeset
1364 """return the next <size> byte from the header"""
14dd49260246 bundle2: move the fromheader closure into the class itself
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21014
diff changeset
1365 offset = self._headeroffset
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1366 data = self._headerdata[offset : (offset + size)]
21019
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
1367 self._headeroffset = offset + size
21015
14dd49260246 bundle2: move the fromheader closure into the class itself
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21014
diff changeset
1368 return data
14dd49260246 bundle2: move the fromheader closure into the class itself
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21014
diff changeset
1369
21016
b477afb1c81e bundle2: move unpackheader closure into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21015
diff changeset
1370 def _unpackheader(self, format):
b477afb1c81e bundle2: move unpackheader closure into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21015
diff changeset
1371 """read given format from header
b477afb1c81e bundle2: move unpackheader closure into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21015
diff changeset
1372
b477afb1c81e bundle2: move unpackheader closure into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21015
diff changeset
1373 This automatically compute the size of the format to read."""
b477afb1c81e bundle2: move unpackheader closure into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21015
diff changeset
1374 data = self._fromheader(struct.calcsize(format))
b477afb1c81e bundle2: move unpackheader closure into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21015
diff changeset
1375 return _unpack(format, data)
b477afb1c81e bundle2: move unpackheader closure into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21015
diff changeset
1376
21608
3cb96ca90c17 bundle2: introduce an ``_initparams`` method
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21607
diff changeset
1377 def _initparams(self, mandatoryparams, advisoryparams):
3cb96ca90c17 bundle2: introduce an ``_initparams`` method
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21607
diff changeset
1378 """internal function to setup all logic related parameters"""
21609
63cc2594ef8a bundle2: make sure unbundled part param are read-only
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21608
diff changeset
1379 # make it read only to prevent people touching it by mistake.
63cc2594ef8a bundle2: make sure unbundled part param are read-only
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21608
diff changeset
1380 self.mandatoryparams = tuple(mandatoryparams)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1381 self.advisoryparams = tuple(advisoryparams)
21610
d6056805f8f4 bundle2: introduce a ``params`` dictionary on unbundled parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21609
diff changeset
1382 # user friendly UI
29591
6215b5537ba5 bundle2: use a sorted dict for holding parameters
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29294
diff changeset
1383 self.params = util.sortdict(self.mandatoryparams)
6215b5537ba5 bundle2: use a sorted dict for holding parameters
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29294
diff changeset
1384 self.params.update(self.advisoryparams)
21612
f221eb0531d9 bundle2: expose mandatory params in a mandatorykeys attribute
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21611
diff changeset
1385 self.mandatorykeys = frozenset(p[0] for p in mandatoryparams)
21608
3cb96ca90c17 bundle2: introduce an ``_initparams`` method
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21607
diff changeset
1386
21019
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
1387 def _readheader(self):
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1388 """read the header and setup the object"""
21016
b477afb1c81e bundle2: move unpackheader closure into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21015
diff changeset
1389 typesize = self._unpackheader(_fparttypesize)[0]
21015
14dd49260246 bundle2: move the fromheader closure into the class itself
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21014
diff changeset
1390 self.type = self._fromheader(typesize)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1391 indebug(self.ui, b'part type: "%s"' % self.type)
21016
b477afb1c81e bundle2: move unpackheader closure into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21015
diff changeset
1392 self.id = self._unpackheader(_fpartid)[0]
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1393 indebug(self.ui, b'part id: "%s"' % pycompat.bytestr(self.id))
23585
94b25d71dd0f bundle2.unbundlepart: decouple mandatory from parttype
Eric Sumner <ericsumner@fb.com>
parents: 23439
diff changeset
1394 # extract mandatory bit from type
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1395 self.mandatory = self.type != self.type.lower()
23585
94b25d71dd0f bundle2.unbundlepart: decouple mandatory from parttype
Eric Sumner <ericsumner@fb.com>
parents: 23439
diff changeset
1396 self.type = self.type.lower()
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1397 ## reading parameters
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1398 # param count
21016
b477afb1c81e bundle2: move unpackheader closure into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21015
diff changeset
1399 mancount, advcount = self._unpackheader(_fpartparamcount)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1400 indebug(self.ui, b'part parameters: %i' % (mancount + advcount))
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1401 # param size
21016
b477afb1c81e bundle2: move unpackheader closure into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21015
diff changeset
1402 fparamsizes = _makefpartparamsizes(mancount + advcount)
b477afb1c81e bundle2: move unpackheader closure into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21015
diff changeset
1403 paramsizes = self._unpackheader(fparamsizes)
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1404 # make it a list of couple again
33674
9a323773216c bundle2: work around zip() being lazy in Python 3
Augie Fackler <augie@google.com>
parents: 33673
diff changeset
1405 paramsizes = list(zip(paramsizes[::2], paramsizes[1::2]))
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1406 # split mandatory from advisory
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1407 mansizes = paramsizes[:mancount]
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1408 advsizes = paramsizes[mancount:]
23139
e53f6b72a0e4 spelling: fixes from proofreading of spell checker issues
Mads Kiilerich <madski@unity3d.com>
parents: 23067
diff changeset
1409 # retrieve param value
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1410 manparams = []
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1411 for key, value in mansizes:
21015
14dd49260246 bundle2: move the fromheader closure into the class itself
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21014
diff changeset
1412 manparams.append((self._fromheader(key), self._fromheader(value)))
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1413 advparams = []
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1414 for key, value in advsizes:
21015
14dd49260246 bundle2: move the fromheader closure into the class itself
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21014
diff changeset
1415 advparams.append((self._fromheader(key), self._fromheader(value)))
21608
3cb96ca90c17 bundle2: introduce an ``_initparams`` method
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21607
diff changeset
1416 self._initparams(manparams, advparams)
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1417 ## part payload
24034
9881a1437799 bundle2.unbundlepart: raise payloadchunks from a closure to a method
Eric Sumner <ericsumner@fb.com>
parents: 24026
diff changeset
1418 self._payloadstream = util.chunkbuffer(self._payloadchunks())
21019
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
1419 # we read the data, tell it
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
1420 self._initialized = True
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
1421
35113
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35112
diff changeset
1422 def _payloadchunks(self):
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35112
diff changeset
1423 """Generator of decoded chunks in the payload."""
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35112
diff changeset
1424 return decodepayloadchunks(self.ui, self._fp)
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35112
diff changeset
1425
35114
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35113
diff changeset
1426 def consume(self):
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35113
diff changeset
1427 """Read the part payload until completion.
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35113
diff changeset
1428
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35113
diff changeset
1429 By consuming the part data, the underlying stream read offset will
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35113
diff changeset
1430 be advanced to the next part (or end of stream).
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35113
diff changeset
1431 """
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35113
diff changeset
1432 if self.consumed:
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35113
diff changeset
1433 return
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35113
diff changeset
1434
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35113
diff changeset
1435 chunk = self.read(32768)
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35113
diff changeset
1436 while chunk:
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35113
diff changeset
1437 self._pos += len(chunk)
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35113
diff changeset
1438 chunk = self.read(32768)
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35113
diff changeset
1439
21019
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
1440 def read(self, size=None):
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
1441 """read payload data"""
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
1442 if not self._initialized:
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
1443 self._readheader()
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
1444 if size is None:
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
1445 data = self._payloadstream.read()
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
1446 else:
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
1447 data = self._payloadstream.read(size)
25334
76cba1ecf55f bundle2: add generic debug output regarding processed part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25333
diff changeset
1448 self._pos += len(data)
21019
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
1449 if size is None or len(data) < size:
25334
76cba1ecf55f bundle2: add generic debug output regarding processed part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25333
diff changeset
1450 if not self.consumed and self._pos:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1451 self.ui.debug(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1452 b'bundle2-input-part: total payload size %i\n' % self._pos
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1453 )
21019
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
1454 self.consumed = True
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
1455 return data
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
1456
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1457
35112
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1458 class seekableunbundlepart(unbundlepart):
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1459 """A bundle2 part in a bundle that is seekable.
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1460
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1461 Regular ``unbundlepart`` instances can only be read once. This class
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1462 extends ``unbundlepart`` to enable bi-directional seeking within the
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1463 part.
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1464
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1465 Bundle2 part data consists of framed chunks. Offsets when seeking
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1466 refer to the decoded data, not the offsets in the underlying bundle2
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1467 stream.
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1468
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1469 To facilitate quickly seeking within the decoded data, instances of this
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1470 class maintain a mapping between offsets in the underlying stream and
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1471 the decoded payload. This mapping will consume memory in proportion
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1472 to the number of chunks within the payload (which almost certainly
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1473 increases in proportion with the size of the part).
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1474 """
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1475
35112
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1476 def __init__(self, ui, header, fp):
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1477 # (payload, file) offsets for chunk starts.
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1478 self._chunkindex = []
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1479
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1480 super(seekableunbundlepart, self).__init__(ui, header, fp)
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1481
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1482 def _payloadchunks(self, chunknum=0):
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1483 '''seek to specified chunk and start yielding data'''
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1484 if len(self._chunkindex) == 0:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1485 assert chunknum == 0, b'Must start with chunk 0'
35112
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1486 self._chunkindex.append((0, self._tellfp()))
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1487 else:
41759
aaad36b88298 cleanup: use () to wrap long lines instead of \
Augie Fackler <augie@google.com>
parents: 40429
diff changeset
1488 assert chunknum < len(self._chunkindex), (
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1489 b'Unknown chunk %d' % chunknum
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1490 )
35112
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1491 self._seekfp(self._chunkindex[chunknum][1])
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1492
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1493 pos = self._chunkindex[chunknum][0]
35113
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35112
diff changeset
1494
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35112
diff changeset
1495 for chunk in decodepayloadchunks(self.ui, self._fp):
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35112
diff changeset
1496 chunknum += 1
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35112
diff changeset
1497 pos += len(chunk)
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35112
diff changeset
1498 if chunknum == len(self._chunkindex):
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35112
diff changeset
1499 self._chunkindex.append((pos, self._tellfp()))
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35112
diff changeset
1500
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35112
diff changeset
1501 yield chunk
35112
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1502
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1503 def _findchunk(self, pos):
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1504 '''for a given payload position, return a chunk number and offset'''
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1505 for chunk, (ppos, fpos) in enumerate(self._chunkindex):
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1506 if ppos == pos:
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1507 return chunk, 0
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1508 elif ppos > pos:
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1509 return chunk - 1, pos - self._chunkindex[chunk - 1][0]
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1510 raise ValueError(b'Unknown chunk')
35112
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1511
24036
c7601086338a bundle2.unbundlepart: tell() implementation
Eric Sumner <ericsumner@fb.com>
parents: 24035
diff changeset
1512 def tell(self):
c7601086338a bundle2.unbundlepart: tell() implementation
Eric Sumner <ericsumner@fb.com>
parents: 24035
diff changeset
1513 return self._pos
c7601086338a bundle2.unbundlepart: tell() implementation
Eric Sumner <ericsumner@fb.com>
parents: 24035
diff changeset
1514
35046
241d9caca11e bundle2: use os.SEEK_* constants
Gregory Szorc <gregory.szorc@gmail.com>
parents: 34820
diff changeset
1515 def seek(self, offset, whence=os.SEEK_SET):
241d9caca11e bundle2: use os.SEEK_* constants
Gregory Szorc <gregory.szorc@gmail.com>
parents: 34820
diff changeset
1516 if whence == os.SEEK_SET:
24037
f0b498cfc5c8 bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents: 24036
diff changeset
1517 newpos = offset
35046
241d9caca11e bundle2: use os.SEEK_* constants
Gregory Szorc <gregory.szorc@gmail.com>
parents: 34820
diff changeset
1518 elif whence == os.SEEK_CUR:
24037
f0b498cfc5c8 bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents: 24036
diff changeset
1519 newpos = self._pos + offset
35046
241d9caca11e bundle2: use os.SEEK_* constants
Gregory Szorc <gregory.szorc@gmail.com>
parents: 34820
diff changeset
1520 elif whence == os.SEEK_END:
24037
f0b498cfc5c8 bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents: 24036
diff changeset
1521 if not self.consumed:
35120
699b2a759319 bundle2: avoid unbound read when seeking
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35119
diff changeset
1522 # Can't use self.consume() here because it advances self._pos.
699b2a759319 bundle2: avoid unbound read when seeking
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35119
diff changeset
1523 chunk = self.read(32768)
699b2a759319 bundle2: avoid unbound read when seeking
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35119
diff changeset
1524 while chunk:
699b2a759319 bundle2: avoid unbound read when seeking
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35119
diff changeset
1525 chunk = self.read(32768)
24037
f0b498cfc5c8 bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents: 24036
diff changeset
1526 newpos = self._chunkindex[-1][0] - offset
f0b498cfc5c8 bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents: 24036
diff changeset
1527 else:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1528 raise ValueError(b'Unknown whence value: %r' % (whence,))
24037
f0b498cfc5c8 bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents: 24036
diff changeset
1529
f0b498cfc5c8 bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents: 24036
diff changeset
1530 if newpos > self._chunkindex[-1][0] and not self.consumed:
35120
699b2a759319 bundle2: avoid unbound read when seeking
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35119
diff changeset
1531 # Can't use self.consume() here because it advances self._pos.
699b2a759319 bundle2: avoid unbound read when seeking
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35119
diff changeset
1532 chunk = self.read(32768)
699b2a759319 bundle2: avoid unbound read when seeking
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35119
diff changeset
1533 while chunk:
699b2a759319 bundle2: avoid unbound read when seeking
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35119
diff changeset
1534 chunk = self.read(32668)
699b2a759319 bundle2: avoid unbound read when seeking
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35119
diff changeset
1535
24037
f0b498cfc5c8 bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents: 24036
diff changeset
1536 if not 0 <= newpos <= self._chunkindex[-1][0]:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1537 raise ValueError(b'Offset out of range')
24037
f0b498cfc5c8 bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents: 24036
diff changeset
1538
f0b498cfc5c8 bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents: 24036
diff changeset
1539 if self._pos != newpos:
f0b498cfc5c8 bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents: 24036
diff changeset
1540 chunk, internaloffset = self._findchunk(newpos)
f0b498cfc5c8 bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents: 24036
diff changeset
1541 self._payloadstream = util.chunkbuffer(self._payloadchunks(chunk))
f0b498cfc5c8 bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents: 24036
diff changeset
1542 adjust = self.read(internaloffset)
f0b498cfc5c8 bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents: 24036
diff changeset
1543 if len(adjust) != internaloffset:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1544 raise error.Abort(_(b'Seek failed\n'))
24037
f0b498cfc5c8 bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents: 24036
diff changeset
1545 self._pos = newpos
f0b498cfc5c8 bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents: 24036
diff changeset
1546
31889
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1547 def _seekfp(self, offset, whence=0):
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1548 """move the underlying file pointer
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1549
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1550 This method is meant for internal usage by the bundle2 protocol only.
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1551 They directly manipulate the low level stream including bundle2 level
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1552 instruction.
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1553
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1554 Do not use it to implement higher-level logic or methods."""
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1555 if self._seekable:
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1556 return self._fp.seek(offset, whence)
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1557 else:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1558 raise NotImplementedError(_(b'File pointer is not seekable'))
31889
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1559
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1560 def _tellfp(self):
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1561 """return the file offset, or None if file is not seekable
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1562
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1563 This method is meant for internal usage by the bundle2 protocol only.
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1564 They directly manipulate the low level stream including bundle2 level
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1565 instruction.
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1566
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1567 Do not use it to implement higher-level logic or methods."""
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1568 if self._seekable:
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1569 try:
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1570 return self._fp.tell()
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1571 except IOError as e:
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1572 if e.errno == errno.ESPIPE:
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1573 self._seekable = False
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1574 else:
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1575 raise
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1576 return None
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1577
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1578
25317
5a5b7046d00e bundle2: add an informative comment to the capability dict
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25315
diff changeset
1579 # These are only the static capabilities.
5a5b7046d00e bundle2: add an informative comment to the capability dict
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25315
diff changeset
1580 # Check the 'getrepocaps' function for the rest.
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1581 capabilities = {
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1582 b'HG20': (),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1583 b'bookmarks': (),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1584 b'error': (b'abort', b'unsupportedcontent', b'pushraced', b'pushkey'),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1585 b'listkeys': (),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1586 b'pushkey': (),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1587 b'digests': tuple(sorted(util.DIGESTS.keys())),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1588 b'remote-changegroup': (b'http', b'https'),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1589 b'hgtagsfnodes': (),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1590 b'rev-branch-cache': (),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1591 b'phases': (b'heads',),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1592 b'stream': (b'v2',),
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1593 }
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1594
22341
2d16b39601b5 obsmarker: move bundle2caps from the localrepo class to the bundle2 module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22340
diff changeset
1595
35783
c97639ad6874 bundle2: specify what capabilities will be used for
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35763
diff changeset
1596 def getrepocaps(repo, allowpushback=False, role=None):
22342
262c5cc126c1 bundle2: introduce a `getrepocaps` to retrieve the bundle2 caps of a repo
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22341
diff changeset
1597 """return the bundle2 capabilities for a given repo
262c5cc126c1 bundle2: introduce a `getrepocaps` to retrieve the bundle2 caps of a repo
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22341
diff changeset
1598
22343
76677a2c1cfd bundle2: advertise the obsmarker part in bundle2 capabilities
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22342
diff changeset
1599 Exists to allow extensions (like evolution) to mutate the capabilities.
35783
c97639ad6874 bundle2: specify what capabilities will be used for
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35763
diff changeset
1600
c97639ad6874 bundle2: specify what capabilities will be used for
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35763
diff changeset
1601 The returned value is used for servers advertising their capabilities as
c97639ad6874 bundle2: specify what capabilities will be used for
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35763
diff changeset
1602 well as clients advertising their capabilities to servers as part of
c97639ad6874 bundle2: specify what capabilities will be used for
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35763
diff changeset
1603 bundle2 requests. The ``role`` argument specifies which is which.
22342
262c5cc126c1 bundle2: introduce a `getrepocaps` to retrieve the bundle2 caps of a repo
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22341
diff changeset
1604 """
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1605 if role not in (b'client', b'server'):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1606 raise error.ProgrammingError(b'role argument must be client or server')
35783
c97639ad6874 bundle2: specify what capabilities will be used for
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35763
diff changeset
1607
22343
76677a2c1cfd bundle2: advertise the obsmarker part in bundle2 capabilities
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22342
diff changeset
1608 caps = capabilities.copy()
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1609 caps[b'changegroup'] = tuple(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1610 sorted(changegroup.supportedincomingversions(repo))
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1611 )
22953
b1d694d3975e obsolete: add exchange option
Durham Goode <durham@fb.com>
parents: 22661
diff changeset
1612 if obsolete.isenabled(repo, obsolete.exchangeopt):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1613 supportedformat = tuple(b'V%i' % v for v in obsolete.formats)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1614 caps[b'obsmarkers'] = supportedformat
23439
743736fc7c41 bundle2-push: provide transaction to reply unbundler
Eric Sumner <ericsumner@fb.com>
parents: 23438
diff changeset
1615 if allowpushback:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1616 caps[b'pushback'] = ()
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1617 cpmode = repo.ui.config(b'server', b'concurrent-push-mode')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1618 if cpmode == b'check-related':
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1619 caps[b'checkheads'] = (b'related',)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1620 if b'phases' in repo.ui.configlist(b'devel', b'legacy.exchange'):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1621 caps.pop(b'phases')
35790
c1b9eb15a51c bundle2: don't advertise stream bundle2 capability when feature disabled
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35788
diff changeset
1622
c1b9eb15a51c bundle2: don't advertise stream bundle2 capability when feature disabled
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35788
diff changeset
1623 # Don't advertise stream clone support in server mode if not configured.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1624 if role == b'server':
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1625 streamsupported = repo.ui.configbool(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1626 b'server', b'uncompressed', untrusted=True
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1627 )
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1628 featuresupported = repo.ui.configbool(b'server', b'bundle2.stream')
35790
c1b9eb15a51c bundle2: don't advertise stream bundle2 capability when feature disabled
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35788
diff changeset
1629
c1b9eb15a51c bundle2: don't advertise stream bundle2 capability when feature disabled
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35788
diff changeset
1630 if not streamsupported or not featuresupported:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1631 caps.pop(b'stream')
35792
1d118f9f4f57 bundle2: always advertise client support for stream parts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35790
diff changeset
1632 # Else always advertise support on client, because payload support
1d118f9f4f57 bundle2: always advertise client support for stream parts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35790
diff changeset
1633 # should always be advertised.
35790
c1b9eb15a51c bundle2: don't advertise stream bundle2 capability when feature disabled
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35788
diff changeset
1634
22343
76677a2c1cfd bundle2: advertise the obsmarker part in bundle2 capabilities
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22342
diff changeset
1635 return caps
22342
262c5cc126c1 bundle2: introduce a `getrepocaps` to retrieve the bundle2 caps of a repo
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22341
diff changeset
1636
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1637
21644
17755dd8c509 bundle2: introduce a bundle2caps function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21628
diff changeset
1638 def bundle2caps(remote):
23139
e53f6b72a0e4 spelling: fixes from proofreading of spell checker issues
Mads Kiilerich <madski@unity3d.com>
parents: 23067
diff changeset
1639 """return the bundle capabilities of a peer as dict"""
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1640 raw = remote.capable(b'bundle2')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1641 if not raw and raw != b'':
21644
17755dd8c509 bundle2: introduce a bundle2caps function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21628
diff changeset
1642 return {}
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1643 capsblob = urlreq.unquote(remote.capable(b'bundle2'))
21644
17755dd8c509 bundle2: introduce a bundle2caps function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21628
diff changeset
1644 return decodecaps(capsblob)
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1645
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1646
22344
9829b7948100 bundle2: add a `obsmarkersversion` function to extract supported version
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22343
diff changeset
1647 def obsmarkersversion(caps):
9829b7948100 bundle2: add a `obsmarkersversion` function to extract supported version
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22343
diff changeset
1648 """extract the list of supported obsmarkers versions from a bundle2caps dict
9829b7948100 bundle2: add a `obsmarkersversion` function to extract supported version
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22343
diff changeset
1649 """
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1650 obscaps = caps.get(b'obsmarkers', ())
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1651 return [int(c[1:]) for c in obscaps if c.startswith(b'V')]
22344
9829b7948100 bundle2: add a `obsmarkersversion` function to extract supported version
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22343
diff changeset
1652
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1653
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1654 def writenewbundle(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1655 ui,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1656 repo,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1657 source,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1658 filename,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1659 bundletype,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1660 outgoing,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1661 opts,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1662 vfs=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1663 compression=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1664 compopts=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1665 ):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1666 if bundletype.startswith(b'HG10'):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1667 cg = changegroup.makechangegroup(repo, outgoing, b'01', source)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1668 return writebundle(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1669 ui,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1670 cg,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1671 filename,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1672 bundletype,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1673 vfs=vfs,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1674 compression=compression,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1675 compopts=compopts,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1676 )
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1677 elif not bundletype.startswith(b'HG20'):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1678 raise error.ProgrammingError(b'unknown bundle type: %s' % bundletype)
32221
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32186
diff changeset
1679
32516
37d70ba1d9d1 bundle: add an experimental knob to include obsmarkers in bundle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32515
diff changeset
1680 caps = {}
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1681 if b'obsolescence' in opts:
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1682 caps[b'obsmarkers'] = (b'V1',)
32516
37d70ba1d9d1 bundle: add an experimental knob to include obsmarkers in bundle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32515
diff changeset
1683 bundle = bundle20(ui, caps)
32221
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32186
diff changeset
1684 bundle.setcompression(compression, compopts)
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32186
diff changeset
1685 _addpartsfromopts(ui, repo, bundle, source, outgoing, opts)
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32186
diff changeset
1686 chunkiter = bundle.getchunks()
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32186
diff changeset
1687
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32186
diff changeset
1688 return changegroup.writechunks(ui, chunkiter, filename, vfs=vfs)
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32186
diff changeset
1689
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1690
32221
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32186
diff changeset
1691 def _addpartsfromopts(ui, repo, bundler, source, outgoing, opts):
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32186
diff changeset
1692 # We should eventually reconcile this logic with the one behind
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32186
diff changeset
1693 # 'exchange.getbundle2partsgenerator'.
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32186
diff changeset
1694 #
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32186
diff changeset
1695 # The type of input from 'getbundle' and 'writenewbundle' are a bit
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32186
diff changeset
1696 # different right now. So we keep them separated for now for the sake of
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32186
diff changeset
1697 # simplicity.
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32186
diff changeset
1698
37005
66c0ff381cfc bundle: condition the changegroup part when creating a new bundle
Boris Feld <boris.feld@octobus.net>
parents: 36968
diff changeset
1699 # we might not always want a changegroup in such bundle, for example in
66c0ff381cfc bundle: condition the changegroup part when creating a new bundle
Boris Feld <boris.feld@octobus.net>
parents: 36968
diff changeset
1700 # stream bundles
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1701 if opts.get(b'changegroup', True):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1702 cgversion = opts.get(b'cg.version')
37005
66c0ff381cfc bundle: condition the changegroup part when creating a new bundle
Boris Feld <boris.feld@octobus.net>
parents: 36968
diff changeset
1703 if cgversion is None:
66c0ff381cfc bundle: condition the changegroup part when creating a new bundle
Boris Feld <boris.feld@octobus.net>
parents: 36968
diff changeset
1704 cgversion = changegroup.safeversion(repo)
66c0ff381cfc bundle: condition the changegroup part when creating a new bundle
Boris Feld <boris.feld@octobus.net>
parents: 36968
diff changeset
1705 cg = changegroup.makechangegroup(repo, outgoing, cgversion, source)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1706 part = bundler.newpart(b'changegroup', data=cg.getchunks())
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1707 part.addparam(b'version', cg.version)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1708 if b'clcount' in cg.extras:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1709 part.addparam(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1710 b'nbchanges', b'%d' % cg.extras[b'clcount'], mandatory=False
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1711 )
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1712 if opts.get(b'phases') and repo.revs(
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1713 b'%ln and secret()', outgoing.missingheads
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1714 ):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1715 part.addparam(
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1716 b'targetphase', b'%d' % phases.secret, mandatory=False
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1717 )
43131
c17a63eb5d4c sidedata: apply basic but tight security around exchange
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43117
diff changeset
1718 if b'exp-sidedata-flag' in repo.requirements:
c17a63eb5d4c sidedata: apply basic but tight security around exchange
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43117
diff changeset
1719 part.addparam(b'exp-sidedata', b'1')
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1720
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1721 if opts.get(b'streamv2', False):
37167
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part
Boris Feld <boris.feld@octobus.net>
parents: 37165
diff changeset
1722 addpartbundlestream2(bundler, repo, stream=True)
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part
Boris Feld <boris.feld@octobus.net>
parents: 37165
diff changeset
1723
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1724 if opts.get(b'tagsfnodescache', True):
37165
6c7a6b04b274 bundlespec: move computing the bundle contentops in parsebundlespec
Boris Feld <boris.feld@octobus.net>
parents: 37084
diff changeset
1725 addparttagsfnodescache(repo, bundler, outgoing)
6c7a6b04b274 bundlespec: move computing the bundle contentops in parsebundlespec
Boris Feld <boris.feld@octobus.net>
parents: 37084
diff changeset
1726
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1727 if opts.get(b'revbranchcache', True):
37165
6c7a6b04b274 bundlespec: move computing the bundle contentops in parsebundlespec
Boris Feld <boris.feld@octobus.net>
parents: 37084
diff changeset
1728 addpartrevbranchcache(repo, bundler, outgoing)
32223
d7f93ebbbbdf bundle: add optional 'tagsfnodecache' data to on disk bundle (issue5543)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32222
diff changeset
1729
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1730 if opts.get(b'obsolescence', False):
32516
37d70ba1d9d1 bundle: add an experimental knob to include obsmarkers in bundle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32515
diff changeset
1731 obsmarkers = repo.obsstore.relevantmarkers(outgoing.missing)
37d70ba1d9d1 bundle: add an experimental knob to include obsmarkers in bundle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32515
diff changeset
1732 buildobsmarkerspart(bundler, obsmarkers)
37d70ba1d9d1 bundle: add an experimental knob to include obsmarkers in bundle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32515
diff changeset
1733
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1734 if opts.get(b'phases', False):
33031
e8c8d81eb864 bundle: add config option to include phases
Martin von Zweigbergk <martinvonz@google.com>
parents: 33030
diff changeset
1735 headsbyphase = phases.subsetphaseheads(repo, outgoing.missing)
34319
5779d096a696 phases: move binary encoding into a reusable function
Boris Feld <boris.feld@octobus.net>
parents: 34318
diff changeset
1736 phasedata = phases.binaryencode(headsbyphase)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1737 bundler.newpart(b'phase-heads', data=phasedata)
33031
e8c8d81eb864 bundle: add config option to include phases
Martin von Zweigbergk <martinvonz@google.com>
parents: 33030
diff changeset
1738
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1739
32222
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32221
diff changeset
1740 def addparttagsfnodescache(repo, bundler, outgoing):
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32221
diff changeset
1741 # we include the tags fnode cache for the bundle changeset
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32221
diff changeset
1742 # (as an optional parts)
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32221
diff changeset
1743 cache = tags.hgtagsfnodescache(repo.unfiltered())
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32221
diff changeset
1744 chunks = []
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32221
diff changeset
1745
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32221
diff changeset
1746 # .hgtags fnodes are only relevant for head changesets. While we could
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32221
diff changeset
1747 # transfer values for all known nodes, there will likely be little to
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32221
diff changeset
1748 # no benefit.
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32221
diff changeset
1749 #
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32221
diff changeset
1750 # We don't bother using a generator to produce output data because
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32221
diff changeset
1751 # a) we only have 40 bytes per head and even esoteric numbers of heads
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32221
diff changeset
1752 # consume little memory (1M heads is 40MB) b) we don't want to send the
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32221
diff changeset
1753 # part if we don't have entries and knowing if we have entries requires
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32221
diff changeset
1754 # cache lookups.
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32221
diff changeset
1755 for node in outgoing.missingheads:
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32221
diff changeset
1756 # Don't compute missing, as this may slow down serving.
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32221
diff changeset
1757 fnode = cache.getfnode(node, computemissing=False)
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32221
diff changeset
1758 if fnode is not None:
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32221
diff changeset
1759 chunks.extend([node, fnode])
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32221
diff changeset
1760
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32221
diff changeset
1761 if chunks:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1762 bundler.newpart(b'hgtagsfnodes', data=b''.join(chunks))
32222
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32221
diff changeset
1763
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1764
36964
79b73be4dda5 rev-branch-cache: add a function to generate a part
Boris Feld <boris.feld@octobus.net>
parents: 36963
diff changeset
1765 def addpartrevbranchcache(repo, bundler, outgoing):
79b73be4dda5 rev-branch-cache: add a function to generate a part
Boris Feld <boris.feld@octobus.net>
parents: 36963
diff changeset
1766 # we include the rev branch cache for the bundle changeset
79b73be4dda5 rev-branch-cache: add a function to generate a part
Boris Feld <boris.feld@octobus.net>
parents: 36963
diff changeset
1767 # (as an optional parts)
79b73be4dda5 rev-branch-cache: add a function to generate a part
Boris Feld <boris.feld@octobus.net>
parents: 36963
diff changeset
1768 cache = repo.revbranchcache()
79b73be4dda5 rev-branch-cache: add a function to generate a part
Boris Feld <boris.feld@octobus.net>
parents: 36963
diff changeset
1769 cl = repo.unfiltered().changelog
79b73be4dda5 rev-branch-cache: add a function to generate a part
Boris Feld <boris.feld@octobus.net>
parents: 36963
diff changeset
1770 branchesdata = collections.defaultdict(lambda: (set(), set()))
79b73be4dda5 rev-branch-cache: add a function to generate a part
Boris Feld <boris.feld@octobus.net>
parents: 36963
diff changeset
1771 for node in outgoing.missing:
79b73be4dda5 rev-branch-cache: add a function to generate a part
Boris Feld <boris.feld@octobus.net>
parents: 36963
diff changeset
1772 branch, close = cache.branchinfo(cl.rev(node))
79b73be4dda5 rev-branch-cache: add a function to generate a part
Boris Feld <boris.feld@octobus.net>
parents: 36963
diff changeset
1773 branchesdata[branch][close].add(node)
79b73be4dda5 rev-branch-cache: add a function to generate a part
Boris Feld <boris.feld@octobus.net>
parents: 36963
diff changeset
1774
79b73be4dda5 rev-branch-cache: add a function to generate a part
Boris Feld <boris.feld@octobus.net>
parents: 36963
diff changeset
1775 def generate():
79b73be4dda5 rev-branch-cache: add a function to generate a part
Boris Feld <boris.feld@octobus.net>
parents: 36963
diff changeset
1776 for branch, (nodes, closed) in sorted(branchesdata.items()):
79b73be4dda5 rev-branch-cache: add a function to generate a part
Boris Feld <boris.feld@octobus.net>
parents: 36963
diff changeset
1777 utf8branch = encoding.fromlocal(branch)
79b73be4dda5 rev-branch-cache: add a function to generate a part
Boris Feld <boris.feld@octobus.net>
parents: 36963
diff changeset
1778 yield rbcstruct.pack(len(utf8branch), len(nodes), len(closed))
79b73be4dda5 rev-branch-cache: add a function to generate a part
Boris Feld <boris.feld@octobus.net>
parents: 36963
diff changeset
1779 yield utf8branch
79b73be4dda5 rev-branch-cache: add a function to generate a part
Boris Feld <boris.feld@octobus.net>
parents: 36963
diff changeset
1780 for n in sorted(nodes):
79b73be4dda5 rev-branch-cache: add a function to generate a part
Boris Feld <boris.feld@octobus.net>
parents: 36963
diff changeset
1781 yield n
79b73be4dda5 rev-branch-cache: add a function to generate a part
Boris Feld <boris.feld@octobus.net>
parents: 36963
diff changeset
1782 for n in sorted(closed):
79b73be4dda5 rev-branch-cache: add a function to generate a part
Boris Feld <boris.feld@octobus.net>
parents: 36963
diff changeset
1783 yield n
79b73be4dda5 rev-branch-cache: add a function to generate a part
Boris Feld <boris.feld@octobus.net>
parents: 36963
diff changeset
1784
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1785 bundler.newpart(b'cache:rev-branch-cache', data=generate(), mandatory=False)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1786
36964
79b73be4dda5 rev-branch-cache: add a function to generate a part
Boris Feld <boris.feld@octobus.net>
parents: 36963
diff changeset
1787
37167
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part
Boris Feld <boris.feld@octobus.net>
parents: 37165
diff changeset
1788 def _formatrequirementsspec(requirements):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1789 requirements = [req for req in requirements if req != b"shared"]
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1790 return urlreq.quote(b','.join(sorted(requirements)))
37167
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part
Boris Feld <boris.feld@octobus.net>
parents: 37165
diff changeset
1791
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1792
37167
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part
Boris Feld <boris.feld@octobus.net>
parents: 37165
diff changeset
1793 def _formatrequirementsparams(requirements):
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part
Boris Feld <boris.feld@octobus.net>
parents: 37165
diff changeset
1794 requirements = _formatrequirementsspec(requirements)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1795 params = b"%s%s" % (urlreq.quote(b"requirements="), requirements)
37167
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part
Boris Feld <boris.feld@octobus.net>
parents: 37165
diff changeset
1796 return params
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part
Boris Feld <boris.feld@octobus.net>
parents: 37165
diff changeset
1797
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1798
37167
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part
Boris Feld <boris.feld@octobus.net>
parents: 37165
diff changeset
1799 def addpartbundlestream2(bundler, repo, **kwargs):
43506
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43495
diff changeset
1800 if not kwargs.get('stream', False):
37167
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part
Boris Feld <boris.feld@octobus.net>
parents: 37165
diff changeset
1801 return
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part
Boris Feld <boris.feld@octobus.net>
parents: 37165
diff changeset
1802
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part
Boris Feld <boris.feld@octobus.net>
parents: 37165
diff changeset
1803 if not streamclone.allowservergeneration(repo):
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1804 raise error.Abort(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1805 _(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1806 b'stream data requested but server does not allow '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1807 b'this feature'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1808 ),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1809 hint=_(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1810 b'well-behaved clients should not be '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1811 b'requesting stream data from servers not '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1812 b'advertising it; the client may be buggy'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1813 ),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1814 )
37167
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part
Boris Feld <boris.feld@octobus.net>
parents: 37165
diff changeset
1815
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part
Boris Feld <boris.feld@octobus.net>
parents: 37165
diff changeset
1816 # Stream clones don't compress well. And compression undermines a
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part
Boris Feld <boris.feld@octobus.net>
parents: 37165
diff changeset
1817 # goal of stream clones, which is to be fast. Communicate the desire
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part
Boris Feld <boris.feld@octobus.net>
parents: 37165
diff changeset
1818 # to avoid compression to consumers of the bundle.
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part
Boris Feld <boris.feld@octobus.net>
parents: 37165
diff changeset
1819 bundler.prefercompressed = False
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part
Boris Feld <boris.feld@octobus.net>
parents: 37165
diff changeset
1820
40338
af62936c2508 streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40147
diff changeset
1821 # get the includes and excludes
43506
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43495
diff changeset
1822 includepats = kwargs.get('includepats')
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43495
diff changeset
1823 excludepats = kwargs.get('excludepats')
40338
af62936c2508 streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40147
diff changeset
1824
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1825 narrowstream = repo.ui.configbool(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1826 b'experimental', b'server.stream-narrow-clones'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1827 )
40338
af62936c2508 streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40147
diff changeset
1828
af62936c2508 streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40147
diff changeset
1829 if (includepats or excludepats) and not narrowstream:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1830 raise error.Abort(_(b'server does not support narrow stream clones'))
40338
af62936c2508 streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40147
diff changeset
1831
40398
0ac794e0e285 streamclone: include obsstore file into stream bundle if client can read it
Anton Shestakov <av6@dwimlabs.net>
parents: 40372
diff changeset
1832 includeobsmarkers = False
0ac794e0e285 streamclone: include obsstore file into stream bundle if client can read it
Anton Shestakov <av6@dwimlabs.net>
parents: 40372
diff changeset
1833 if repo.obsstore:
0ac794e0e285 streamclone: include obsstore file into stream bundle if client can read it
Anton Shestakov <av6@dwimlabs.net>
parents: 40372
diff changeset
1834 remoteversions = obsmarkersversion(bundler.capabilities)
40399
4ab6e7b4fe8a streamclone: abort when client needs to handle obsmarkers, but doesn't
Anton Shestakov <av6@dwimlabs.net>
parents: 40398
diff changeset
1835 if not remoteversions:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1836 raise error.Abort(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1837 _(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1838 b'server has obsolescence markers, but client '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1839 b'cannot receive them via stream clone'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1840 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1841 )
40399
4ab6e7b4fe8a streamclone: abort when client needs to handle obsmarkers, but doesn't
Anton Shestakov <av6@dwimlabs.net>
parents: 40398
diff changeset
1842 elif repo.obsstore._version in remoteversions:
40398
0ac794e0e285 streamclone: include obsstore file into stream bundle if client can read it
Anton Shestakov <av6@dwimlabs.net>
parents: 40372
diff changeset
1843 includeobsmarkers = True
0ac794e0e285 streamclone: include obsstore file into stream bundle if client can read it
Anton Shestakov <av6@dwimlabs.net>
parents: 40372
diff changeset
1844
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1845 filecount, bytecount, it = streamclone.generatev2(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1846 repo, includepats, excludepats, includeobsmarkers
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1847 )
37167
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part
Boris Feld <boris.feld@octobus.net>
parents: 37165
diff changeset
1848 requirements = _formatrequirementsspec(repo.requirements)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1849 part = bundler.newpart(b'stream2', data=it)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1850 part.addparam(b'bytecount', b'%d' % bytecount, mandatory=True)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1851 part.addparam(b'filecount', b'%d' % filecount, mandatory=True)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1852 part.addparam(b'requirements', requirements, mandatory=True)
37167
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part
Boris Feld <boris.feld@octobus.net>
parents: 37165
diff changeset
1853
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1854
32515
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32387
diff changeset
1855 def buildobsmarkerspart(bundler, markers):
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32387
diff changeset
1856 """add an obsmarker part to the bundler with <markers>
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32387
diff changeset
1857
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32387
diff changeset
1858 No part is created if markers is empty.
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32387
diff changeset
1859 Raises ValueError if the bundler doesn't support any known obsmarker format.
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32387
diff changeset
1860 """
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32387
diff changeset
1861 if not markers:
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32387
diff changeset
1862 return None
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32387
diff changeset
1863
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32387
diff changeset
1864 remoteversions = obsmarkersversion(bundler.capabilities)
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32387
diff changeset
1865 version = obsolete.commonversion(remoteversions)
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32387
diff changeset
1866 if version is None:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1867 raise ValueError(b'bundler does not support common obsmarker format')
32515
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32387
diff changeset
1868 stream = obsolete.encodemarkers(markers, True, version=version)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1869 return bundler.newpart(b'obsmarkers', data=stream)
32515
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32387
diff changeset
1870
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1871
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1872 def writebundle(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1873 ui, cg, filename, bundletype, vfs=None, compression=None, compopts=None
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1874 ):
28666
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1875 """Write a bundle file and return its filename.
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1876
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1877 Existing files will not be overwritten.
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1878 If no filename is specified, a temporary file is created.
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1879 bz2 compression can be turned off.
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1880 The bundle file will be deleted in case of errors.
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1881 """
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1882
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1883 if bundletype == b"HG20":
28666
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1884 bundle = bundle20(ui)
30757
511a4bf52754 bundle2: allow compression options to be passed to compressor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30439
diff changeset
1885 bundle.setcompression(compression, compopts)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1886 part = bundle.newpart(b'changegroup', data=cg.getchunks())
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1887 part.addparam(b'version', cg.version)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1888 if b'clcount' in cg.extras:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1889 part.addparam(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1890 b'nbchanges', b'%d' % cg.extras[b'clcount'], mandatory=False
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1891 )
28666
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1892 chunkiter = bundle.getchunks()
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1893 else:
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1894 # compression argument is only for the bundle2 case
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1895 assert compression is None
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1896 if cg.version != b'01':
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1897 raise error.Abort(
43117
8ff1ecfadcd1 cleanup: join string literals that are already on one line
Martin von Zweigbergk <martinvonz@google.com>
parents: 43115
diff changeset
1898 _(b'old bundle types only supports v1 changegroups')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1899 )
28666
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1900 header, comp = bundletypes[bundletype]
30351
f81002f736d7 bundle2: use new compression engine API for compression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30177
diff changeset
1901 if comp not in util.compengines.supportedbundletypes:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1902 raise error.Abort(_(b'unknown stream compression type: %s') % comp)
30351
f81002f736d7 bundle2: use new compression engine API for compression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30177
diff changeset
1903 compengine = util.compengines.forbundletype(comp)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1904
28666
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1905 def chunkiter():
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1906 yield header
30757
511a4bf52754 bundle2: allow compression options to be passed to compressor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30439
diff changeset
1907 for chunk in compengine.compressstream(cg.getchunks(), compopts):
30357
5925bda42dbd bundle2: use compressstream compression engine API
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30353
diff changeset
1908 yield chunk
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1909
28666
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1910 chunkiter = chunkiter()
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1911
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1912 # parse the changegroup data, otherwise we will block
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1913 # in case of sshrepo because we don't know the end of the stream
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1914 return changegroup.writechunks(ui, chunkiter, filename, vfs=vfs)
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1915
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1916
33037
d765ad56081f bundle: make combinechangegroupresults() take a bundleoperation
Martin von Zweigbergk <martinvonz@google.com>
parents: 33036
diff changeset
1917 def combinechangegroupresults(op):
33036
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33035
diff changeset
1918 """logic to combine 0 or more addchangegroup results into one"""
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1919 results = [r.get(b'return', 0) for r in op.records[b'changegroup']]
33036
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33035
diff changeset
1920 changedheads = 0
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33035
diff changeset
1921 result = 1
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33035
diff changeset
1922 for ret in results:
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33035
diff changeset
1923 # If any changegroup result is 0, return 0
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33035
diff changeset
1924 if ret == 0:
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33035
diff changeset
1925 result = 0
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33035
diff changeset
1926 break
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33035
diff changeset
1927 if ret < -1:
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33035
diff changeset
1928 changedheads += ret + 1
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33035
diff changeset
1929 elif ret > 1:
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33035
diff changeset
1930 changedheads += ret - 1
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33035
diff changeset
1931 if changedheads > 0:
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33035
diff changeset
1932 result = 1 + changedheads
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33035
diff changeset
1933 elif changedheads < 0:
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33035
diff changeset
1934 result = -1 + changedheads
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33035
diff changeset
1935 return result
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33035
diff changeset
1936
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1937
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1938 @parthandler(
43131
c17a63eb5d4c sidedata: apply basic but tight security around exchange
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43117
diff changeset
1939 b'changegroup',
c17a63eb5d4c sidedata: apply basic but tight security around exchange
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43117
diff changeset
1940 (
c17a63eb5d4c sidedata: apply basic but tight security around exchange
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43117
diff changeset
1941 b'version',
c17a63eb5d4c sidedata: apply basic but tight security around exchange
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43117
diff changeset
1942 b'nbchanges',
c17a63eb5d4c sidedata: apply basic but tight security around exchange
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43117
diff changeset
1943 b'exp-sidedata',
c17a63eb5d4c sidedata: apply basic but tight security around exchange
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43117
diff changeset
1944 b'treemanifest',
c17a63eb5d4c sidedata: apply basic but tight security around exchange
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43117
diff changeset
1945 b'targetphase',
c17a63eb5d4c sidedata: apply basic but tight security around exchange
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43117
diff changeset
1946 ),
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1947 )
20998
93a3c5b58635 bundle2: use reply part to return result of addchangegroup
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20997
diff changeset
1948 def handlechangegroup(op, inpart):
20950
c7ceae0faf69 bundle2: first crude version of bundling changeset with bundle2
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20949
diff changeset
1949 """apply a changegroup part on the repo
c7ceae0faf69 bundle2: first crude version of bundling changeset with bundle2
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20949
diff changeset
1950
c7ceae0faf69 bundle2: first crude version of bundling changeset with bundle2
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20949
diff changeset
1951 This is a very early implementation that will massive rework before being
c7ceae0faf69 bundle2: first crude version of bundling changeset with bundle2
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20949
diff changeset
1952 inflicted to any end-user.
c7ceae0faf69 bundle2: first crude version of bundling changeset with bundle2
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20949
diff changeset
1953 """
39700
b10d145837bc localrepo: extract resolving of opener options to standalone functions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38783
diff changeset
1954 from . import localrepo
b10d145837bc localrepo: extract resolving of opener options to standalone functions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38783
diff changeset
1955
32930
af31d531dda0 changegroup: let callers pass in transaction to apply() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 32892
diff changeset
1956 tr = op.gettransaction()
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1957 unpackerversion = inpart.params.get(b'version', b'01')
23170
02e8f9b60052 bundle2: support a "version" argument in `changegroup` part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23169
diff changeset
1958 # We should raise an appropriate exception here
27751
a40e2f7fe49d changegroup: hide packermap behind methods
Martin von Zweigbergk <martinvonz@google.com>
parents: 27734
diff changeset
1959 cg = changegroup.getunbundler(unpackerversion, inpart, None)
23001
4df9b5e62f70 bundle2: add a comment about addchangegroup source and url
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22953
diff changeset
1960 # the source and url passed here are overwritten by the one contained in
4df9b5e62f70 bundle2: add a comment about addchangegroup source and url
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22953
diff changeset
1961 # the transaction.hookargs argument. So 'bundle2' is a placeholder
25518
ca656f3dffd7 bundle2: provide number of changesets information to 'addchangegroup'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25507
diff changeset
1962 nbchangesets = None
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1963 if b'nbchanges' in inpart.params:
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1964 nbchangesets = int(inpart.params.get(b'nbchanges'))
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1965 if (
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1966 b'treemanifest' in inpart.params
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1967 and b'treemanifest' not in op.repo.requirements
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1968 ):
27734
5c0fd878779c treemanifests: set bundle2 part parameter indicating treemanifest
Martin von Zweigbergk <martinvonz@google.com>
parents: 26829
diff changeset
1969 if len(op.repo.changelog) != 0:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1970 raise error.Abort(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1971 _(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1972 b"bundle contains tree manifests, but local repo is "
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1973 b"non-empty and does not use tree manifests"
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1974 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1975 )
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1976 op.repo.requirements.add(b'treemanifest')
39700
b10d145837bc localrepo: extract resolving of opener options to standalone functions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38783
diff changeset
1977 op.repo.svfs.options = localrepo.resolvestorevfsoptions(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1978 op.repo.ui, op.repo.requirements, op.repo.features
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1979 )
27734
5c0fd878779c treemanifests: set bundle2 part parameter indicating treemanifest
Martin von Zweigbergk <martinvonz@google.com>
parents: 26829
diff changeset
1980 op.repo._writerequirements()
43131
c17a63eb5d4c sidedata: apply basic but tight security around exchange
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43117
diff changeset
1981
c17a63eb5d4c sidedata: apply basic but tight security around exchange
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43117
diff changeset
1982 bundlesidedata = bool(b'exp-sidedata' in inpart.params)
c17a63eb5d4c sidedata: apply basic but tight security around exchange
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43117
diff changeset
1983 reposidedata = bool(b'exp-sidedata-flag' in op.repo.requirements)
c17a63eb5d4c sidedata: apply basic but tight security around exchange
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43117
diff changeset
1984 if reposidedata and not bundlesidedata:
c17a63eb5d4c sidedata: apply basic but tight security around exchange
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43117
diff changeset
1985 msg = b"repository is using sidedata but the bundle source do not"
c17a63eb5d4c sidedata: apply basic but tight security around exchange
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43117
diff changeset
1986 hint = b'this is currently unsupported'
c17a63eb5d4c sidedata: apply basic but tight security around exchange
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43117
diff changeset
1987 raise error.Abort(msg, hint=hint)
c17a63eb5d4c sidedata: apply basic but tight security around exchange
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43117
diff changeset
1988
33407
39d4e5a66f5a bundle2: support the 'targetphase' parameter for the changegroup part
Boris Feld <boris.feld@octobus.net>
parents: 33252
diff changeset
1989 extrakwargs = {}
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1990 targetphase = inpart.params.get(b'targetphase')
33407
39d4e5a66f5a bundle2: support the 'targetphase' parameter for the changegroup part
Boris Feld <boris.feld@octobus.net>
parents: 33252
diff changeset
1991 if targetphase is not None:
43506
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43495
diff changeset
1992 extrakwargs['targetphase'] = int(targetphase)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1993 ret = _processchangegroup(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1994 op,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1995 cg,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1996 tr,
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1997 b'bundle2',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1998 b'bundle2',
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1999 expectedtotal=nbchangesets,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2000 **extrakwargs
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2001 )
20998
93a3c5b58635 bundle2: use reply part to return result of addchangegroup
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20997
diff changeset
2002 if op.reply is not None:
23139
e53f6b72a0e4 spelling: fixes from proofreading of spell checker issues
Mads Kiilerich <madski@unity3d.com>
parents: 23067
diff changeset
2003 # This is definitely not the final form of this
20998
93a3c5b58635 bundle2: use reply part to return result of addchangegroup
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20997
diff changeset
2004 # return. But one need to start somewhere.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2005 part = op.reply.newpart(b'reply:changegroup', mandatory=False)
33675
24b7c75c6441 bundle2: use bytestr() instead of str() to convert part id to bytes
Augie Fackler <augie@google.com>
parents: 33674
diff changeset
2006 part.addparam(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2007 b'in-reply-to', pycompat.bytestr(inpart.id), mandatory=False
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2008 )
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2009 part.addparam(b'return', b'%i' % ret, mandatory=False)
21019
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
2010 assert not inpart.read()
20950
c7ceae0faf69 bundle2: first crude version of bundling changeset with bundle2
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20949
diff changeset
2011
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2012
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2013 _remotechangegroupparams = tuple(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2014 [b'url', b'size', b'digests']
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2015 + [b'digest:%s' % k for k in util.DIGESTS.keys()]
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2016 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2017
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2018
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2019 @parthandler(b'remote-changegroup', _remotechangegroupparams)
23029
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2020 def handleremotechangegroup(op, inpart):
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2021 """apply a bundle10 on the repo, given an url and validation information
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2022
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2023 All the information about the remote bundle to import are given as
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2024 parameters. The parameters include:
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2025 - url: the url to the bundle10.
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2026 - size: the bundle10 file size. It is used to validate what was
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2027 retrieved by the client matches the server knowledge about the bundle.
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2028 - digests: a space separated list of the digest types provided as
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2029 parameters.
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2030 - digest:<digest-type>: the hexadecimal representation of the digest with
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2031 that name. Like the size, it is used to validate what was retrieved by
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2032 the client matches what the server knows about the bundle.
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2033
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2034 When multiple digest types are given, all of them are checked.
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2035 """
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2036 try:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2037 raw_url = inpart.params[b'url']
23029
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2038 except KeyError:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2039 raise error.Abort(_(b'remote-changegroup: missing "%s" param') % b'url')
23029
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2040 parsed_url = util.url(raw_url)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2041 if parsed_url.scheme not in capabilities[b'remote-changegroup']:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2042 raise error.Abort(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2043 _(b'remote-changegroup does not support %s urls')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2044 % parsed_url.scheme
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2045 )
23029
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2046
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2047 try:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2048 size = int(inpart.params[b'size'])
23029
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2049 except ValueError:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2050 raise error.Abort(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2051 _(b'remote-changegroup: invalid value for param "%s"') % b'size'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2052 )
23029
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2053 except KeyError:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2054 raise error.Abort(
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2055 _(b'remote-changegroup: missing "%s" param') % b'size'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2056 )
23029
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2057
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2058 digests = {}
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2059 for typ in inpart.params.get(b'digests', b'').split():
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2060 param = b'digest:%s' % typ
23029
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2061 try:
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2062 value = inpart.params[param]
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2063 except KeyError:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2064 raise error.Abort(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2065 _(b'remote-changegroup: missing "%s" param') % param
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2066 )
23029
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2067 digests[typ] = value
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2068
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2069 real_part = util.digestchecker(url.open(op.ui, raw_url), size, digests)
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2070
32930
af31d531dda0 changegroup: let callers pass in transaction to apply() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 32892
diff changeset
2071 tr = op.gettransaction()
25919
8221fefaea08 bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
2072 from . import exchange
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2073
23029
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2074 cg = exchange.readbundle(op.repo.ui, real_part, raw_url)
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2075 if not isinstance(cg, changegroup.cg1unpacker):
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2076 raise error.Abort(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2077 _(b'%s: not a bundle version 1.0') % util.hidepassword(raw_url)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2078 )
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2079 ret = _processchangegroup(op, cg, tr, b'bundle2', b'bundle2')
23029
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2080 if op.reply is not None:
23139
e53f6b72a0e4 spelling: fixes from proofreading of spell checker issues
Mads Kiilerich <madski@unity3d.com>
parents: 23067
diff changeset
2081 # This is definitely not the final form of this
23029
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2082 # return. But one need to start somewhere.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2083 part = op.reply.newpart(b'reply:changegroup')
33675
24b7c75c6441 bundle2: use bytestr() instead of str() to convert part id to bytes
Augie Fackler <augie@google.com>
parents: 33674
diff changeset
2084 part.addparam(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2085 b'in-reply-to', pycompat.bytestr(inpart.id), mandatory=False
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2086 )
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2087 part.addparam(b'return', b'%i' % ret, mandatory=False)
23029
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2088 try:
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2089 real_part.validate()
26587
56b2bcea2529 error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26565
diff changeset
2090 except error.Abort as e:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2091 raise error.Abort(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2092 _(b'bundle at %s is corrupted:\n%s')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2093 % (util.hidepassword(raw_url), bytes(e))
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2094 )
23029
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2095 assert not inpart.read()
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2096
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2097
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2098 @parthandler(b'reply:changegroup', (b'return', b'in-reply-to'))
22548
8a1ae133770a bundle2: rename functions that have the same name
Mike Hommey <mh@glandium.org>
parents: 22390
diff changeset
2099 def handlereplychangegroup(op, inpart):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2100 ret = int(inpart.params[b'return'])
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2101 replyto = int(inpart.params[b'in-reply-to'])
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2102 op.records.add(b'changegroup', {b'return': ret}, replyto)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2103
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2104
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2105 @parthandler(b'check:bookmarks')
35258
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35120
diff changeset
2106 def handlecheckbookmarks(op, inpart):
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35120
diff changeset
2107 """check location of bookmarks
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35120
diff changeset
2108
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35120
diff changeset
2109 This part is to be used to detect push race regarding bookmark, it
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35120
diff changeset
2110 contains binary encoded (bookmark, node) tuple. If the local state does
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35120
diff changeset
2111 not marks the one in the part, a PushRaced exception is raised
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35120
diff changeset
2112 """
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35120
diff changeset
2113 bookdata = bookmarks.binarydecode(inpart)
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35120
diff changeset
2114
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2115 msgstandard = (
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2116 b'remote repository changed while pushing - please try again '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2117 b'(bookmark "%s" move from %s to %s)'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2118 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2119 msgmissing = (
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2120 b'remote repository changed while pushing - please try again '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2121 b'(bookmark "%s" is missing, expected %s)'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2122 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2123 msgexist = (
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2124 b'remote repository changed while pushing - please try again '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2125 b'(bookmark "%s" set on %s, expected missing)'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2126 )
35258
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35120
diff changeset
2127 for book, node in bookdata:
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35120
diff changeset
2128 currentnode = op.repo._bookmarks.get(book)
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35120
diff changeset
2129 if currentnode != node:
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35120
diff changeset
2130 if node is None:
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35120
diff changeset
2131 finalmsg = msgexist % (book, nodemod.short(currentnode))
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35120
diff changeset
2132 elif currentnode is None:
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35120
diff changeset
2133 finalmsg = msgmissing % (book, nodemod.short(node))
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35120
diff changeset
2134 else:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2135 finalmsg = msgstandard % (
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2136 book,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2137 nodemod.short(node),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2138 nodemod.short(currentnode),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2139 )
35258
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35120
diff changeset
2140 raise error.PushRaced(finalmsg)
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35120
diff changeset
2141
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2142
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2143 @parthandler(b'check:heads')
22548
8a1ae133770a bundle2: rename functions that have the same name
Mike Hommey <mh@glandium.org>
parents: 22390
diff changeset
2144 def handlecheckheads(op, inpart):
21060
0bea9db7543b bundle2: add a "check:heads" handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21024
diff changeset
2145 """check that head of the repo did not change
0bea9db7543b bundle2: add a "check:heads" handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21024
diff changeset
2146
0bea9db7543b bundle2: add a "check:heads" handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21024
diff changeset
2147 This is used to detect a push race when using unbundle.
0bea9db7543b bundle2: add a "check:heads" handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21024
diff changeset
2148 This replaces the "heads" argument of unbundle."""
0bea9db7543b bundle2: add a "check:heads" handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21024
diff changeset
2149 h = inpart.read(20)
0bea9db7543b bundle2: add a "check:heads" handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21024
diff changeset
2150 heads = []
0bea9db7543b bundle2: add a "check:heads" handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21024
diff changeset
2151 while len(h) == 20:
0bea9db7543b bundle2: add a "check:heads" handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21024
diff changeset
2152 heads.append(h)
0bea9db7543b bundle2: add a "check:heads" handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21024
diff changeset
2153 h = inpart.read(20)
0bea9db7543b bundle2: add a "check:heads" handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21024
diff changeset
2154 assert not h
26565
ee1bcb9aa6e4 bundle2: add op.gettransaction() to handlers that need the lock
Durham Goode <durham@fb.com>
parents: 26542
diff changeset
2155 # Trigger a transaction so that we are guaranteed to have the lock now.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2156 if op.ui.configbool(b'experimental', b'bundle2lazylocking'):
26565
ee1bcb9aa6e4 bundle2: add op.gettransaction() to handlers that need the lock
Durham Goode <durham@fb.com>
parents: 26542
diff changeset
2157 op.gettransaction()
29294
077d0535f51f bundle2: don't assume ordering of heads checked after push
Mads Kiilerich <madski@unity3d.com>
parents: 28883
diff changeset
2158 if sorted(heads) != sorted(op.repo.heads()):
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2159 raise error.PushRaced(
43117
8ff1ecfadcd1 cleanup: join string literals that are already on one line
Martin von Zweigbergk <martinvonz@google.com>
parents: 43115
diff changeset
2160 b'remote repository changed while pushing - please try again'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2161 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2162
21130
1ff06386217f bundle2: introduce `replycaps` part for on-demand reply
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21129
diff changeset
2163
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2164 @parthandler(b'check:updated-heads')
32709
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32516
diff changeset
2165 def handlecheckupdatedheads(op, inpart):
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32516
diff changeset
2166 """check for race on the heads touched by a push
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32516
diff changeset
2167
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32516
diff changeset
2168 This is similar to 'check:heads' but focus on the heads actually updated
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32516
diff changeset
2169 during the push. If other activities happen on unrelated heads, it is
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32516
diff changeset
2170 ignored.
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32516
diff changeset
2171
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32516
diff changeset
2172 This allow server with high traffic to avoid push contention as long as
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32516
diff changeset
2173 unrelated parts of the graph are involved."""
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32516
diff changeset
2174 h = inpart.read(20)
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32516
diff changeset
2175 heads = []
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32516
diff changeset
2176 while len(h) == 20:
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32516
diff changeset
2177 heads.append(h)
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32516
diff changeset
2178 h = inpart.read(20)
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32516
diff changeset
2179 assert not h
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32516
diff changeset
2180 # trigger a transaction so that we are guaranteed to have the lock now.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2181 if op.ui.configbool(b'experimental', b'bundle2lazylocking'):
32709
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32516
diff changeset
2182 op.gettransaction()
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32516
diff changeset
2183
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32516
diff changeset
2184 currentheads = set()
42002
662ffdde5adf branchcache: rename itervalues() to iterheads()
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41792
diff changeset
2185 for ls in op.repo.branchmap().iterheads():
32709
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32516
diff changeset
2186 currentheads.update(ls)
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32516
diff changeset
2187
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32516
diff changeset
2188 for h in heads:
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32516
diff changeset
2189 if h not in currentheads:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2190 raise error.PushRaced(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2191 b'remote repository changed while pushing - '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2192 b'please try again'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2193 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2194
32709
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32516
diff changeset
2195
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2196 @parthandler(b'check:phases')
34820
a95067b1dca6 phase: introduce a new 'check:phases' part
Boris Feld <boris.feld@octobus.net>
parents: 34637
diff changeset
2197 def handlecheckphases(op, inpart):
a95067b1dca6 phase: introduce a new 'check:phases' part
Boris Feld <boris.feld@octobus.net>
parents: 34637
diff changeset
2198 """check that phase boundaries of the repository did not change
a95067b1dca6 phase: introduce a new 'check:phases' part
Boris Feld <boris.feld@octobus.net>
parents: 34637
diff changeset
2199
a95067b1dca6 phase: introduce a new 'check:phases' part
Boris Feld <boris.feld@octobus.net>
parents: 34637
diff changeset
2200 This is used to detect a push race.
a95067b1dca6 phase: introduce a new 'check:phases' part
Boris Feld <boris.feld@octobus.net>
parents: 34637
diff changeset
2201 """
a95067b1dca6 phase: introduce a new 'check:phases' part
Boris Feld <boris.feld@octobus.net>
parents: 34637
diff changeset
2202 phasetonodes = phases.binarydecode(inpart)
a95067b1dca6 phase: introduce a new 'check:phases' part
Boris Feld <boris.feld@octobus.net>
parents: 34637
diff changeset
2203 unfi = op.repo.unfiltered()
a95067b1dca6 phase: introduce a new 'check:phases' part
Boris Feld <boris.feld@octobus.net>
parents: 34637
diff changeset
2204 cl = unfi.changelog
a95067b1dca6 phase: introduce a new 'check:phases' part
Boris Feld <boris.feld@octobus.net>
parents: 34637
diff changeset
2205 phasecache = unfi._phasecache
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2206 msg = (
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2207 b'remote repository changed while pushing - please try again '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2208 b'(%s is %s expected %s)'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2209 )
34820
a95067b1dca6 phase: introduce a new 'check:phases' part
Boris Feld <boris.feld@octobus.net>
parents: 34637
diff changeset
2210 for expectedphase, nodes in enumerate(phasetonodes):
a95067b1dca6 phase: introduce a new 'check:phases' part
Boris Feld <boris.feld@octobus.net>
parents: 34637
diff changeset
2211 for n in nodes:
a95067b1dca6 phase: introduce a new 'check:phases' part
Boris Feld <boris.feld@octobus.net>
parents: 34637
diff changeset
2212 actualphase = phasecache.phase(unfi, cl.rev(n))
a95067b1dca6 phase: introduce a new 'check:phases' part
Boris Feld <boris.feld@octobus.net>
parents: 34637
diff changeset
2213 if actualphase != expectedphase:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2214 finalmsg = msg % (
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2215 nodemod.short(n),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2216 phases.phasenames[actualphase],
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2217 phases.phasenames[expectedphase],
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2218 )
34820
a95067b1dca6 phase: introduce a new 'check:phases' part
Boris Feld <boris.feld@octobus.net>
parents: 34637
diff changeset
2219 raise error.PushRaced(finalmsg)
a95067b1dca6 phase: introduce a new 'check:phases' part
Boris Feld <boris.feld@octobus.net>
parents: 34637
diff changeset
2220
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2221
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2222 @parthandler(b'output')
21131
b7435117d951 bundle2: capture remote stdout while unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21130
diff changeset
2223 def handleoutput(op, inpart):
b7435117d951 bundle2: capture remote stdout while unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21130
diff changeset
2224 """forward output captured on the server to the client"""
b7435117d951 bundle2: capture remote stdout while unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21130
diff changeset
2225 for line in inpart.read().splitlines():
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2226 op.ui.status(_(b'remote: %s\n') % line)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2227
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2228
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2229 @parthandler(b'replycaps')
21130
1ff06386217f bundle2: introduce `replycaps` part for on-demand reply
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21129
diff changeset
2230 def handlereplycaps(op, inpart):
1ff06386217f bundle2: introduce `replycaps` part for on-demand reply
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21129
diff changeset
2231 """Notify that a reply bundle should be created
1ff06386217f bundle2: introduce `replycaps` part for on-demand reply
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21129
diff changeset
2232
21138
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
2233 The payload contains the capabilities information for the reply"""
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
2234 caps = decodecaps(inpart.read())
21130
1ff06386217f bundle2: introduce `replycaps` part for on-demand reply
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21129
diff changeset
2235 if op.reply is None:
21135
98fbf3adfd83 bundle2: add capabilities support in `replycaps` part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21134
diff changeset
2236 op.reply = bundle20(op.ui, caps)
21131
b7435117d951 bundle2: capture remote stdout while unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21130
diff changeset
2237
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2238
26829
58f1645f72c3 bundle2: attribute remote failures to remote (issue4788)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26802
diff changeset
2239 class AbortFromPart(error.Abort):
58f1645f72c3 bundle2: attribute remote failures to remote (issue4788)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26802
diff changeset
2240 """Sub-class of Abort that denotes an error from a bundle2 part."""
58f1645f72c3 bundle2: attribute remote failures to remote (issue4788)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26802
diff changeset
2241
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2242
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2243 @parthandler(b'error:abort', (b'message', b'hint'))
24741
bb67e52362d6 bundle2: fix names for error part handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24733
diff changeset
2244 def handleerrorabort(op, inpart):
21177
952af771bc17 bundle2: gracefully handle abort during unbundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21176
diff changeset
2245 """Used to transmit abort error over the wire"""
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2246 raise AbortFromPart(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2247 inpart.params[b'message'], hint=inpart.params.get(b'hint')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2248 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2249
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2250
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2251 @parthandler(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2252 b'error:pushkey',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2253 (b'namespace', b'key', b'new', b'old', b'ret', b'in-reply-to'),
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2254 )
25493
d8e7b0781ad7 bundle2: convey PushkeyFailed error over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25492
diff changeset
2255 def handleerrorpushkey(op, inpart):
d8e7b0781ad7 bundle2: convey PushkeyFailed error over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25492
diff changeset
2256 """Used to transmit failure of a mandatory pushkey over the wire"""
d8e7b0781ad7 bundle2: convey PushkeyFailed error over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25492
diff changeset
2257 kwargs = {}
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2258 for name in (b'namespace', b'key', b'new', b'old', b'ret'):
25493
d8e7b0781ad7 bundle2: convey PushkeyFailed error over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25492
diff changeset
2259 value = inpart.params.get(name)
d8e7b0781ad7 bundle2: convey PushkeyFailed error over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25492
diff changeset
2260 if value is not None:
d8e7b0781ad7 bundle2: convey PushkeyFailed error over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25492
diff changeset
2261 kwargs[name] = value
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2262 raise error.PushkeyFailed(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2263 inpart.params[b'in-reply-to'], **pycompat.strkwargs(kwargs)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2264 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2265
25493
d8e7b0781ad7 bundle2: convey PushkeyFailed error over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25492
diff changeset
2266
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2267 @parthandler(b'error:unsupportedcontent', (b'parttype', b'params'))
24741
bb67e52362d6 bundle2: fix names for error part handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24733
diff changeset
2268 def handleerrorunsupportedcontent(op, inpart):
21619
292331e906d7 bundle2: rename b2x:error:unknownpart to b2x:error:unsupportedcontent
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21618
diff changeset
2269 """Used to transmit unknown content error over the wire"""
21622
457492741007 bundle2: support transmission of params error over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21620
diff changeset
2270 kwargs = {}
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2271 parttype = inpart.params.get(b'parttype')
21627
3e8bcc90f07c bundle2: support None parttype in BundleValueError
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21626
diff changeset
2272 if parttype is not None:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2273 kwargs[b'parttype'] = parttype
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2274 params = inpart.params.get(b'params')
21622
457492741007 bundle2: support transmission of params error over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21620
diff changeset
2275 if params is not None:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2276 kwargs[b'params'] = params.split(b'\0')
21622
457492741007 bundle2: support transmission of params error over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21620
diff changeset
2277
36427
247b473f408e bundle2: **strkwargs love on various kwargs constructions
Augie Fackler <augie@google.com>
parents: 36273
diff changeset
2278 raise error.BundleUnknownFeatureError(**pycompat.strkwargs(kwargs))
21186
9f3652e851f8 bundle2: gracefully handle PushRaced error during unbundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21185
diff changeset
2279
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2280
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2281 @parthandler(b'error:pushraced', (b'message',))
24741
bb67e52362d6 bundle2: fix names for error part handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24733
diff changeset
2282 def handleerrorpushraced(op, inpart):
21186
9f3652e851f8 bundle2: gracefully handle PushRaced error during unbundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21185
diff changeset
2283 """Used to transmit push race error over the wire"""
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2284 raise error.ResponseError(_(b'push failed:'), inpart.params[b'message'])
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2285
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2286
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2287 @parthandler(b'listkeys', (b'namespace',))
21655
35095f332846 bundle: introduce a listkey handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21644
diff changeset
2288 def handlelistkeys(op, inpart):
35095f332846 bundle: introduce a listkey handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21644
diff changeset
2289 """retrieve pushkey namespace content stored in a bundle2"""
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2290 namespace = inpart.params[b'namespace']
21655
35095f332846 bundle: introduce a listkey handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21644
diff changeset
2291 r = pushkey.decodekeys(inpart.read())
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2292 op.records.add(b'listkeys', (namespace, r))
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2293
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2294
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2295 @parthandler(b'pushkey', (b'namespace', b'key', b'old', b'new'))
21660
e87d2a12d41b bundle2: add ``pushkey`` support
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21655
diff changeset
2296 def handlepushkey(op, inpart):
e87d2a12d41b bundle2: add ``pushkey`` support
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21655
diff changeset
2297 """process a pushkey request"""
e87d2a12d41b bundle2: add ``pushkey`` support
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21655
diff changeset
2298 dec = pushkey.decode
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2299 namespace = dec(inpart.params[b'namespace'])
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2300 key = dec(inpart.params[b'key'])
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2301 old = dec(inpart.params[b'old'])
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2302 new = dec(inpart.params[b'new'])
26565
ee1bcb9aa6e4 bundle2: add op.gettransaction() to handlers that need the lock
Durham Goode <durham@fb.com>
parents: 26542
diff changeset
2303 # Grab the transaction to ensure that we have the lock before performing the
ee1bcb9aa6e4 bundle2: add op.gettransaction() to handlers that need the lock
Durham Goode <durham@fb.com>
parents: 26542
diff changeset
2304 # pushkey.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2305 if op.ui.configbool(b'experimental', b'bundle2lazylocking'):
26565
ee1bcb9aa6e4 bundle2: add op.gettransaction() to handlers that need the lock
Durham Goode <durham@fb.com>
parents: 26542
diff changeset
2306 op.gettransaction()
21660
e87d2a12d41b bundle2: add ``pushkey`` support
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21655
diff changeset
2307 ret = op.repo.pushkey(namespace, key, old, new)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2308 record = {b'namespace': namespace, b'key': key, b'old': old, b'new': new}
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2309 op.records.add(b'pushkey', record)
21660
e87d2a12d41b bundle2: add ``pushkey`` support
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21655
diff changeset
2310 if op.reply is not None:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2311 rpart = op.reply.newpart(b'reply:pushkey')
33675
24b7c75c6441 bundle2: use bytestr() instead of str() to convert part id to bytes
Augie Fackler <augie@google.com>
parents: 33674
diff changeset
2312 rpart.addparam(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2313 b'in-reply-to', pycompat.bytestr(inpart.id), mandatory=False
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2314 )
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2315 rpart.addparam(b'return', b'%i' % ret, mandatory=False)
25481
6de96cb31176 bundle2: abort when a mandatory pushkey part fails
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25401
diff changeset
2316 if inpart.mandatory and not ret:
25484
a5192774e925 bundle2: introduce a PushkeyFail error to abort unbundle on pushkey error
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25481
diff changeset
2317 kwargs = {}
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2318 for key in (b'namespace', b'key', b'new', b'old', b'ret'):
25484
a5192774e925 bundle2: introduce a PushkeyFail error to abort unbundle on pushkey error
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25481
diff changeset
2319 if key in inpart.params:
a5192774e925 bundle2: introduce a PushkeyFail error to abort unbundle on pushkey error
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25481
diff changeset
2320 kwargs[key] = inpart.params[key]
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2321 raise error.PushkeyFailed(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2322 partid=b'%d' % inpart.id, **pycompat.strkwargs(kwargs)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2323 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2324
21660
e87d2a12d41b bundle2: add ``pushkey`` support
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21655
diff changeset
2325
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2326 @parthandler(b'bookmarks')
35260
af5507203d01 bookmark: introduce a 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35259
diff changeset
2327 def handlebookmark(op, inpart):
af5507203d01 bookmark: introduce a 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35259
diff changeset
2328 """transmit bookmark information
af5507203d01 bookmark: introduce a 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35259
diff changeset
2329
35266
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35265
diff changeset
2330 The part contains binary encoded bookmark information.
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35265
diff changeset
2331
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35265
diff changeset
2332 The exact behavior of this part can be controlled by the 'bookmarks' mode
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35265
diff changeset
2333 on the bundle operation.
35260
af5507203d01 bookmark: introduce a 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35259
diff changeset
2334
35266
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35265
diff changeset
2335 When mode is 'apply' (the default) the bookmark information is applied as
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35265
diff changeset
2336 is to the unbundling repository. Make sure a 'check:bookmarks' part is
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35265
diff changeset
2337 issued earlier to check for push races in such update. This behavior is
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35265
diff changeset
2338 suitable for pushing.
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35265
diff changeset
2339
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35265
diff changeset
2340 When mode is 'records', the information is recorded into the 'bookmarks'
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35265
diff changeset
2341 records of the bundle operation. This behavior is suitable for pulling.
35260
af5507203d01 bookmark: introduce a 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35259
diff changeset
2342 """
af5507203d01 bookmark: introduce a 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35259
diff changeset
2343 changes = bookmarks.binarydecode(inpart)
35261
f392066d127c bookmark: add pushkey hook compatiblity to the bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35260
diff changeset
2344
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2345 pushkeycompat = op.repo.ui.configbool(
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2346 b'server', b'bookmarks-pushkey-compat'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2347 )
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2348 bookmarksmode = op.modes.get(b'bookmarks', b'apply')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2349
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2350 if bookmarksmode == b'apply':
35266
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35265
diff changeset
2351 tr = op.gettransaction()
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35265
diff changeset
2352 bookstore = op.repo._bookmarks
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35265
diff changeset
2353 if pushkeycompat:
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35265
diff changeset
2354 allhooks = []
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35265
diff changeset
2355 for book, node in changes:
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35265
diff changeset
2356 hookargs = tr.hookargs.copy()
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2357 hookargs[b'pushkeycompat'] = b'1'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2358 hookargs[b'namespace'] = b'bookmarks'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2359 hookargs[b'key'] = book
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2360 hookargs[b'old'] = nodemod.hex(bookstore.get(book, b''))
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2361 hookargs[b'new'] = nodemod.hex(
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2362 node if node is not None else b''
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2363 )
35266
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35265
diff changeset
2364 allhooks.append(hookargs)
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35265
diff changeset
2365
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35265
diff changeset
2366 for hookargs in allhooks:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2367 op.repo.hook(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2368 b'prepushkey', throw=True, **pycompat.strkwargs(hookargs)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2369 )
35266
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35265
diff changeset
2370
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35265
diff changeset
2371 bookstore.applychanges(op.repo, op.gettransaction(), changes)
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35265
diff changeset
2372
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35265
diff changeset
2373 if pushkeycompat:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2374
43778
888bd39ed555 lock: pass "success" boolean to _afterlock callbacks
Kyle Lippincott <spectral@google.com>
parents: 43506
diff changeset
2375 def runhook(unused_success):
35266
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35265
diff changeset
2376 for hookargs in allhooks:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2377 op.repo.hook(b'pushkey', **pycompat.strkwargs(hookargs))
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2378
35266
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35265
diff changeset
2379 op.repo._afterlock(runhook)
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35265
diff changeset
2380
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2381 elif bookmarksmode == b'records':
35261
f392066d127c bookmark: add pushkey hook compatiblity to the bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35260
diff changeset
2382 for book, node in changes:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2383 record = {b'bookmark': book, b'node': node}
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2384 op.records.add(b'bookmarks', record)
35266
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35265
diff changeset
2385 else:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2386 raise error.ProgrammingError(
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2387 b'unkown bookmark mode: %s' % bookmarksmode
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2388 )
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2389
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2390
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2391 @parthandler(b'phase-heads')
33031
e8c8d81eb864 bundle: add config option to include phases
Martin von Zweigbergk <martinvonz@google.com>
parents: 33030
diff changeset
2392 def handlephases(op, inpart):
e8c8d81eb864 bundle: add config option to include phases
Martin von Zweigbergk <martinvonz@google.com>
parents: 33030
diff changeset
2393 """apply phases from bundle part to repo"""
34320
12c42bcd4133 phases: move the binary decoding function in the phases module
Boris Feld <boris.feld@octobus.net>
parents: 34319
diff changeset
2394 headsbyphase = phases.binarydecode(inpart)
34321
4ef472b975ff bundle2: only grab a transaction when 'phase-heads' affect the repository
Boris Feld <boris.feld@octobus.net>
parents: 34320
diff changeset
2395 phases.updatephases(op.repo.unfiltered(), op.gettransaction, headsbyphase)
33031
e8c8d81eb864 bundle: add config option to include phases
Martin von Zweigbergk <martinvonz@google.com>
parents: 33030
diff changeset
2396
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2397
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2398 @parthandler(b'reply:pushkey', (b'return', b'in-reply-to'))
21660
e87d2a12d41b bundle2: add ``pushkey`` support
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21655
diff changeset
2399 def handlepushkeyreply(op, inpart):
e87d2a12d41b bundle2: add ``pushkey`` support
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21655
diff changeset
2400 """retrieve the result of a pushkey request"""
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2401 ret = int(inpart.params[b'return'])
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2402 partid = int(inpart.params[b'in-reply-to'])
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2403 op.records.add(b'pushkey', {b'return': ret}, partid)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2404
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2405
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2406 @parthandler(b'obsmarkers')
22336
60786c8a2f70 bundle2: add an obsmarkers part handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21900
diff changeset
2407 def handleobsmarker(op, inpart):
60786c8a2f70 bundle2: add an obsmarkers part handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21900
diff changeset
2408 """add a stream of obsmarkers to the repo"""
60786c8a2f70 bundle2: add an obsmarkers part handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21900
diff changeset
2409 tr = op.gettransaction()
24733
c00e4338fa4b obsolete: experimental flag to get debug about obsmarkers exchange
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24686
diff changeset
2410 markerdata = inpart.read()
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2411 if op.ui.config(b'experimental', b'obsmarkers-exchange-debug'):
43080
86e4daa2d54c cleanup: mark some ui.(status|note|warn|write) calls as not needing i18n
Augie Fackler <augie@google.com>
parents: 43077
diff changeset
2412 op.ui.writenoi18n(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2413 b'obsmarker-exchange: %i bytes received\n' % len(markerdata)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2414 )
26685
ea390d889d3a bundle2: gracefully skip 'obsmarkers' part if evolution is disabled
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26587
diff changeset
2415 # The mergemarkers call will crash if marker creation is not enabled.
ea390d889d3a bundle2: gracefully skip 'obsmarkers' part if evolution is disabled
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26587
diff changeset
2416 # we want to avoid this if the part is advisory.
ea390d889d3a bundle2: gracefully skip 'obsmarkers' part if evolution is disabled
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26587
diff changeset
2417 if not inpart.mandatory and op.repo.obsstore.readonly:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2418 op.repo.ui.debug(
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2419 b'ignoring obsolescence markers, feature not enabled\n'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2420 )
26685
ea390d889d3a bundle2: gracefully skip 'obsmarkers' part if evolution is disabled
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26587
diff changeset
2421 return
24733
c00e4338fa4b obsolete: experimental flag to get debug about obsmarkers exchange
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24686
diff changeset
2422 new = op.repo.obsstore.mergemarkers(tr, markerdata)
32326
99515353c72a obsolete: invalidate "volatile" set cache after merging marker
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32024
diff changeset
2423 op.repo.invalidatevolatilesets()
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2424 op.records.add(b'obsmarkers', {b'new': new})
22340
394a17de6a2d obsmarker: produce a reply part for markers received through bundle2
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22338
diff changeset
2425 if op.reply is not None:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2426 rpart = op.reply.newpart(b'reply:obsmarkers')
33675
24b7c75c6441 bundle2: use bytestr() instead of str() to convert part id to bytes
Augie Fackler <augie@google.com>
parents: 33674
diff changeset
2427 rpart.addparam(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2428 b'in-reply-to', pycompat.bytestr(inpart.id), mandatory=False
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2429 )
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2430 rpart.addparam(b'new', b'%i' % new, mandatory=False)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2431
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2432
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2433 @parthandler(b'reply:obsmarkers', (b'new', b'in-reply-to'))
25506
86472038dfd2 bundle2: rename duplicate handlepushkeyreply to handleobsmarkerreply
Martin von Zweigbergk <martinvonz@google.com>
parents: 25493
diff changeset
2434 def handleobsmarkerreply(op, inpart):
22340
394a17de6a2d obsmarker: produce a reply part for markers received through bundle2
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22338
diff changeset
2435 """retrieve the result of a pushkey request"""
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2436 ret = int(inpart.params[b'new'])
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2437 partid = int(inpart.params[b'in-reply-to'])
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2438 op.records.add(b'obsmarkers', {b'new': ret}, partid)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2439
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2440
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2441 @parthandler(b'hgtagsfnodes')
25401
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
2442 def handlehgtagsfnodes(op, inpart):
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
2443 """Applies .hgtags fnodes cache entries to the local repo.
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
2444
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
2445 Payload is pairs of 20 byte changeset nodes and filenodes.
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
2446 """
26565
ee1bcb9aa6e4 bundle2: add op.gettransaction() to handlers that need the lock
Durham Goode <durham@fb.com>
parents: 26542
diff changeset
2447 # Grab the transaction so we ensure that we have the lock at this point.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2448 if op.ui.configbool(b'experimental', b'bundle2lazylocking'):
26565
ee1bcb9aa6e4 bundle2: add op.gettransaction() to handlers that need the lock
Durham Goode <durham@fb.com>
parents: 26542
diff changeset
2449 op.gettransaction()
25401
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
2450 cache = tags.hgtagsfnodescache(op.repo.unfiltered())
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
2451
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
2452 count = 0
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
2453 while True:
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
2454 node = inpart.read(20)
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
2455 fnode = inpart.read(20)
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
2456 if len(node) < 20 or len(fnode) < 20:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2457 op.ui.debug(b'ignoring incomplete received .hgtags fnodes data\n')
25401
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
2458 break
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
2459 cache.setfnode(node, fnode)
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
2460 count += 1
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
2461
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
2462 cache.write()
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2463 op.ui.debug(b'applied %i hgtags fnodes cache entries\n' % count)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2464
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2465
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2466 rbcstruct = struct.Struct(b'>III')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2467
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2468
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2469 @parthandler(b'cache:rev-branch-cache')
36963
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part
Boris Feld <boris.feld@octobus.net>
parents: 36428
diff changeset
2470 def handlerbc(op, inpart):
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part
Boris Feld <boris.feld@octobus.net>
parents: 36428
diff changeset
2471 """receive a rev-branch-cache payload and update the local cache
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part
Boris Feld <boris.feld@octobus.net>
parents: 36428
diff changeset
2472
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part
Boris Feld <boris.feld@octobus.net>
parents: 36428
diff changeset
2473 The payload is a series of data related to each branch
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part
Boris Feld <boris.feld@octobus.net>
parents: 36428
diff changeset
2474
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part
Boris Feld <boris.feld@octobus.net>
parents: 36428
diff changeset
2475 1) branch name length
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part
Boris Feld <boris.feld@octobus.net>
parents: 36428
diff changeset
2476 2) number of open heads
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part
Boris Feld <boris.feld@octobus.net>
parents: 36428
diff changeset
2477 3) number of closed heads
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part
Boris Feld <boris.feld@octobus.net>
parents: 36428
diff changeset
2478 4) open heads nodes
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part
Boris Feld <boris.feld@octobus.net>
parents: 36428
diff changeset
2479 5) closed heads nodes
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part
Boris Feld <boris.feld@octobus.net>
parents: 36428
diff changeset
2480 """
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part
Boris Feld <boris.feld@octobus.net>
parents: 36428
diff changeset
2481 total = 0
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part
Boris Feld <boris.feld@octobus.net>
parents: 36428
diff changeset
2482 rawheader = inpart.read(rbcstruct.size)
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part
Boris Feld <boris.feld@octobus.net>
parents: 36428
diff changeset
2483 cache = op.repo.revbranchcache()
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part
Boris Feld <boris.feld@octobus.net>
parents: 36428
diff changeset
2484 cl = op.repo.unfiltered().changelog
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part
Boris Feld <boris.feld@octobus.net>
parents: 36428
diff changeset
2485 while rawheader:
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part
Boris Feld <boris.feld@octobus.net>
parents: 36428
diff changeset
2486 header = rbcstruct.unpack(rawheader)
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part
Boris Feld <boris.feld@octobus.net>
parents: 36428
diff changeset
2487 total += header[1] + header[2]
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part
Boris Feld <boris.feld@octobus.net>
parents: 36428
diff changeset
2488 utf8branch = inpart.read(header[0])
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part
Boris Feld <boris.feld@octobus.net>
parents: 36428
diff changeset
2489 branch = encoding.tolocal(utf8branch)
38783
e7aa113b14f7 global: use pycompat.xrange()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38701
diff changeset
2490 for x in pycompat.xrange(header[1]):
36963
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part
Boris Feld <boris.feld@octobus.net>
parents: 36428
diff changeset
2491 node = inpart.read(20)
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part
Boris Feld <boris.feld@octobus.net>
parents: 36428
diff changeset
2492 rev = cl.rev(node)
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part
Boris Feld <boris.feld@octobus.net>
parents: 36428
diff changeset
2493 cache.setdata(branch, rev, node, False)
38783
e7aa113b14f7 global: use pycompat.xrange()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38701
diff changeset
2494 for x in pycompat.xrange(header[2]):
36963
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part
Boris Feld <boris.feld@octobus.net>
parents: 36428
diff changeset
2495 node = inpart.read(20)
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part
Boris Feld <boris.feld@octobus.net>
parents: 36428
diff changeset
2496 rev = cl.rev(node)
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part
Boris Feld <boris.feld@octobus.net>
parents: 36428
diff changeset
2497 cache.setdata(branch, rev, node, True)
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part
Boris Feld <boris.feld@octobus.net>
parents: 36428
diff changeset
2498 rawheader = inpart.read(rbcstruct.size)
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part
Boris Feld <boris.feld@octobus.net>
parents: 36428
diff changeset
2499 cache.write()
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part
Boris Feld <boris.feld@octobus.net>
parents: 36428
diff changeset
2500
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2501
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2502 @parthandler(b'pushvars')
33693
db3dc11356ed pushvars: move fb extension pushvars to core
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33682
diff changeset
2503 def bundle2getvars(op, part):
db3dc11356ed pushvars: move fb extension pushvars to core
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33682
diff changeset
2504 '''unbundle a bundle2 containing shellvars on the server'''
db3dc11356ed pushvars: move fb extension pushvars to core
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33682
diff changeset
2505 # An option to disable unbundling on server-side for security reasons
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2506 if op.ui.configbool(b'push', b'pushvars.server'):
33693
db3dc11356ed pushvars: move fb extension pushvars to core
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33682
diff changeset
2507 hookargs = {}
db3dc11356ed pushvars: move fb extension pushvars to core
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33682
diff changeset
2508 for key, value in part.advisoryparams:
db3dc11356ed pushvars: move fb extension pushvars to core
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33682
diff changeset
2509 key = key.upper()
db3dc11356ed pushvars: move fb extension pushvars to core
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33682
diff changeset
2510 # We want pushed variables to have USERVAR_ prepended so we know
db3dc11356ed pushvars: move fb extension pushvars to core
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33682
diff changeset
2511 # they came from the --pushvar flag.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2512 key = b"USERVAR_" + key
33693
db3dc11356ed pushvars: move fb extension pushvars to core
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33682
diff changeset
2513 hookargs[key] = value
db3dc11356ed pushvars: move fb extension pushvars to core
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33682
diff changeset
2514 op.addhookargs(hookargs)
35758
b996ddf5963d bundle2: add a 'stream' part handler for stream cloning
Boris Feld <boris.feld@octobus.net>
parents: 35655
diff changeset
2515
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2516
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2517 @parthandler(b'stream2', (b'requirements', b'filecount', b'bytecount'))
35788
b116a66bcc44 bundle2: move version of stream clone into part name
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35787
diff changeset
2518 def handlestreamv2bundle(op, part):
35758
b996ddf5963d bundle2: add a 'stream' part handler for stream cloning
Boris Feld <boris.feld@octobus.net>
parents: 35655
diff changeset
2519
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2520 requirements = urlreq.unquote(part.params[b'requirements']).split(b',')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2521 filecount = int(part.params[b'filecount'])
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2522 bytecount = int(part.params[b'bytecount'])
35758
b996ddf5963d bundle2: add a 'stream' part handler for stream cloning
Boris Feld <boris.feld@octobus.net>
parents: 35655
diff changeset
2523
b996ddf5963d bundle2: add a 'stream' part handler for stream cloning
Boris Feld <boris.feld@octobus.net>
parents: 35655
diff changeset
2524 repo = op.repo
b996ddf5963d bundle2: add a 'stream' part handler for stream cloning
Boris Feld <boris.feld@octobus.net>
parents: 35655
diff changeset
2525 if len(repo):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2526 msg = _(b'cannot apply stream clone to non empty repository')
35758
b996ddf5963d bundle2: add a 'stream' part handler for stream cloning
Boris Feld <boris.feld@octobus.net>
parents: 35655
diff changeset
2527 raise error.Abort(msg)
b996ddf5963d bundle2: add a 'stream' part handler for stream cloning
Boris Feld <boris.feld@octobus.net>
parents: 35655
diff changeset
2528
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2529 repo.ui.debug(b'applying stream bundle\n')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2530 streamclone.applybundlev2(repo, part, filecount, bytecount, requirements)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2531
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2532
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2533 def widen_bundle(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2534 bundler, repo, oldmatcher, newmatcher, common, known, cgversion, ellipses
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2535 ):
40072
1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39850
diff changeset
2536 """generates bundle2 for widening a narrow clone
1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39850
diff changeset
2537
42417
a97b12f726e4 narrow: pass the bundle to bundle2.widen_bundle() instead of generating there
Pulkit Goyal <7895pulkit@gmail.com>
parents: 42143
diff changeset
2538 bundler is the bundle to which data should be added
40072
1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39850
diff changeset
2539 repo is the localrepository instance
40344
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40338
diff changeset
2540 oldmatcher matches what the client already has
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40338
diff changeset
2541 newmatcher matches what the client needs (including what it already has)
40072
1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39850
diff changeset
2542 common is set of common heads between server and client
1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39850
diff changeset
2543 known is a set of revs known on the client side (used in ellipses)
1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39850
diff changeset
2544 cgversion is the changegroup version to send
1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39850
diff changeset
2545 ellipses is boolean value telling whether to send ellipses data or not
1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39850
diff changeset
2546
1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39850
diff changeset
2547 returns bundle2 of the data required for extending
1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39850
diff changeset
2548 """
1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39850
diff changeset
2549 commonnodes = set()
1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39850
diff changeset
2550 cl = repo.changelog
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2551 for r in repo.revs(b"::%ln", common):
40072
1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39850
diff changeset
2552 commonnodes.add(cl.node(r))
1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39850
diff changeset
2553 if commonnodes:
1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39850
diff changeset
2554 # XXX: we should only send the filelogs (and treemanifest). user
1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39850
diff changeset
2555 # already has the changelog and manifest
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2556 packer = changegroup.getbundler(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2557 cgversion,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2558 repo,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2559 oldmatcher=oldmatcher,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2560 matcher=newmatcher,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2561 fullnodes=commonnodes,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2562 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2563 cgdata = packer.generate(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2564 {nodemod.nullid},
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2565 list(commonnodes),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2566 False,
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2567 b'narrow_widen',
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2568 changelog=False,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2569 )
40072
1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39850
diff changeset
2570
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2571 part = bundler.newpart(b'changegroup', data=cgdata)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2572 part.addparam(b'version', cgversion)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2573 if b'treemanifest' in repo.requirements:
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2574 part.addparam(b'treemanifest', b'1')
43131
c17a63eb5d4c sidedata: apply basic but tight security around exchange
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43117
diff changeset
2575 if b'exp-sidedata-flag' in repo.requirements:
c17a63eb5d4c sidedata: apply basic but tight security around exchange
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43117
diff changeset
2576 part.addparam(b'exp-sidedata', b'1')
40072
1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39850
diff changeset
2577
1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39850
diff changeset
2578 return bundler