annotate mercurial/bundle2.py @ 50896:b2b8c25f9462

hgwebmod: use sysstr to check for attribute presence We do not need bytes here.
author Pierre-Yves David <pierre-yves.david@octobus.net>
date Wed, 30 Aug 2023 13:28:09 +0200
parents 0e936b950731
children d718eddf01d9
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
20801
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
1 # bundle2.py - generic container format to transmit arbitrary data.
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
2 #
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
3 # Copyright 2013 Facebook, Inc.
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
4 #
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
5 # This software may be used and distributed according to the terms of the
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
6 # GNU General Public License version 2 or any later version.
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
7 """Handling of the new bundle2 format
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
8
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
9 The goal of bundle2 is to act as an atomically packet to transmit a set of
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
10 payloads in an application agnostic way. It consist in a sequence of "parts"
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
11 that will be handed to and processed by the application layer.
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
12
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
13
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
14 General format architecture
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
15 ===========================
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
16
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
17 The format is architectured as follow
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
18
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
19 - magic string
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
20 - stream level parameters
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
21 - payload parts (any number)
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
22 - end of stream marker.
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
23
20856
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
24 the Binary format
20801
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
25 ============================
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
26
21024
7731a2281cf0 spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents: 21020
diff changeset
27 All numbers are unsigned and big-endian.
20801
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
28
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
29 stream level parameters
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
30 ------------------------
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
31
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
32 Binary format is as follow
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
33
23009
90f86ad3d4ff bundle2: change header size and make them signed (new format)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23008
diff changeset
34 :params size: int32
20801
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
35
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
36 The total number of Bytes used by the parameters
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
37
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
38 :params value: arbitrary number of Bytes
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
39
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
40 A blob of `params size` containing the serialized version of all stream level
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
41 parameters.
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
42
21024
7731a2281cf0 spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents: 21020
diff changeset
43 The blob contains a space separated list of parameters. Parameters with value
20811
9785c3f8f598 bundle2: urlquote stream parameter name and value
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20810
diff changeset
44 are stored in the form `<name>=<value>`. Both name and value are urlquoted.
20804
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
45
20813
8c74b3ce5b70 bundle2: refuse empty parameter name
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20812
diff changeset
46 Empty name are obviously forbidden.
8c74b3ce5b70 bundle2: refuse empty parameter name
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20812
diff changeset
47
20844
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
48 Name MUST start with a letter. If this first letter is lower case, the
21024
7731a2281cf0 spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents: 21020
diff changeset
49 parameter is advisory and can be safely ignored. However when the first
20844
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
50 letter is capital, the parameter is mandatory and the bundling process MUST
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
51 stop if he is not able to proceed it.
20814
8532f5e1b9df bundle2: force the first char of parameter to be an letter.
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20813
diff changeset
52
20808
4c9130c7a29f bundle2: clarify stream parameter design in the documentation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20805
diff changeset
53 Stream parameters use a simple textual format for two main reasons:
20804
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
54
21024
7731a2281cf0 spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents: 21020
diff changeset
55 - Stream level parameters should remain simple and we want to discourage any
20808
4c9130c7a29f bundle2: clarify stream parameter design in the documentation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20805
diff changeset
56 crazy usage.
21024
7731a2281cf0 spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents: 21020
diff changeset
57 - Textual data allow easy human inspection of a bundle2 header in case of
20808
4c9130c7a29f bundle2: clarify stream parameter design in the documentation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20805
diff changeset
58 troubles.
4c9130c7a29f bundle2: clarify stream parameter design in the documentation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20805
diff changeset
59
4c9130c7a29f bundle2: clarify stream parameter design in the documentation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20805
diff changeset
60 Any Applicative level options MUST go into a bundle2 part instead.
20801
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
61
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
62 Payload part
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
63 ------------------------
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
64
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
65 Binary format is as follow
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
66
23009
90f86ad3d4ff bundle2: change header size and make them signed (new format)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23008
diff changeset
67 :header size: int32
20801
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
68
25507
5bee483705e8 bundle2: clarify in docstring that header size is for a single header
Martin von Zweigbergk <martinvonz@google.com>
parents: 25506
diff changeset
69 The total number of Bytes used by the part header. When the header is empty
20801
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
70 (size = 0) this is interpreted as the end of stream marker.
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
71
20856
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
72 :header:
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
73
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
74 The header defines how to interpret the part. It contains two piece of
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
75 data: the part type, and the part parameters.
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
76
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
77 The part type is used to route an application level handler, that can
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
78 interpret payload.
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
79
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
80 Part parameters are passed to the application level handler. They are
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
81 meant to convey information that will help the application level object to
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
82 interpret the part payload.
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
83
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
84 The binary format of the header is has follow
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
85
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
86 :typesize: (one byte)
20877
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
87
23916
a3f7c781786b bundle2: fix parttype enforcement
Matt Mackall <mpm@selenic.com>
parents: 23868
diff changeset
88 :parttype: alphanumerical part name (restricted to [a-zA-Z0-9_:-]*)
20877
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
89
20995
e995d104c87f bundle2: add an integer id to part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20952
diff changeset
90 :partid: A 32bits integer (unique in the bundle) that can be used to refer
e995d104c87f bundle2: add an integer id to part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20952
diff changeset
91 to this part.
e995d104c87f bundle2: add an integer id to part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20952
diff changeset
92
20877
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
93 :parameters:
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
94
21024
7731a2281cf0 spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents: 21020
diff changeset
95 Part's parameter may have arbitrary content, the binary structure is::
20877
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
96
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
97 <mandatory-count><advisory-count><param-sizes><param-data>
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
98
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
99 :mandatory-count: 1 byte, number of mandatory parameters
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
100
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
101 :advisory-count: 1 byte, number of advisory parameters
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
102
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
103 :param-sizes:
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
104
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
105 N couple of bytes, where N is the total number of parameters. Each
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
106 couple contains (<size-of-key>, <size-of-value) for one parameter.
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
107
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
108 :param-data:
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
109
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
110 A blob of bytes from which each parameter key and value can be
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
111 retrieved using the list of size couples stored in the previous
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
112 field.
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
113
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
114 Mandatory parameters comes first, then the advisory ones.
20856
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
115
21607
054fa5176fa7 bundle2: forbid duplicate parameter keys
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21606
diff changeset
116 Each parameter's key MUST be unique within the part.
054fa5176fa7 bundle2: forbid duplicate parameter keys
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21606
diff changeset
117
20856
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
118 :payload:
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
119
20876
ddd56f3eb786 bundle2: support for bundling and unbundling payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20864
diff changeset
120 payload is a series of `<chunksize><chunkdata>`.
ddd56f3eb786 bundle2: support for bundling and unbundling payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20864
diff changeset
121
23009
90f86ad3d4ff bundle2: change header size and make them signed (new format)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23008
diff changeset
122 `chunksize` is an int32, `chunkdata` are plain bytes (as much as
20876
ddd56f3eb786 bundle2: support for bundling and unbundling payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20864
diff changeset
123 `chunksize` says)` The payload part is concluded by a zero size chunk.
ddd56f3eb786 bundle2: support for bundling and unbundling payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20864
diff changeset
124
ddd56f3eb786 bundle2: support for bundling and unbundling payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20864
diff changeset
125 The current implementation always produces either zero or one chunk.
21024
7731a2281cf0 spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents: 21020
diff changeset
126 This is an implementation limitation that will ultimately be lifted.
20891
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20890
diff changeset
127
23009
90f86ad3d4ff bundle2: change header size and make them signed (new format)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23008
diff changeset
128 `chunksize` can be negative to trigger special case processing. No such
90f86ad3d4ff bundle2: change header size and make them signed (new format)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23008
diff changeset
129 processing is in place yet.
90f86ad3d4ff bundle2: change header size and make them signed (new format)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23008
diff changeset
130
20891
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20890
diff changeset
131 Bundle processing
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20890
diff changeset
132 ============================
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20890
diff changeset
133
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20890
diff changeset
134 Each part is processed in order using a "part handler". Handler are registered
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20890
diff changeset
135 for a certain part type.
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20890
diff changeset
136
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20890
diff changeset
137 The matching of a part to its handler is case insensitive. The case of the
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20890
diff changeset
138 part type is used to know if a part is mandatory or advisory. If the Part type
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20890
diff changeset
139 contains any uppercase char it is considered mandatory. When no handler is
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20890
diff changeset
140 known for a Mandatory part, the process is aborted and an exception is raised.
20892
6fe95448596d bundle2: read the whole bundle from stream on abort
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20891
diff changeset
141 If the part is advisory and no handler is known, the part is ignored. When the
6fe95448596d bundle2: read the whole bundle from stream on abort
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20891
diff changeset
142 process is aborted, the full bundle is still read from the stream to keep the
6fe95448596d bundle2: read the whole bundle from stream on abort
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20891
diff changeset
143 channel usable. But none of the part read from an abort are processed. In the
6fe95448596d bundle2: read the whole bundle from stream on abort
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20891
diff changeset
144 future, dropping the stream may become an option for channel we do not care to
6fe95448596d bundle2: read the whole bundle from stream on abort
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20891
diff changeset
145 preserve.
20801
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
146 """
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
147
25919
8221fefaea08 bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
148
36964
79b73be4dda5 rev-branch-cache: add a function to generate a part
Boris Feld <boris.feld@octobus.net>
parents: 36963
diff changeset
149 import collections
24026
3daef83a1873 bundle2.unpackermixin: control for underlying file descriptor
Eric Sumner <ericsumner@fb.com>
parents: 23916
diff changeset
150 import errno
35046
241d9caca11e bundle2: use os.SEEK_* constants
Gregory Szorc <gregory.szorc@gmail.com>
parents: 34820
diff changeset
151 import os
25919
8221fefaea08 bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
152 import re
20814
8532f5e1b9df bundle2: force the first char of parameter to be an letter.
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20813
diff changeset
153 import string
25919
8221fefaea08 bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
154 import struct
8221fefaea08 bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
155 import sys
20804
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
156
25919
8221fefaea08 bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
157 from .i18n import _
46113
59fa3890d40a node: import symbols explicitly
Joerg Sonnenberger <joerg@bec.de>
parents: 45942
diff changeset
158 from .node import (
59fa3890d40a node: import symbols explicitly
Joerg Sonnenberger <joerg@bec.de>
parents: 45942
diff changeset
159 hex,
59fa3890d40a node: import symbols explicitly
Joerg Sonnenberger <joerg@bec.de>
parents: 45942
diff changeset
160 short,
59fa3890d40a node: import symbols explicitly
Joerg Sonnenberger <joerg@bec.de>
parents: 45942
diff changeset
161 )
25919
8221fefaea08 bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
162 from . import (
35258
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35120
diff changeset
163 bookmarks,
25919
8221fefaea08 bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
164 changegroup,
36963
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part
Boris Feld <boris.feld@octobus.net>
parents: 36428
diff changeset
165 encoding,
25919
8221fefaea08 bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
166 error,
8221fefaea08 bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
167 obsolete,
33031
e8c8d81eb864 bundle: add config option to include phases
Martin von Zweigbergk <martinvonz@google.com>
parents: 33030
diff changeset
168 phases,
25919
8221fefaea08 bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
169 pushkey,
30030
0f6d6fdd3c2a pycompat: provide 'ispy3' constant
Yuya Nishihara <yuya@tcha.org>
parents: 29937
diff changeset
170 pycompat,
45372
77b8588dd84e requirements: introduce new requirements related module
Pulkit Goyal <7895pulkit@gmail.com>
parents: 45371
diff changeset
171 requirements,
45106
a03c177a4679 scmutil: add writereporequirements() and route requires writing through it
Pulkit Goyal <7895pulkit@gmail.com>
parents: 44372
diff changeset
172 scmutil,
35758
b996ddf5963d bundle2: add a 'stream' part handler for stream cloning
Boris Feld <boris.feld@octobus.net>
parents: 35655
diff changeset
173 streamclone,
25919
8221fefaea08 bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
174 tags,
8221fefaea08 bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
175 url,
8221fefaea08 bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
176 util,
8221fefaea08 bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
177 )
46907
ffd3e823a7e5 urlutil: extract `url` related code from `util` into the new module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46780
diff changeset
178 from .utils import (
ffd3e823a7e5 urlutil: extract `url` related code from `util` into the new module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46780
diff changeset
179 stringutil,
ffd3e823a7e5 urlutil: extract `url` related code from `util` into the new module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46780
diff changeset
180 urlutil,
ffd3e823a7e5 urlutil: extract `url` related code from `util` into the new module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46780
diff changeset
181 )
47226
19d4802cb304 sidedata: add a 'side-data' repository feature and use it
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47128
diff changeset
182 from .interfaces import repository
20802
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
183
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28672
diff changeset
184 urlerr = util.urlerr
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28672
diff changeset
185 urlreq = util.urlreq
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28672
diff changeset
186
20804
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
187 _pack = struct.pack
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
188 _unpack = struct.unpack
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
189
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
190 _fstreamparamsize = b'>i'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
191 _fpartheadersize = b'>i'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
192 _fparttypesize = b'>B'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
193 _fpartid = b'>I'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
194 _fpayloadsize = b'>i'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
195 _fpartparamcount = b'>BB'
20877
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
196
35793
4fb2bb61597c bundle2: increase payload part chunk size to 32kb
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35792
diff changeset
197 preferedchunksize = 32768
21001
c93bb6a08fa1 bundle2: support chunk iterator as part data
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21000
diff changeset
198
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
199 _parttypeforbidden = re.compile(b'[^a-zA-Z0-9_:-]')
23868
405eacbecc52 bundle2: enforce parttype as alphanumerical
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23590
diff changeset
200
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
201
25313
8f2c362bcb58 bundle2: introduce a specific function for bundling debug message
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25181
diff changeset
202 def outdebug(ui, message):
8f2c362bcb58 bundle2: introduce a specific function for bundling debug message
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25181
diff changeset
203 """debug regarding output stream (bundling)"""
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
204 if ui.configbool(b'devel', b'bundle2.debug'):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
205 ui.debug(b'bundle2-output: %s\n' % message)
25313
8f2c362bcb58 bundle2: introduce a specific function for bundling debug message
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25181
diff changeset
206
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
207
25318
64dd8f25fcf4 bundle2: introduce a specific function for debug messages while unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25317
diff changeset
208 def indebug(ui, message):
64dd8f25fcf4 bundle2: introduce a specific function for debug messages while unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25317
diff changeset
209 """debug on input stream (unbundling)"""
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
210 if ui.configbool(b'devel', b'bundle2.debug'):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
211 ui.debug(b'bundle2-input: %s\n' % message)
25318
64dd8f25fcf4 bundle2: introduce a specific function for debug messages while unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25317
diff changeset
212
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
213
23868
405eacbecc52 bundle2: enforce parttype as alphanumerical
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23590
diff changeset
214 def validateparttype(parttype):
405eacbecc52 bundle2: enforce parttype as alphanumerical
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23590
diff changeset
215 """raise ValueError if a parttype contains invalid character"""
23916
a3f7c781786b bundle2: fix parttype enforcement
Matt Mackall <mpm@selenic.com>
parents: 23868
diff changeset
216 if _parttypeforbidden.search(parttype):
23868
405eacbecc52 bundle2: enforce parttype as alphanumerical
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23590
diff changeset
217 raise ValueError(parttype)
405eacbecc52 bundle2: enforce parttype as alphanumerical
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23590
diff changeset
218
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
219
20877
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
220 def _makefpartparamsizes(nbparams):
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
221 """return a struct format to read part parameter sizes
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
222
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
223 The number parameters is variable so we need to build that format
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
224 dynamically.
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
225 """
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
226 return b'>' + (b'BB' * nbparams)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
227
20804
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
228
20890
ec7fc110faee bundle2: introduce a `parthandler` decorator
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20889
diff changeset
229 parthandlermapping = {}
20889
deed5edb72de bundle2: first version of a bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20887
diff changeset
230
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
231
21623
5b26d82e4e2a bundle2: make it possible to declare params handled by a part handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21622
diff changeset
232 def parthandler(parttype, params=()):
20890
ec7fc110faee bundle2: introduce a `parthandler` decorator
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20889
diff changeset
233 """decorator that register a function as a bundle2 part handler
ec7fc110faee bundle2: introduce a `parthandler` decorator
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20889
diff changeset
234
ec7fc110faee bundle2: introduce a `parthandler` decorator
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20889
diff changeset
235 eg::
ec7fc110faee bundle2: introduce a `parthandler` decorator
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20889
diff changeset
236
21624
d61066d787c8 bundle2: declare supported parameters for all handlers
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21623
diff changeset
237 @parthandler('myparttype', ('mandatory', 'param', 'handled'))
20890
ec7fc110faee bundle2: introduce a `parthandler` decorator
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20889
diff changeset
238 def myparttypehandler(...):
ec7fc110faee bundle2: introduce a `parthandler` decorator
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20889
diff changeset
239 '''process a part of type "my part".'''
ec7fc110faee bundle2: introduce a `parthandler` decorator
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20889
diff changeset
240 ...
ec7fc110faee bundle2: introduce a `parthandler` decorator
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20889
diff changeset
241 """
23868
405eacbecc52 bundle2: enforce parttype as alphanumerical
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23590
diff changeset
242 validateparttype(parttype)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
243
20890
ec7fc110faee bundle2: introduce a `parthandler` decorator
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20889
diff changeset
244 def _decorator(func):
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
245 lparttype = parttype.lower() # enforce lower case matching.
20891
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20890
diff changeset
246 assert lparttype not in parthandlermapping
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20890
diff changeset
247 parthandlermapping[lparttype] = func
21623
5b26d82e4e2a bundle2: make it possible to declare params handled by a part handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21622
diff changeset
248 func.params = frozenset(params)
20890
ec7fc110faee bundle2: introduce a `parthandler` decorator
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20889
diff changeset
249 return func
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
250
20890
ec7fc110faee bundle2: introduce a `parthandler` decorator
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20889
diff changeset
251 return _decorator
20889
deed5edb72de bundle2: first version of a bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20887
diff changeset
252
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
253
48946
642e31cb55f0 py3: use class X: instead of class X(object):
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48913
diff changeset
254 class unbundlerecords:
20949
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
255 """keep record of what happens during and unbundle
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
256
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
257 New records are added using `records.add('cat', obj)`. Where 'cat' is a
21024
7731a2281cf0 spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents: 21020
diff changeset
258 category of record and obj is an arbitrary object.
20949
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
259
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
260 `records['cat']` will return all entries of this category 'cat'.
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
261
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
262 Iterating on the object itself will yield `('category', obj)` tuples
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
263 for all entries.
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
264
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
265 All iterations happens in chronological order.
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
266 """
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
267
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
268 def __init__(self):
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
269 self._categories = {}
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
270 self._sequences = []
20996
ed3c5e18a047 bundle2: add reply awareness to unbundlerecords
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20995
diff changeset
271 self._replies = {}
20949
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
272
20996
ed3c5e18a047 bundle2: add reply awareness to unbundlerecords
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20995
diff changeset
273 def add(self, category, entry, inreplyto=None):
20949
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
274 """add a new record of a given category.
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
275
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
276 The entry can then be retrieved in the list returned by
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
277 self['category']."""
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
278 self._categories.setdefault(category, []).append(entry)
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
279 self._sequences.append((category, entry))
20996
ed3c5e18a047 bundle2: add reply awareness to unbundlerecords
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20995
diff changeset
280 if inreplyto is not None:
ed3c5e18a047 bundle2: add reply awareness to unbundlerecords
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20995
diff changeset
281 self.getreplies(inreplyto).add(category, entry)
ed3c5e18a047 bundle2: add reply awareness to unbundlerecords
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20995
diff changeset
282
ed3c5e18a047 bundle2: add reply awareness to unbundlerecords
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20995
diff changeset
283 def getreplies(self, partid):
23139
e53f6b72a0e4 spelling: fixes from proofreading of spell checker issues
Mads Kiilerich <madski@unity3d.com>
parents: 23067
diff changeset
284 """get the records that are replies to a specific part"""
20996
ed3c5e18a047 bundle2: add reply awareness to unbundlerecords
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20995
diff changeset
285 return self._replies.setdefault(partid, unbundlerecords())
20949
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
286
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
287 def __getitem__(self, cat):
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
288 return tuple(self._categories.get(cat, ()))
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
289
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
290 def __iter__(self):
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
291 return iter(self._sequences)
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
292
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
293 def __len__(self):
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
294 return len(self._sequences)
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
295
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
296 def __nonzero__(self):
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
297 return bool(self._sequences)
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
298
31476
413b44003462 py3: add __bool__ to every class defining __nonzero__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30965
diff changeset
299 __bool__ = __nonzero__
413b44003462 py3: add __bool__ to every class defining __nonzero__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30965
diff changeset
300
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
301
48946
642e31cb55f0 py3: use class X: instead of class X(object):
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48913
diff changeset
302 class bundleoperation:
20948
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
303 """an object that represents a single bundling process
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
304
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
305 Its purpose is to carry unbundle-related objects and states.
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
306
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
307 A new object should be created at the beginning of each bundle processing.
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
308 The object is to be returned by the processing function.
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
309
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
310 The object has very little content now it will ultimately contain:
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
311 * an access to the repo the bundle is applied to,
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
312 * a ui object,
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
313 * a way to retrieve a transaction to add changes to the repo,
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
314 * a way to record the result of processing each part,
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
315 * a way to construct a bundle response when applicable.
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
316 """
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
317
49765
4188e75af983 bundleoperation: optionnaly record the `remote` that produced the bundle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49648
diff changeset
318 def __init__(
4188e75af983 bundleoperation: optionnaly record the `remote` that produced the bundle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49648
diff changeset
319 self,
4188e75af983 bundleoperation: optionnaly record the `remote` that produced the bundle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49648
diff changeset
320 repo,
4188e75af983 bundleoperation: optionnaly record the `remote` that produced the bundle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49648
diff changeset
321 transactiongetter,
4188e75af983 bundleoperation: optionnaly record the `remote` that produced the bundle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49648
diff changeset
322 captureoutput=True,
4188e75af983 bundleoperation: optionnaly record the `remote` that produced the bundle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49648
diff changeset
323 source=b'',
4188e75af983 bundleoperation: optionnaly record the `remote` that produced the bundle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49648
diff changeset
324 remote=None,
4188e75af983 bundleoperation: optionnaly record the `remote` that produced the bundle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49648
diff changeset
325 ):
20948
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
326 self.repo = repo
49765
4188e75af983 bundleoperation: optionnaly record the `remote` that produced the bundle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49648
diff changeset
327 # the peer object who produced this bundle if available
4188e75af983 bundleoperation: optionnaly record the `remote` that produced the bundle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49648
diff changeset
328 self.remote = remote
20948
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
329 self.ui = repo.ui
20949
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
330 self.records = unbundlerecords()
20997
d7df4b7378ae bundle2: produce a bundle2 reply
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20996
diff changeset
331 self.reply = None
24878
e530cde6d115 bundle2: disable ouput capture unless we use http (issue4613 issue4615)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24851
diff changeset
332 self.captureoutput = captureoutput
33664
f3407d56a6e8 bundle2: add the capability to store hookargs on bundle operation object
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33541
diff changeset
333 self.hookargs = {}
33665
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33664
diff changeset
334 self._gettransaction = transactiongetter
35265
1f30cbac34b6 bundle2: add a 'modes' dictionary to the bundle operation
Boris Feld <boris.feld@octobus.net>
parents: 35261
diff changeset
335 # carries value that can modify part behavior
1f30cbac34b6 bundle2: add a 'modes' dictionary to the bundle operation
Boris Feld <boris.feld@octobus.net>
parents: 35261
diff changeset
336 self.modes = {}
37235
7e906d8a825f bundle2: add 'source' atrribute to bundleoperation class
Pulkit Goyal <7895pulkit@gmail.com>
parents: 37167
diff changeset
337 self.source = source
33665
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33664
diff changeset
338
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33664
diff changeset
339 def gettransaction(self):
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33664
diff changeset
340 transaction = self._gettransaction()
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33664
diff changeset
341
33810
b3a41f4d837b bundle2: relax the condition to update transaction.hookargs
Yuya Nishihara <yuya@tcha.org>
parents: 33809
diff changeset
342 if self.hookargs:
33665
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33664
diff changeset
343 # the ones added to the transaction supercede those added
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33664
diff changeset
344 # to the operation.
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33664
diff changeset
345 self.hookargs.update(transaction.hookargs)
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33664
diff changeset
346 transaction.hookargs = self.hookargs
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33664
diff changeset
347
33810
b3a41f4d837b bundle2: relax the condition to update transaction.hookargs
Yuya Nishihara <yuya@tcha.org>
parents: 33809
diff changeset
348 # mark the hookargs as flushed. further attempts to add to
b3a41f4d837b bundle2: relax the condition to update transaction.hookargs
Yuya Nishihara <yuya@tcha.org>
parents: 33809
diff changeset
349 # hookargs will result in an abort.
b3a41f4d837b bundle2: relax the condition to update transaction.hookargs
Yuya Nishihara <yuya@tcha.org>
parents: 33809
diff changeset
350 self.hookargs = None
33665
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33664
diff changeset
351
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33664
diff changeset
352 return transaction
33664
f3407d56a6e8 bundle2: add the capability to store hookargs on bundle operation object
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33541
diff changeset
353
f3407d56a6e8 bundle2: add the capability to store hookargs on bundle operation object
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33541
diff changeset
354 def addhookargs(self, hookargs):
33665
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33664
diff changeset
355 if self.hookargs is None:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
356 raise error.ProgrammingError(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
357 b'attempted to add hookargs to '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
358 b'operation after transaction started'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
359 )
33664
f3407d56a6e8 bundle2: add the capability to store hookargs on bundle operation object
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33541
diff changeset
360 self.hookargs.update(hookargs)
20948
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
361
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
362
20952
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20950
diff changeset
363 class TransactionUnavailable(RuntimeError):
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20950
diff changeset
364 pass
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20950
diff changeset
365
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
366
20952
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20950
diff changeset
367 def _notransaction():
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20950
diff changeset
368 """default method to get a transaction while processing a bundle
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20950
diff changeset
369
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20950
diff changeset
370 Raise an exception to highlight the fact that no transaction was expected
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20950
diff changeset
371 to be created"""
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20950
diff changeset
372 raise TransactionUnavailable()
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20950
diff changeset
373
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
374
49765
4188e75af983 bundleoperation: optionnaly record the `remote` that produced the bundle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49648
diff changeset
375 def applybundle(repo, unbundler, tr, source, url=None, remote=None, **kwargs):
26790
28a6c2d72097 bundle2: introduce an "applybundle" function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26698
diff changeset
376 # transform me into unbundler.apply() as soon as the freeze is lifted
33043
18c2489ac96d bundle: make applybundle() delegate v1 bundles to applybundle1()
Martin von Zweigbergk <martinvonz@google.com>
parents: 33040
diff changeset
377 if isinstance(unbundler, unbundle20):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
378 tr.hookargs[b'bundle2'] = b'1'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
379 if source is not None and b'source' not in tr.hookargs:
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
380 tr.hookargs[b'source'] = source
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
381 if url is not None and b'url' not in tr.hookargs:
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
382 tr.hookargs[b'url'] = url
49765
4188e75af983 bundleoperation: optionnaly record the `remote` that produced the bundle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49648
diff changeset
383 return processbundle(
4188e75af983 bundleoperation: optionnaly record the `remote` that produced the bundle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49648
diff changeset
384 repo, unbundler, lambda: tr, source=source, remote=remote
4188e75af983 bundleoperation: optionnaly record the `remote` that produced the bundle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49648
diff changeset
385 )
33043
18c2489ac96d bundle: make applybundle() delegate v1 bundles to applybundle1()
Martin von Zweigbergk <martinvonz@google.com>
parents: 33040
diff changeset
386 else:
33044
1d2b68952950 bundle: inline applybundle1()
Martin von Zweigbergk <martinvonz@google.com>
parents: 33043
diff changeset
387 # the transactiongetter won't be used, but we might as well set it
49765
4188e75af983 bundleoperation: optionnaly record the `remote` that produced the bundle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49648
diff changeset
388 op = bundleoperation(repo, lambda: tr, source=source, remote=remote)
33044
1d2b68952950 bundle: inline applybundle1()
Martin von Zweigbergk <martinvonz@google.com>
parents: 33043
diff changeset
389 _processchangegroup(op, unbundler, tr, source, url, **kwargs)
1d2b68952950 bundle: inline applybundle1()
Martin von Zweigbergk <martinvonz@google.com>
parents: 33043
diff changeset
390 return op
26790
28a6c2d72097 bundle2: introduce an "applybundle" function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26698
diff changeset
391
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
392
48946
642e31cb55f0 py3: use class X: instead of class X(object):
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48913
diff changeset
393 class partiterator:
34152
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
394 def __init__(self, repo, op, unbundler):
34151
550343626bb2 bundle2: move part counter to partiterator
Durham Goode <durham@fb.com>
parents: 34150
diff changeset
395 self.repo = repo
34152
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
396 self.op = op
34150
e9e0e1143fc5 bundle2: move part iterator a separate class
Durham Goode <durham@fb.com>
parents: 34101
diff changeset
397 self.unbundler = unbundler
34151
550343626bb2 bundle2: move part counter to partiterator
Durham Goode <durham@fb.com>
parents: 34150
diff changeset
398 self.iterator = None
550343626bb2 bundle2: move part counter to partiterator
Durham Goode <durham@fb.com>
parents: 34150
diff changeset
399 self.count = 0
34258
e71890f27767 bundle2: move processpart stream maintenance into part iterator
Durham Goode <durham@fb.com>
parents: 34249
diff changeset
400 self.current = None
34150
e9e0e1143fc5 bundle2: move part iterator a separate class
Durham Goode <durham@fb.com>
parents: 34101
diff changeset
401
e9e0e1143fc5 bundle2: move part iterator a separate class
Durham Goode <durham@fb.com>
parents: 34101
diff changeset
402 def __enter__(self):
34151
550343626bb2 bundle2: move part counter to partiterator
Durham Goode <durham@fb.com>
parents: 34150
diff changeset
403 def func():
42931
181ee2118a96 bundle2: fix an off-by-one in debug message of number of parts
Martin von Zweigbergk <martinvonz@google.com>
parents: 42894
diff changeset
404 itr = enumerate(self.unbundler.iterparts(), 1)
34151
550343626bb2 bundle2: move part counter to partiterator
Durham Goode <durham@fb.com>
parents: 34150
diff changeset
405 for count, p in itr:
550343626bb2 bundle2: move part counter to partiterator
Durham Goode <durham@fb.com>
parents: 34150
diff changeset
406 self.count = count
34258
e71890f27767 bundle2: move processpart stream maintenance into part iterator
Durham Goode <durham@fb.com>
parents: 34249
diff changeset
407 self.current = p
34151
550343626bb2 bundle2: move part counter to partiterator
Durham Goode <durham@fb.com>
parents: 34150
diff changeset
408 yield p
35114
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35113
diff changeset
409 p.consume()
34258
e71890f27767 bundle2: move processpart stream maintenance into part iterator
Durham Goode <durham@fb.com>
parents: 34249
diff changeset
410 self.current = None
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
411
34151
550343626bb2 bundle2: move part counter to partiterator
Durham Goode <durham@fb.com>
parents: 34150
diff changeset
412 self.iterator = func()
550343626bb2 bundle2: move part counter to partiterator
Durham Goode <durham@fb.com>
parents: 34150
diff changeset
413 return self.iterator
34150
e9e0e1143fc5 bundle2: move part iterator a separate class
Durham Goode <durham@fb.com>
parents: 34101
diff changeset
414
34152
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
415 def __exit__(self, type, exc, tb):
34151
550343626bb2 bundle2: move part counter to partiterator
Durham Goode <durham@fb.com>
parents: 34150
diff changeset
416 if not self.iterator:
550343626bb2 bundle2: move part counter to partiterator
Durham Goode <durham@fb.com>
parents: 34150
diff changeset
417 return
550343626bb2 bundle2: move part counter to partiterator
Durham Goode <durham@fb.com>
parents: 34150
diff changeset
418
34637
5f79f5f8487a bundle2: immediate exit for ctrl+c (issue5692)
Durham Goode <durham@fb.com>
parents: 34323
diff changeset
419 # Only gracefully abort in a normal exception situation. User aborts
5f79f5f8487a bundle2: immediate exit for ctrl+c (issue5692)
Durham Goode <durham@fb.com>
parents: 34323
diff changeset
420 # like Ctrl+C throw a KeyboardInterrupt which is not a base Exception,
5f79f5f8487a bundle2: immediate exit for ctrl+c (issue5692)
Durham Goode <durham@fb.com>
parents: 34323
diff changeset
421 # and should not gracefully cleanup.
5f79f5f8487a bundle2: immediate exit for ctrl+c (issue5692)
Durham Goode <durham@fb.com>
parents: 34323
diff changeset
422 if isinstance(exc, Exception):
34152
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
423 # Any exceptions seeking to the end of the bundle at this point are
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
424 # almost certainly related to the underlying stream being bad.
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
425 # And, chances are that the exception we're handling is related to
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
426 # getting in that bad state. So, we swallow the seeking error and
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
427 # re-raise the original error.
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
428 seekerror = False
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
429 try:
34637
5f79f5f8487a bundle2: immediate exit for ctrl+c (issue5692)
Durham Goode <durham@fb.com>
parents: 34323
diff changeset
430 if self.current:
5f79f5f8487a bundle2: immediate exit for ctrl+c (issue5692)
Durham Goode <durham@fb.com>
parents: 34323
diff changeset
431 # consume the part content to not corrupt the stream.
35114
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35113
diff changeset
432 self.current.consume()
34637
5f79f5f8487a bundle2: immediate exit for ctrl+c (issue5692)
Durham Goode <durham@fb.com>
parents: 34323
diff changeset
433
34152
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
434 for part in self.iterator:
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
435 # consume the bundle content
35114
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35113
diff changeset
436 part.consume()
34152
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
437 except Exception:
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
438 seekerror = True
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
439
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
440 # Small hack to let caller code distinguish exceptions from bundle2
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
441 # processing from processing the old format. This is mostly needed
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
442 # to handle different return codes to unbundle according to the type
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
443 # of bundle. We should probably clean up or drop this return code
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
444 # craziness in a future version.
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
445 exc.duringunbundle2 = True
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
446 salvaged = []
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
447 replycaps = None
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
448 if self.op.reply is not None:
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
449 salvaged = self.op.reply.salvageoutput()
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
450 replycaps = self.op.reply.capabilities
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
451 exc._replycaps = replycaps
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
452 exc._bundle2salvagedoutput = salvaged
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
453
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
454 # Re-raising from a variable loses the original stack. So only use
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
455 # that form if we need to.
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
456 if seekerror:
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
457 raise exc
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
458
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
459 self.repo.ui.debug(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
460 b'bundle2-input-bundle: %i parts total\n' % self.count
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
461 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
462
34150
e9e0e1143fc5 bundle2: move part iterator a separate class
Durham Goode <durham@fb.com>
parents: 34101
diff changeset
463
49765
4188e75af983 bundleoperation: optionnaly record the `remote` that produced the bundle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49648
diff changeset
464 def processbundle(
4188e75af983 bundleoperation: optionnaly record the `remote` that produced the bundle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49648
diff changeset
465 repo,
4188e75af983 bundleoperation: optionnaly record the `remote` that produced the bundle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49648
diff changeset
466 unbundler,
4188e75af983 bundleoperation: optionnaly record the `remote` that produced the bundle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49648
diff changeset
467 transactiongetter=None,
4188e75af983 bundleoperation: optionnaly record the `remote` that produced the bundle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49648
diff changeset
468 op=None,
4188e75af983 bundleoperation: optionnaly record the `remote` that produced the bundle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49648
diff changeset
469 source=b'',
4188e75af983 bundleoperation: optionnaly record the `remote` that produced the bundle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49648
diff changeset
470 remote=None,
4188e75af983 bundleoperation: optionnaly record the `remote` that produced the bundle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49648
diff changeset
471 ):
20889
deed5edb72de bundle2: first version of a bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20887
diff changeset
472 """This function process a bundle, apply effect to/from a repo
deed5edb72de bundle2: first version of a bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20887
diff changeset
473
20947
c33d7bf53812 bundle2: feed a unbundle20 to the `processbundle` function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20892
diff changeset
474 It iterates over each part then searches for and uses the proper handling
c33d7bf53812 bundle2: feed a unbundle20 to the `processbundle` function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20892
diff changeset
475 code to process the part. Parts are processed in order.
20889
deed5edb72de bundle2: first version of a bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20887
diff changeset
476
20891
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20890
diff changeset
477 Unknown Mandatory part will abort the process.
24851
df0ce98c882f bundle2: also save output when error happens during part processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24849
diff changeset
478
df0ce98c882f bundle2: also save output when error happens during part processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24849
diff changeset
479 It is temporarily possible to provide a prebuilt bundleoperation to the
df0ce98c882f bundle2: also save output when error happens during part processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24849
diff changeset
480 function. This is used to ensure output is properly propagated in case of
df0ce98c882f bundle2: also save output when error happens during part processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24849
diff changeset
481 an error during the unbundling. This output capturing part will likely be
df0ce98c882f bundle2: also save output when error happens during part processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24849
diff changeset
482 reworked and this ability will probably go away in the process.
20889
deed5edb72de bundle2: first version of a bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20887
diff changeset
483 """
24851
df0ce98c882f bundle2: also save output when error happens during part processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24849
diff changeset
484 if op is None:
df0ce98c882f bundle2: also save output when error happens during part processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24849
diff changeset
485 if transactiongetter is None:
df0ce98c882f bundle2: also save output when error happens during part processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24849
diff changeset
486 transactiongetter = _notransaction
49765
4188e75af983 bundleoperation: optionnaly record the `remote` that produced the bundle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49648
diff changeset
487 op = bundleoperation(
4188e75af983 bundleoperation: optionnaly record the `remote` that produced the bundle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49648
diff changeset
488 repo,
4188e75af983 bundleoperation: optionnaly record the `remote` that produced the bundle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49648
diff changeset
489 transactiongetter,
4188e75af983 bundleoperation: optionnaly record the `remote` that produced the bundle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49648
diff changeset
490 source=source,
4188e75af983 bundleoperation: optionnaly record the `remote` that produced the bundle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49648
diff changeset
491 remote=remote,
4188e75af983 bundleoperation: optionnaly record the `remote` that produced the bundle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49648
diff changeset
492 )
20889
deed5edb72de bundle2: first version of a bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20887
diff changeset
493 # todo:
deed5edb72de bundle2: first version of a bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20887
diff changeset
494 # - replace this is a init function soon.
deed5edb72de bundle2: first version of a bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20887
diff changeset
495 # - exception catching
deed5edb72de bundle2: first version of a bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20887
diff changeset
496 unbundler.params
25331
53ca47ef645d bundle2: add generic debug output regarding processed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25324
diff changeset
497 if repo.ui.debugflag:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
498 msg = [b'bundle2-input-bundle:']
25331
53ca47ef645d bundle2: add generic debug output regarding processed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25324
diff changeset
499 if unbundler.params:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
500 msg.append(b' %i params' % len(unbundler.params))
33770
119e1c6be1ce bundle2: fix transaction availability detection
Boris Feld <boris.feld@octobus.net>
parents: 33711
diff changeset
501 if op._gettransaction is None or op._gettransaction is _notransaction:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
502 msg.append(b' no-transaction')
25331
53ca47ef645d bundle2: add generic debug output regarding processed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25324
diff changeset
503 else:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
504 msg.append(b' with-transaction')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
505 msg.append(b'\n')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
506 repo.ui.debug(b''.join(msg))
34150
e9e0e1143fc5 bundle2: move part iterator a separate class
Durham Goode <durham@fb.com>
parents: 34101
diff changeset
507
34261
f010097c885c bundle2: move part processing to a separate function
Durham Goode <durham@fb.com>
parents: 34260
diff changeset
508 processparts(repo, op, unbundler)
f010097c885c bundle2: move part processing to a separate function
Durham Goode <durham@fb.com>
parents: 34260
diff changeset
509
f010097c885c bundle2: move part processing to a separate function
Durham Goode <durham@fb.com>
parents: 34260
diff changeset
510 return op
f010097c885c bundle2: move part processing to a separate function
Durham Goode <durham@fb.com>
parents: 34260
diff changeset
511
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
512
34261
f010097c885c bundle2: move part processing to a separate function
Durham Goode <durham@fb.com>
parents: 34260
diff changeset
513 def processparts(repo, op, unbundler):
34152
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
514 with partiterator(repo, op, unbundler) as parts:
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
515 for part in parts:
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34151
diff changeset
516 _processpart(op, part)
25332
0cdd1c811705 bundle2: add generic debug output at the end of bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25331
diff changeset
517
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
518
33038
f0efd2bffe1e bundle: extract _processchangegroup() method
Martin von Zweigbergk <martinvonz@google.com>
parents: 33037
diff changeset
519 def _processchangegroup(op, cg, tr, source, url, **kwargs):
49767
f1887500f3ec delta-find: add a `delta-reuse-policy` on configuration `path`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49765
diff changeset
520 if op.remote is not None and op.remote.path is not None:
f1887500f3ec delta-find: add a `delta-reuse-policy` on configuration `path`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49765
diff changeset
521 remote_path = op.remote.path
f1887500f3ec delta-find: add a `delta-reuse-policy` on configuration `path`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49765
diff changeset
522 kwargs = kwargs.copy()
f1887500f3ec delta-find: add a `delta-reuse-policy` on configuration `path`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49765
diff changeset
523 kwargs['delta_base_reuse_policy'] = remote_path.delta_reuse_policy
33461
bb72031f0ea8 changegroup: stop returning and recording added nodes in 'cg.apply'
Boris Feld <boris.feld@octobus.net>
parents: 33460
diff changeset
524 ret = cg.apply(op.repo, tr, source, url, **kwargs)
45942
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45682
diff changeset
525 op.records.add(
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45682
diff changeset
526 b'changegroup',
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45682
diff changeset
527 {
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45682
diff changeset
528 b'return': ret,
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45682
diff changeset
529 },
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45682
diff changeset
530 )
33038
f0efd2bffe1e bundle: extract _processchangegroup() method
Martin von Zweigbergk <martinvonz@google.com>
parents: 33037
diff changeset
531 return ret
f0efd2bffe1e bundle: extract _processchangegroup() method
Martin von Zweigbergk <martinvonz@google.com>
parents: 33037
diff changeset
532
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
533
34259
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34258
diff changeset
534 def _gethandler(op, part):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
535 status = b'unknown' # used by debug output
34259
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34258
diff changeset
536 try:
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34258
diff changeset
537 handler = parthandlermapping.get(part.type)
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34258
diff changeset
538 if handler is None:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
539 status = b'unsupported-type'
34259
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34258
diff changeset
540 raise error.BundleUnknownFeatureError(parttype=part.type)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
541 indebug(op.ui, b'found a handler for part %s' % part.type)
34259
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34258
diff changeset
542 unknownparams = part.mandatorykeys - handler.params
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34258
diff changeset
543 if unknownparams:
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34258
diff changeset
544 unknownparams = list(unknownparams)
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34258
diff changeset
545 unknownparams.sort()
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
546 status = b'unsupported-params (%s)' % b', '.join(unknownparams)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
547 raise error.BundleUnknownFeatureError(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
548 parttype=part.type, params=unknownparams
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
549 )
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
550 status = b'supported'
34259
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34258
diff changeset
551 except error.BundleUnknownFeatureError as exc:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
552 if part.mandatory: # mandatory parts
34259
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34258
diff changeset
553 raise
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
554 indebug(op.ui, b'ignoring unsupported advisory part %s' % exc)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
555 return # skip to part processing
34259
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34258
diff changeset
556 finally:
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34258
diff changeset
557 if op.ui.debugflag:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
558 msg = [b'bundle2-input-part: "%s"' % part.type]
34259
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34258
diff changeset
559 if not part.mandatory:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
560 msg.append(b' (advisory)')
34259
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34258
diff changeset
561 nbmp = len(part.mandatorykeys)
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34258
diff changeset
562 nbap = len(part.params) - nbmp
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34258
diff changeset
563 if nbmp or nbap:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
564 msg.append(b' (params:')
34259
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34258
diff changeset
565 if nbmp:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
566 msg.append(b' %i mandatory' % nbmp)
34259
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34258
diff changeset
567 if nbap:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
568 msg.append(b' %i advisory' % nbmp)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
569 msg.append(b')')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
570 msg.append(b' %s\n' % status)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
571 op.ui.debug(b''.join(msg))
34259
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34258
diff changeset
572
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34258
diff changeset
573 return handler
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34258
diff changeset
574
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
575
23008
d3137827016a bundle2: extract processing of part into its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23001
diff changeset
576 def _processpart(op, part):
d3137827016a bundle2: extract processing of part into its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23001
diff changeset
577 """process a single part from a bundle
d3137827016a bundle2: extract processing of part into its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23001
diff changeset
578
d3137827016a bundle2: extract processing of part into its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23001
diff changeset
579 The part is guaranteed to have been fully consumed when the function exits
d3137827016a bundle2: extract processing of part into its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23001
diff changeset
580 (even if an exception is raised)."""
34260
cc7b37c90616 bundle2: remove unnecessary try finally
Durham Goode <durham@fb.com>
parents: 34259
diff changeset
581 handler = _gethandler(op, part)
cc7b37c90616 bundle2: remove unnecessary try finally
Durham Goode <durham@fb.com>
parents: 34259
diff changeset
582 if handler is None:
cc7b37c90616 bundle2: remove unnecessary try finally
Durham Goode <durham@fb.com>
parents: 34259
diff changeset
583 return
23008
d3137827016a bundle2: extract processing of part into its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23001
diff changeset
584
34260
cc7b37c90616 bundle2: remove unnecessary try finally
Durham Goode <durham@fb.com>
parents: 34259
diff changeset
585 # handler is called outside the above try block so that we don't
cc7b37c90616 bundle2: remove unnecessary try finally
Durham Goode <durham@fb.com>
parents: 34259
diff changeset
586 # risk catching KeyErrors from anything other than the
cc7b37c90616 bundle2: remove unnecessary try finally
Durham Goode <durham@fb.com>
parents: 34259
diff changeset
587 # parthandlermapping lookup (any KeyError raised by handler()
cc7b37c90616 bundle2: remove unnecessary try finally
Durham Goode <durham@fb.com>
parents: 34259
diff changeset
588 # itself represents a defect of a different variety).
cc7b37c90616 bundle2: remove unnecessary try finally
Durham Goode <durham@fb.com>
parents: 34259
diff changeset
589 output = None
cc7b37c90616 bundle2: remove unnecessary try finally
Durham Goode <durham@fb.com>
parents: 34259
diff changeset
590 if op.captureoutput and op.reply is not None:
cc7b37c90616 bundle2: remove unnecessary try finally
Durham Goode <durham@fb.com>
parents: 34259
diff changeset
591 op.ui.pushbuffer(error=True, subproc=True)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
592 output = b''
34260
cc7b37c90616 bundle2: remove unnecessary try finally
Durham Goode <durham@fb.com>
parents: 34259
diff changeset
593 try:
cc7b37c90616 bundle2: remove unnecessary try finally
Durham Goode <durham@fb.com>
parents: 34259
diff changeset
594 handler(op, part)
23008
d3137827016a bundle2: extract processing of part into its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23001
diff changeset
595 finally:
34260
cc7b37c90616 bundle2: remove unnecessary try finally
Durham Goode <durham@fb.com>
parents: 34259
diff changeset
596 if output is not None:
cc7b37c90616 bundle2: remove unnecessary try finally
Durham Goode <durham@fb.com>
parents: 34259
diff changeset
597 output = op.ui.popbuffer()
cc7b37c90616 bundle2: remove unnecessary try finally
Durham Goode <durham@fb.com>
parents: 34259
diff changeset
598 if output:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
599 outpart = op.reply.newpart(b'output', data=output, mandatory=False)
34260
cc7b37c90616 bundle2: remove unnecessary try finally
Durham Goode <durham@fb.com>
parents: 34259
diff changeset
600 outpart.addparam(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
601 b'in-reply-to', pycompat.bytestr(part.id), mandatory=False
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
602 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
603
23008
d3137827016a bundle2: extract processing of part into its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23001
diff changeset
604
21138
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
605 def decodecaps(blob):
23139
e53f6b72a0e4 spelling: fixes from proofreading of spell checker issues
Mads Kiilerich <madski@unity3d.com>
parents: 23067
diff changeset
606 """decode a bundle2 caps bytes blob into a dictionary
21138
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
607
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
608 The blob is a list of capabilities (one per line)
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
609 Capabilities may have values using a line of the form::
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
610
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
611 capability=value1,value2,value3
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
612
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
613 The values are always a list."""
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
614 caps = {}
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
615 for line in blob.splitlines():
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
616 if not line:
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
617 continue
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
618 if b'=' not in line:
21138
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
619 key, vals = line, ()
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
620 else:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
621 key, vals = line.split(b'=', 1)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
622 vals = vals.split(b',')
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28672
diff changeset
623 key = urlreq.unquote(key)
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28672
diff changeset
624 vals = [urlreq.unquote(v) for v in vals]
21138
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
625 caps[key] = vals
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
626 return caps
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
627
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
628
21139
2b8c82f7f11d bundle2: capabilities encoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21138
diff changeset
629 def encodecaps(caps):
2b8c82f7f11d bundle2: capabilities encoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21138
diff changeset
630 """encode a bundle2 caps dictionary into a bytes blob"""
2b8c82f7f11d bundle2: capabilities encoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21138
diff changeset
631 chunks = []
2b8c82f7f11d bundle2: capabilities encoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21138
diff changeset
632 for ca in sorted(caps):
2b8c82f7f11d bundle2: capabilities encoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21138
diff changeset
633 vals = caps[ca]
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28672
diff changeset
634 ca = urlreq.quote(ca)
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28672
diff changeset
635 vals = [urlreq.quote(v) for v in vals]
21139
2b8c82f7f11d bundle2: capabilities encoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21138
diff changeset
636 if vals:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
637 ca = b"%s=%s" % (ca, b','.join(vals))
21139
2b8c82f7f11d bundle2: capabilities encoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21138
diff changeset
638 chunks.append(ca)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
639 return b'\n'.join(chunks)
21139
2b8c82f7f11d bundle2: capabilities encoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21138
diff changeset
640
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
641
28666
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
642 bundletypes = {
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
643 b"": (b"", b'UN'), # only when using unbundle on ssh and old http servers
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
644 # since the unification ssh accepts a header but there
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
645 # is no capability signaling it.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
646 b"HG20": (), # special-cased below
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
647 b"HG10UN": (b"HG10UN", b'UN'),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
648 b"HG10BZ": (b"HG10", b'BZ'),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
649 b"HG10GZ": (b"HG10GZ", b'GZ'),
28666
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
650 }
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
651
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
652 # hgweb uses this list to communicate its preferred type
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
653 bundlepriority = [b'HG10GZ', b'HG10BZ', b'HG10UN']
28666
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
654
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
655
48946
642e31cb55f0 py3: use class X: instead of class X(object):
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48913
diff changeset
656 class bundle20:
20801
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
657 """represent an outgoing bundle2 container
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
658
21599
57cd844d7a5b bundle2: have ``newpart`` automatically add the part to the bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21598
diff changeset
659 Use the `addparam` method to add stream level parameter. and `newpart` to
20856
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
660 populate it. Then call `getchunks` to retrieve all the binary chunks of
21024
7731a2281cf0 spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents: 21020
diff changeset
661 data that compose the bundle2 container."""
20801
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
662
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
663 _magicstring = b'HG20'
24640
685639f9430d bundle20: move magic string into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24071
diff changeset
664
21134
2f8c4fa237f5 bundle2: adds a capabilities attribute on bundler20
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21133
diff changeset
665 def __init__(self, ui, capabilities=()):
20842
938718d72624 bundle2: print debug information during bundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20814
diff changeset
666 self.ui = ui
20801
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
667 self._params = []
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
668 self._parts = []
21136
b6fd496e5c72 bundle2: support for capabilities with values
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21135
diff changeset
669 self.capabilities = dict(capabilities)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
670 self._compengine = util.compengines.forbundletype(b'UN')
30757
511a4bf52754 bundle2: allow compression options to be passed to compressor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30439
diff changeset
671 self._compopts = None
35787
a84dbc87dae9 exchange: send bundle2 stream clones uncompressed
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35783
diff changeset
672 # If compression is being handled by a consumer of the raw
a84dbc87dae9 exchange: send bundle2 stream clones uncompressed
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35783
diff changeset
673 # data (e.g. the wire protocol), unsetting this flag tells
a84dbc87dae9 exchange: send bundle2 stream clones uncompressed
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35783
diff changeset
674 # consumers that the bundle is best left uncompressed.
a84dbc87dae9 exchange: send bundle2 stream clones uncompressed
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35783
diff changeset
675 self.prefercompressed = True
26404
795f02a24b87 bundle2: allow compressed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26396
diff changeset
676
30757
511a4bf52754 bundle2: allow compression options to be passed to compressor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30439
diff changeset
677 def setcompression(self, alg, compopts=None):
26404
795f02a24b87 bundle2: allow compressed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26396
diff changeset
678 """setup core part compression to <alg>"""
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
679 if alg in (None, b'UN'):
26404
795f02a24b87 bundle2: allow compressed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26396
diff changeset
680 return
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
681 assert not any(n.lower() == b'compression' for n, v in self._params)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
682 self.addparam(b'Compression', alg)
30351
f81002f736d7 bundle2: use new compression engine API for compression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30177
diff changeset
683 self._compengine = util.compengines.forbundletype(alg)
30757
511a4bf52754 bundle2: allow compression options to be passed to compressor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30439
diff changeset
684 self._compopts = compopts
20801
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
685
21900
b8bd97085ec9 bundle2: add a ``bundle20.nbparts`` property
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21660
diff changeset
686 @property
b8bd97085ec9 bundle2: add a ``bundle20.nbparts`` property
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21660
diff changeset
687 def nbparts(self):
b8bd97085ec9 bundle2: add a ``bundle20.nbparts`` property
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21660
diff changeset
688 """total number of parts added to the bundler"""
b8bd97085ec9 bundle2: add a ``bundle20.nbparts`` property
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21660
diff changeset
689 return len(self._parts)
b8bd97085ec9 bundle2: add a ``bundle20.nbparts`` property
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21660
diff changeset
690
21597
1daad9dcdba2 bundle2: small doc update on the bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21186
diff changeset
691 # methods used to defines the bundle2 content
20804
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
692 def addparam(self, name, value=None):
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
693 """add a stream level parameter"""
20813
8c74b3ce5b70 bundle2: refuse empty parameter name
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20812
diff changeset
694 if not name:
38609
d474b3b44d4f bundle2: use ProgrammingError to report bad use of addparam()
Yuya Nishihara <yuya@tcha.org>
parents: 37842
diff changeset
695 raise error.ProgrammingError(b'empty parameter name')
43488
ea25b4673231 bundle2: more pytype suppressions around bytestring
Augie Fackler <augie@google.com>
parents: 43131
diff changeset
696 if name[0:1] not in pycompat.bytestr(
ea25b4673231 bundle2: more pytype suppressions around bytestring
Augie Fackler <augie@google.com>
parents: 43131
diff changeset
697 string.ascii_letters # pytype: disable=wrong-arg-types
ea25b4673231 bundle2: more pytype suppressions around bytestring
Augie Fackler <augie@google.com>
parents: 43131
diff changeset
698 ):
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
699 raise error.ProgrammingError(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
700 b'non letter first character: %s' % name
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
701 )
20804
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
702 self._params.append((name, value))
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
703
20856
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
704 def addpart(self, part):
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
705 """add a new part to the bundle2 container
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
706
21024
7731a2281cf0 spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents: 21020
diff changeset
707 Parts contains the actual applicative payload."""
20995
e995d104c87f bundle2: add an integer id to part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20952
diff changeset
708 assert part.id is None
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
709 part.id = len(self._parts) # very cheap counter
20856
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
710 self._parts.append(part)
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
711
21598
1b0dbb91de5b bundle2: add a ``newpart`` method to ``bundle20``
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21597
diff changeset
712 def newpart(self, typeid, *args, **kwargs):
21602
cc33ae50bab3 bundle2: warn about error during initialization in ``newpart`` docstring
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21601
diff changeset
713 """create a new part and add it to the containers
cc33ae50bab3 bundle2: warn about error during initialization in ``newpart`` docstring
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21601
diff changeset
714
cc33ae50bab3 bundle2: warn about error during initialization in ``newpart`` docstring
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21601
diff changeset
715 As the part is directly added to the containers. For now, this means
cc33ae50bab3 bundle2: warn about error during initialization in ``newpart`` docstring
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21601
diff changeset
716 that any failure to properly initialize the part after calling
cc33ae50bab3 bundle2: warn about error during initialization in ``newpart`` docstring
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21601
diff changeset
717 ``newpart`` should result in a failure of the whole bundling process.
cc33ae50bab3 bundle2: warn about error during initialization in ``newpart`` docstring
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21601
diff changeset
718
cc33ae50bab3 bundle2: warn about error during initialization in ``newpart`` docstring
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21601
diff changeset
719 You can still fall back to manually create and add if you need better
cc33ae50bab3 bundle2: warn about error during initialization in ``newpart`` docstring
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21601
diff changeset
720 control."""
21598
1b0dbb91de5b bundle2: add a ``newpart`` method to ``bundle20``
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21597
diff changeset
721 part = bundlepart(typeid, *args, **kwargs)
21599
57cd844d7a5b bundle2: have ``newpart`` automatically add the part to the bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21598
diff changeset
722 self.addpart(part)
21598
1b0dbb91de5b bundle2: add a ``newpart`` method to ``bundle20``
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21597
diff changeset
723 return part
1b0dbb91de5b bundle2: add a ``newpart`` method to ``bundle20``
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21597
diff changeset
724
21597
1daad9dcdba2 bundle2: small doc update on the bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21186
diff changeset
725 # methods used to generate the bundle2 stream
20801
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
726 def getchunks(self):
25322
1f7c0403f8be bundle2: add generic debug output regarding generated bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25321
diff changeset
727 if self.ui.debugflag:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
728 msg = [b'bundle2-output-bundle: "%s",' % self._magicstring]
25322
1f7c0403f8be bundle2: add generic debug output regarding generated bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25321
diff changeset
729 if self._params:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
730 msg.append(b' (%i params)' % len(self._params))
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
731 msg.append(b' %i parts total\n' % len(self._parts))
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
732 self.ui.debug(b''.join(msg))
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
733 outdebug(self.ui, b'start emission of %s stream' % self._magicstring)
24640
685639f9430d bundle20: move magic string into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24071
diff changeset
734 yield self._magicstring
20804
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
735 param = self._paramchunk()
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
736 outdebug(self.ui, b'bundle parameter: %s' % param)
20804
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
737 yield _pack(_fstreamparamsize, len(param))
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
738 if param:
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
739 yield param
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
740 for chunk in self._compengine.compressstream(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
741 self._getcorechunk(), self._compopts
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
742 ):
30357
5925bda42dbd bundle2: use compressstream compression engine API
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30353
diff changeset
743 yield chunk
20802
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
744
20804
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
745 def _paramchunk(self):
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
746 """return a encoded version of all stream parameters"""
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
747 blocks = []
20809
b93bb639451a bundle2: support for bundling parameter value
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20808
diff changeset
748 for par, value in self._params:
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28672
diff changeset
749 par = urlreq.quote(par)
20809
b93bb639451a bundle2: support for bundling parameter value
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20808
diff changeset
750 if value is not None:
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28672
diff changeset
751 value = urlreq.quote(value)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
752 par = b'%s=%s' % (par, value)
20809
b93bb639451a bundle2: support for bundling parameter value
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20808
diff changeset
753 blocks.append(par)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
754 return b' '.join(blocks)
20804
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
755
26396
d90c30801cdf bundle20: extract core payload generation in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26395
diff changeset
756 def _getcorechunk(self):
d90c30801cdf bundle20: extract core payload generation in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26395
diff changeset
757 """yield chunk for the core part of the bundle
d90c30801cdf bundle20: extract core payload generation in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26395
diff changeset
758
d90c30801cdf bundle20: extract core payload generation in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26395
diff changeset
759 (all but headers and parameters)"""
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
760 outdebug(self.ui, b'start of parts')
26396
d90c30801cdf bundle20: extract core payload generation in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26395
diff changeset
761 for part in self._parts:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
762 outdebug(self.ui, b'bundle part: "%s"' % part.type)
26396
d90c30801cdf bundle20: extract core payload generation in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26395
diff changeset
763 for chunk in part.getchunks(ui=self.ui):
d90c30801cdf bundle20: extract core payload generation in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26395
diff changeset
764 yield chunk
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
765 outdebug(self.ui, b'end of bundle')
26396
d90c30801cdf bundle20: extract core payload generation in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26395
diff changeset
766 yield _pack(_fpartheadersize, 0)
d90c30801cdf bundle20: extract core payload generation in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26395
diff changeset
767
24794
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24793
diff changeset
768 def salvageoutput(self):
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24793
diff changeset
769 """return a list with a copy of all output parts in the bundle
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24793
diff changeset
770
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24793
diff changeset
771 This is meant to be used during error handling to make sure we preserve
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24793
diff changeset
772 server output"""
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24793
diff changeset
773 salvaged = []
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24793
diff changeset
774 for part in self._parts:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
775 if part.type.startswith(b'output'):
24794
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24793
diff changeset
776 salvaged.append(part.copy())
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24793
diff changeset
777 return salvaged
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24793
diff changeset
778
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24793
diff changeset
779
48946
642e31cb55f0 py3: use class X: instead of class X(object):
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48913
diff changeset
780 class unpackermixin:
21013
a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21005
diff changeset
781 """A mixin to extract bytes and struct data from a stream"""
20802
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
782
21013
a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21005
diff changeset
783 def __init__(self, fp):
20802
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
784 self._fp = fp
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
785
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
786 def _unpack(self, format):
31862
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31861
diff changeset
787 """unpack this struct format from the stream
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31861
diff changeset
788
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31861
diff changeset
789 This method is meant for internal usage by the bundle2 protocol only.
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31861
diff changeset
790 They directly manipulate the low level stream including bundle2 level
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31861
diff changeset
791 instruction.
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31861
diff changeset
792
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31861
diff changeset
793 Do not use it to implement higher-level logic or methods."""
20802
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
794 data = self._readexact(struct.calcsize(format))
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
795 return _unpack(format, data)
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
796
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
797 def _readexact(self, size):
31862
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31861
diff changeset
798 """read exactly <size> bytes from the stream
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31861
diff changeset
799
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31861
diff changeset
800 This method is meant for internal usage by the bundle2 protocol only.
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31861
diff changeset
801 They directly manipulate the low level stream including bundle2 level
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31861
diff changeset
802 instruction.
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31861
diff changeset
803
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31861
diff changeset
804 Do not use it to implement higher-level logic or methods."""
20802
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
805 return changegroup.readexactly(self._fp, size)
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
806
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
807
25640
39f0064a3079 bundle2.getunbundler: rename "header" to "magicstring"
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25518
diff changeset
808 def getunbundler(ui, fp, magicstring=None):
39f0064a3079 bundle2.getunbundler: rename "header" to "magicstring"
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25518
diff changeset
809 """return a valid unbundler object for a given magicstring"""
39f0064a3079 bundle2.getunbundler: rename "header" to "magicstring"
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25518
diff changeset
810 if magicstring is None:
39f0064a3079 bundle2.getunbundler: rename "header" to "magicstring"
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25518
diff changeset
811 magicstring = changegroup.readexactly(fp, 4)
39f0064a3079 bundle2.getunbundler: rename "header" to "magicstring"
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25518
diff changeset
812 magic, version = magicstring[0:2], magicstring[2:4]
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
813 if magic != b'HG':
33127
0aae80d14fed bundle2: add some debugging information to the not-a-bundle error
Siddharth Agarwal <sid0@fb.com>
parents: 33044
diff changeset
814 ui.debug(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
815 b"error: invalid magic: %r (version %r), should be 'HG'\n"
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
816 % (magic, version)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
817 )
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
818 raise error.Abort(_(b'not a Mercurial bundle'))
24648
5cac3accdaa1 unbundle20: allow generic dispatch between unbundlers
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24642
diff changeset
819 unbundlerclass = formatmap.get(version)
5cac3accdaa1 unbundle20: allow generic dispatch between unbundlers
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24642
diff changeset
820 if unbundlerclass is None:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
821 raise error.Abort(_(b'unknown bundle version %s') % version)
24648
5cac3accdaa1 unbundle20: allow generic dispatch between unbundlers
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24642
diff changeset
822 unbundler = unbundlerclass(ui, fp)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
823 indebug(ui, b'start processing of %s stream' % magicstring)
24642
54e5c239c2d9 unbundle20: move header parsing into the 'getunbundler' function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24641
diff changeset
824 return unbundler
24641
60fecc5b14a4 unbundle20: retrieve unbundler instances through a factory function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24640
diff changeset
825
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
826
21013
a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21005
diff changeset
827 class unbundle20(unpackermixin):
a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21005
diff changeset
828 """interpret a bundle2 stream
a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21005
diff changeset
829
21129
07bcbf326c8d bundle2: use an official iterparts method to unbundle parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21066
diff changeset
830 This class is fed with a binary stream and yields parts through its
07bcbf326c8d bundle2: use an official iterparts method to unbundle parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21066
diff changeset
831 `iterparts` methods."""
21013
a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21005
diff changeset
832
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
833 _magicstring = b'HG20'
26542
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
834
24642
54e5c239c2d9 unbundle20: move header parsing into the 'getunbundler' function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24641
diff changeset
835 def __init__(self, ui, fp):
21066
5ecfe76d0d96 bundle2: make header reading optional
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21062
diff changeset
836 """If header is specified, we do not read it out of the stream."""
21013
a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21005
diff changeset
837 self.ui = ui
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
838 self._compengine = util.compengines.forbundletype(b'UN')
26802
42f705f2c02d bundle2: make unbundle.compressed return True when compressed
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26795
diff changeset
839 self._compressed = None
21013
a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21005
diff changeset
840 super(unbundle20, self).__init__(fp)
a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21005
diff changeset
841
20802
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
842 @util.propertycache
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
843 def params(self):
21024
7731a2281cf0 spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents: 21020
diff changeset
844 """dictionary of stream level parameters"""
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
845 indebug(self.ui, b'reading bundle2 stream parameters')
20805
c5aaeca0cfbf bundle2: support for unbundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20804
diff changeset
846 params = {}
c5aaeca0cfbf bundle2: support for unbundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20804
diff changeset
847 paramssize = self._unpack(_fstreamparamsize)[0]
23011
006a81d07e57 bundle2: detect and disallow a negative chunk size
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23010
diff changeset
848 if paramssize < 0:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
849 raise error.BundleValueError(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
850 b'negative bundle param size: %i' % paramssize
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
851 )
20805
c5aaeca0cfbf bundle2: support for unbundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20804
diff changeset
852 if paramssize:
26541
d40029b4296e bundle2: split parameter retrieval and processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26404
diff changeset
853 params = self._readexact(paramssize)
d40029b4296e bundle2: split parameter retrieval and processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26404
diff changeset
854 params = self._processallparams(params)
20805
c5aaeca0cfbf bundle2: support for unbundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20804
diff changeset
855 return params
20802
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
856
26541
d40029b4296e bundle2: split parameter retrieval and processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26404
diff changeset
857 def _processallparams(self, paramsblock):
47062
f38bf44e077f black: make codebase compatible with black v21.4b2 and v20.8b1
Kyle Lippincott <spectral@google.com>
parents: 46907
diff changeset
858 """ """
29591
6215b5537ba5 bundle2: use a sorted dict for holding parameters
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29294
diff changeset
859 params = util.sortdict()
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
860 for p in paramsblock.split(b' '):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
861 p = p.split(b'=', 1)
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28672
diff changeset
862 p = [urlreq.unquote(i) for i in p]
26541
d40029b4296e bundle2: split parameter retrieval and processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26404
diff changeset
863 if len(p) < 2:
d40029b4296e bundle2: split parameter retrieval and processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26404
diff changeset
864 p.append(None)
d40029b4296e bundle2: split parameter retrieval and processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26404
diff changeset
865 self._processparam(*p)
d40029b4296e bundle2: split parameter retrieval and processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26404
diff changeset
866 params[p[0]] = p[1]
d40029b4296e bundle2: split parameter retrieval and processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26404
diff changeset
867 return params
d40029b4296e bundle2: split parameter retrieval and processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26404
diff changeset
868
20844
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
869 def _processparam(self, name, value):
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
870 """process a parameter, applying its effect if needed
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
871
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
872 Parameter starting with a lower case letter are advisory and will be
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
873 ignored when unknown. Those starting with an upper case letter are
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
874 mandatory and will this function will raise a KeyError when unknown.
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
875
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
876 Note: no option are currently supported. Any input will be either
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
877 ignored or failing.
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
878 """
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
879 if not name:
43506
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43495
diff changeset
880 raise ValueError('empty parameter name')
43488
ea25b4673231 bundle2: more pytype suppressions around bytestring
Augie Fackler <augie@google.com>
parents: 43131
diff changeset
881 if name[0:1] not in pycompat.bytestr(
ea25b4673231 bundle2: more pytype suppressions around bytestring
Augie Fackler <augie@google.com>
parents: 43131
diff changeset
882 string.ascii_letters # pytype: disable=wrong-arg-types
ea25b4673231 bundle2: more pytype suppressions around bytestring
Augie Fackler <augie@google.com>
parents: 43131
diff changeset
883 ):
43506
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43495
diff changeset
884 raise ValueError('non letter first character: %s' % name)
26395
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26393
diff changeset
885 try:
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26393
diff changeset
886 handler = b2streamparamsmap[name.lower()]
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26393
diff changeset
887 except KeyError:
34277
a48b3db0baba bundle2: stop using %r to quote part names
Augie Fackler <augie@google.com>
parents: 34261
diff changeset
888 if name[0:1].islower():
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
889 indebug(self.ui, b"ignoring unknown parameter %s" % name)
26395
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26393
diff changeset
890 else:
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26393
diff changeset
891 raise error.BundleUnknownFeatureError(params=(name,))
20844
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
892 else:
26395
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26393
diff changeset
893 handler(self, name, value)
20844
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
894
26542
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
895 def _forwardchunks(self):
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
896 """utility to transfer a bundle2 as binary
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
897
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
898 This is made necessary by the fact the 'getbundle' command over 'ssh'
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
899 have no way to know then the reply end, relying on the bundle to be
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
900 interpreted to know its end. This is terrible and we are sorry, but we
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
901 needed to move forward to get general delta enabled.
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
902 """
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
903 yield self._magicstring
43421
be384a2052aa py3: don't use bytes with vars() or __dict__
Martin von Zweigbergk <martinvonz@google.com>
parents: 43131
diff changeset
904 assert 'params' not in vars(self)
26542
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
905 paramssize = self._unpack(_fstreamparamsize)[0]
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
906 if paramssize < 0:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
907 raise error.BundleValueError(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
908 b'negative bundle param size: %i' % paramssize
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
909 )
26542
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
910 if paramssize:
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
911 params = self._readexact(paramssize)
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
912 self._processallparams(params)
42143
29569f2db929 bundle2: handle compression in _forwardchunks
Joerg Sonnenberger <joerg@bec.de>
parents: 42057
diff changeset
913 # The payload itself is decompressed below, so drop
29569f2db929 bundle2: handle compression in _forwardchunks
Joerg Sonnenberger <joerg@bec.de>
parents: 42057
diff changeset
914 # the compression parameter passed down to compensate.
29569f2db929 bundle2: handle compression in _forwardchunks
Joerg Sonnenberger <joerg@bec.de>
parents: 42057
diff changeset
915 outparams = []
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
916 for p in params.split(b' '):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
917 k, v = p.split(b'=', 1)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
918 if k.lower() != b'compression':
42143
29569f2db929 bundle2: handle compression in _forwardchunks
Joerg Sonnenberger <joerg@bec.de>
parents: 42057
diff changeset
919 outparams.append(p)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
920 outparams = b' '.join(outparams)
42143
29569f2db929 bundle2: handle compression in _forwardchunks
Joerg Sonnenberger <joerg@bec.de>
parents: 42057
diff changeset
921 yield _pack(_fstreamparamsize, len(outparams))
29569f2db929 bundle2: handle compression in _forwardchunks
Joerg Sonnenberger <joerg@bec.de>
parents: 42057
diff changeset
922 yield outparams
29569f2db929 bundle2: handle compression in _forwardchunks
Joerg Sonnenberger <joerg@bec.de>
parents: 42057
diff changeset
923 else:
29569f2db929 bundle2: handle compression in _forwardchunks
Joerg Sonnenberger <joerg@bec.de>
parents: 42057
diff changeset
924 yield _pack(_fstreamparamsize, paramssize)
26542
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
925 # From there, payload might need to be decompressed
30353
d045b4091197 bundle2: use compression engines API to obtain decompressor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30351
diff changeset
926 self._fp = self._compengine.decompressorreader(self._fp)
26542
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
927 emptycount = 0
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
928 while emptycount < 2:
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
929 # so we can brainlessly loop
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
930 assert _fpartheadersize == _fpayloadsize
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
931 size = self._unpack(_fpartheadersize)[0]
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
932 yield _pack(_fpartheadersize, size)
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
933 if size:
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
934 emptycount = 0
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
935 else:
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
936 emptycount += 1
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
937 continue
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
938 if size == flaginterrupt:
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
939 continue
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
940 elif size < 0:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
941 raise error.BundleValueError(b'negative chunk size: %i')
26542
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
942 yield self._readexact(size)
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
943
35116
da91e7309daf bundle2: don't use seekable bundle2 parts by default (issue5691)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35115
diff changeset
944 def iterparts(self, seekable=False):
20802
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
945 """yield all parts contained in the stream"""
35116
da91e7309daf bundle2: don't use seekable bundle2 parts by default (issue5691)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35115
diff changeset
946 cls = seekableunbundlepart if seekable else unbundlepart
20802
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
947 # make sure param have been loaded
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
948 self.params
26404
795f02a24b87 bundle2: allow compressed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26396
diff changeset
949 # From there, payload need to be decompressed
30353
d045b4091197 bundle2: use compression engines API to obtain decompressor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30351
diff changeset
950 self._fp = self._compengine.decompressorreader(self._fp)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
951 indebug(self.ui, b'start extraction of bundle2 parts')
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
952 headerblock = self._readpartheader()
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
953 while headerblock is not None:
35116
da91e7309daf bundle2: don't use seekable bundle2 parts by default (issue5691)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35115
diff changeset
954 part = cls(self.ui, headerblock, self._fp)
20802
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
955 yield part
35114
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35113
diff changeset
956 # Ensure part is fully consumed so we can start reading the next
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35113
diff changeset
957 # part.
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35113
diff changeset
958 part.consume()
35115
2b72bc88043f bundle2: only seek to beginning of part in bundlerepo
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35114
diff changeset
959
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
960 headerblock = self._readpartheader()
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
961 indebug(self.ui, b'end of bundle2 stream')
20802
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
962
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
963 def _readpartheader(self):
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
964 """reads a part header size and return the bytes blob
20864
9a75d2559cff bundle2: support unbundling empty part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20856
diff changeset
965
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
966 returns None if empty"""
20864
9a75d2559cff bundle2: support unbundling empty part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20856
diff changeset
967 headersize = self._unpack(_fpartheadersize)[0]
23011
006a81d07e57 bundle2: detect and disallow a negative chunk size
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23010
diff changeset
968 if headersize < 0:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
969 raise error.BundleValueError(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
970 b'negative part header size: %i' % headersize
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
971 )
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
972 indebug(self.ui, b'part header size: %i' % headersize)
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
973 if headersize:
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
974 return self._readexact(headersize)
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
975 return None
20864
9a75d2559cff bundle2: support unbundling empty part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20856
diff changeset
976
24071
184a2f6f40da bundle2.unbundle20: add compressed() method
Eric Sumner <ericsumner@fb.com>
parents: 24070
diff changeset
977 def compressed(self):
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
978 self.params # load params
26802
42f705f2c02d bundle2: make unbundle.compressed return True when compressed
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26795
diff changeset
979 return self._compressed
20802
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
980
31863
cd7aaf344d83 bundle2: move the 'close' method off the unpackermixin
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31862
diff changeset
981 def close(self):
cd7aaf344d83 bundle2: move the 'close' method off the unpackermixin
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31862
diff changeset
982 """close underlying file"""
43115
4aa72cdf616f py3: delete b'' prefix from safehasattr arguments
Martin von Zweigbergk <martinvonz@google.com>
parents: 43080
diff changeset
983 if util.safehasattr(self._fp, 'close'):
31863
cd7aaf344d83 bundle2: move the 'close' method off the unpackermixin
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31862
diff changeset
984 return self._fp.close()
cd7aaf344d83 bundle2: move the 'close' method off the unpackermixin
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31862
diff changeset
985
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
986
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
987 formatmap = {b'20': unbundle20}
24648
5cac3accdaa1 unbundle20: allow generic dispatch between unbundlers
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24642
diff changeset
988
26395
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26393
diff changeset
989 b2streamparamsmap = {}
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26393
diff changeset
990
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
991
26395
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26393
diff changeset
992 def b2streamparamhandler(name):
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26393
diff changeset
993 """register a handler for a stream level parameter"""
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
994
26395
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26393
diff changeset
995 def decorator(func):
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26393
diff changeset
996 assert name not in formatmap
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26393
diff changeset
997 b2streamparamsmap[name] = func
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26393
diff changeset
998 return func
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
999
26395
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26393
diff changeset
1000 return decorator
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26393
diff changeset
1001
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1002
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1003 @b2streamparamhandler(b'compression')
26404
795f02a24b87 bundle2: allow compressed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26396
diff changeset
1004 def processcompression(unbundler, param, value):
795f02a24b87 bundle2: allow compressed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26396
diff changeset
1005 """read compression parameter and install payload decompression"""
30353
d045b4091197 bundle2: use compression engines API to obtain decompressor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30351
diff changeset
1006 if value not in util.compengines.supportedbundletypes:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1007 raise error.BundleUnknownFeatureError(params=(param,), values=(value,))
30353
d045b4091197 bundle2: use compression engines API to obtain decompressor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30351
diff changeset
1008 unbundler._compengine = util.compengines.forbundletype(value)
26802
42f705f2c02d bundle2: make unbundle.compressed return True when compressed
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26795
diff changeset
1009 if value is not None:
42f705f2c02d bundle2: make unbundle.compressed return True when compressed
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26795
diff changeset
1010 unbundler._compressed = True
26404
795f02a24b87 bundle2: allow compressed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26396
diff changeset
1011
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1012
48946
642e31cb55f0 py3: use class X: instead of class X(object):
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48913
diff changeset
1013 class bundlepart:
20856
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
1014 """A bundle2 part contains application level payload
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
1015
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
1016 The part `type` is used to route the part to the application level
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
1017 handler.
21604
c399bf961cb9 bundle2: the ability to set ``data`` attribute of the part is now official
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21603
diff changeset
1018
c399bf961cb9 bundle2: the ability to set ``data`` attribute of the part is now official
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21603
diff changeset
1019 The part payload is contained in ``part.data``. It could be raw bytes or a
21605
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
1020 generator of byte chunks.
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
1021
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
1022 You can add parameters to the part using the ``addparam`` method.
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
1023 Parameters can be either mandatory (default) or advisory. Remote side
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
1024 should be able to safely ignore the advisory ones.
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
1025
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
1026 Both data and parameters cannot be modified after the generation has begun.
20856
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
1027 """
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
1028
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1029 def __init__(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1030 self,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1031 parttype,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1032 mandatoryparams=(),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1033 advisoryparams=(),
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1034 data=b'',
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1035 mandatory=True,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1036 ):
23868
405eacbecc52 bundle2: enforce parttype as alphanumerical
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23590
diff changeset
1037 validateparttype(parttype)
20995
e995d104c87f bundle2: add an integer id to part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20952
diff changeset
1038 self.id = None
20856
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
1039 self.type = parttype
21604
c399bf961cb9 bundle2: the ability to set ``data`` attribute of the part is now official
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21603
diff changeset
1040 self._data = data
21605
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
1041 self._mandatoryparams = list(mandatoryparams)
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
1042 self._advisoryparams = list(advisoryparams)
21607
054fa5176fa7 bundle2: forbid duplicate parameter keys
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21606
diff changeset
1043 # checking for duplicated entries
054fa5176fa7 bundle2: forbid duplicate parameter keys
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21606
diff changeset
1044 self._seenparams = set()
054fa5176fa7 bundle2: forbid duplicate parameter keys
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21606
diff changeset
1045 for pname, __ in self._mandatoryparams + self._advisoryparams:
054fa5176fa7 bundle2: forbid duplicate parameter keys
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21606
diff changeset
1046 if pname in self._seenparams:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1047 raise error.ProgrammingError(b'duplicated params: %s' % pname)
21607
054fa5176fa7 bundle2: forbid duplicate parameter keys
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21606
diff changeset
1048 self._seenparams.add(pname)
21601
7ff01befc7ec bundle2: track life cycle of parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21600
diff changeset
1049 # status of the part's generation:
7ff01befc7ec bundle2: track life cycle of parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21600
diff changeset
1050 # - None: not started,
7ff01befc7ec bundle2: track life cycle of parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21600
diff changeset
1051 # - False: currently generated,
7ff01befc7ec bundle2: track life cycle of parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21600
diff changeset
1052 # - True: generation done.
7ff01befc7ec bundle2: track life cycle of parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21600
diff changeset
1053 self._generated = None
23590
4440c7cc3728 bundle2.bundlepart: make mandatory part flag explicit in API
Eric Sumner <ericsumner@fb.com>
parents: 23586
diff changeset
1054 self.mandatory = mandatory
20856
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
1055
30872
1f51b4658f21 bundle2: implement a basic __repr__ for bundle2 part
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 30871
diff changeset
1056 def __repr__(self):
44256
74172a234dd3 py3: fully fix bundlepart.__repr__ to return str not bytes
Kyle Lippincott <spectral@google.com>
parents: 44247
diff changeset
1057 cls = "%s.%s" % (self.__class__.__module__, self.__class__.__name__)
74172a234dd3 py3: fully fix bundlepart.__repr__ to return str not bytes
Kyle Lippincott <spectral@google.com>
parents: 44247
diff changeset
1058 return '<%s object at %x; id: %s; type: %s; mandatory: %s>' % (
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1059 cls,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1060 id(self),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1061 self.id,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1062 self.type,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1063 self.mandatory,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1064 )
30872
1f51b4658f21 bundle2: implement a basic __repr__ for bundle2 part
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 30871
diff changeset
1065
24793
2ec894582ea2 bundle2: add a 'copy' method on parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24743
diff changeset
1066 def copy(self):
2ec894582ea2 bundle2: add a 'copy' method on parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24743
diff changeset
1067 """return a copy of the part
2ec894582ea2 bundle2: add a 'copy' method on parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24743
diff changeset
1068
2ec894582ea2 bundle2: add a 'copy' method on parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24743
diff changeset
1069 The new part have the very same content but no partid assigned yet.
2ec894582ea2 bundle2: add a 'copy' method on parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24743
diff changeset
1070 Parts with generated data cannot be copied."""
43115
4aa72cdf616f py3: delete b'' prefix from safehasattr arguments
Martin von Zweigbergk <martinvonz@google.com>
parents: 43080
diff changeset
1071 assert not util.safehasattr(self.data, 'next')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1072 return self.__class__(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1073 self.type,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1074 self._mandatoryparams,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1075 self._advisoryparams,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1076 self._data,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1077 self.mandatory,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1078 )
24793
2ec894582ea2 bundle2: add a 'copy' method on parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24743
diff changeset
1079
21604
c399bf961cb9 bundle2: the ability to set ``data`` attribute of the part is now official
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21603
diff changeset
1080 # methods used to defines the part content
27879
52a4ad62b006 cleanup: use modern @property/@foo.setter property specification
Augie Fackler <augie@google.com>
parents: 27751
diff changeset
1081 @property
52a4ad62b006 cleanup: use modern @property/@foo.setter property specification
Augie Fackler <augie@google.com>
parents: 27751
diff changeset
1082 def data(self):
52a4ad62b006 cleanup: use modern @property/@foo.setter property specification
Augie Fackler <augie@google.com>
parents: 27751
diff changeset
1083 return self._data
52a4ad62b006 cleanup: use modern @property/@foo.setter property specification
Augie Fackler <augie@google.com>
parents: 27751
diff changeset
1084
52a4ad62b006 cleanup: use modern @property/@foo.setter property specification
Augie Fackler <augie@google.com>
parents: 27751
diff changeset
1085 @data.setter
52a4ad62b006 cleanup: use modern @property/@foo.setter property specification
Augie Fackler <augie@google.com>
parents: 27751
diff changeset
1086 def data(self, data):
21604
c399bf961cb9 bundle2: the ability to set ``data`` attribute of the part is now official
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21603
diff changeset
1087 if self._generated is not None:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1088 raise error.ReadOnlyPartError(b'part is being generated')
21604
c399bf961cb9 bundle2: the ability to set ``data`` attribute of the part is now official
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21603
diff changeset
1089 self._data = data
c399bf961cb9 bundle2: the ability to set ``data`` attribute of the part is now official
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21603
diff changeset
1090
21605
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
1091 @property
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
1092 def mandatoryparams(self):
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
1093 # make it an immutable tuple to force people through ``addparam``
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
1094 return tuple(self._mandatoryparams)
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
1095
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
1096 @property
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
1097 def advisoryparams(self):
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
1098 # make it an immutable tuple to force people through ``addparam``
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
1099 return tuple(self._advisoryparams)
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
1100
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1101 def addparam(self, name, value=b'', mandatory=True):
31861
6d055cd6669f bundle2: add documention to 'part.addparams'
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31647
diff changeset
1102 """add a parameter to the part
6d055cd6669f bundle2: add documention to 'part.addparams'
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31647
diff changeset
1103
6d055cd6669f bundle2: add documention to 'part.addparams'
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31647
diff changeset
1104 If 'mandatory' is set to True, the remote handler must claim support
6d055cd6669f bundle2: add documention to 'part.addparams'
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31647
diff changeset
1105 for this parameter or the unbundling will be aborted.
6d055cd6669f bundle2: add documention to 'part.addparams'
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31647
diff changeset
1106
6d055cd6669f bundle2: add documention to 'part.addparams'
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31647
diff changeset
1107 The 'name' and 'value' cannot exceed 255 bytes each.
6d055cd6669f bundle2: add documention to 'part.addparams'
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31647
diff changeset
1108 """
21605
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
1109 if self._generated is not None:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1110 raise error.ReadOnlyPartError(b'part is being generated')
21607
054fa5176fa7 bundle2: forbid duplicate parameter keys
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21606
diff changeset
1111 if name in self._seenparams:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1112 raise ValueError(b'duplicated params: %s' % name)
21607
054fa5176fa7 bundle2: forbid duplicate parameter keys
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21606
diff changeset
1113 self._seenparams.add(name)
21605
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
1114 params = self._advisoryparams
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
1115 if mandatory:
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
1116 params = self._mandatoryparams
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
1117 params.append((name, value))
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
1118
21601
7ff01befc7ec bundle2: track life cycle of parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21600
diff changeset
1119 # methods used to generates the bundle2 stream
25321
b44ee346211f bundle2: add debug output for part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25320
diff changeset
1120 def getchunks(self, ui):
21601
7ff01befc7ec bundle2: track life cycle of parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21600
diff changeset
1121 if self._generated is not None:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1122 raise error.ProgrammingError(b'part can only be consumed once')
21601
7ff01befc7ec bundle2: track life cycle of parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21600
diff changeset
1123 self._generated = False
25323
21a25fb81d2c bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25322
diff changeset
1124
21a25fb81d2c bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25322
diff changeset
1125 if ui.debugflag:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1126 msg = [b'bundle2-output-part: "%s"' % self.type]
25323
21a25fb81d2c bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25322
diff changeset
1127 if not self.mandatory:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1128 msg.append(b' (advisory)')
25323
21a25fb81d2c bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25322
diff changeset
1129 nbmp = len(self.mandatoryparams)
21a25fb81d2c bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25322
diff changeset
1130 nbap = len(self.advisoryparams)
21a25fb81d2c bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25322
diff changeset
1131 if nbmp or nbap:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1132 msg.append(b' (params:')
25323
21a25fb81d2c bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25322
diff changeset
1133 if nbmp:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1134 msg.append(b' %i mandatory' % nbmp)
25323
21a25fb81d2c bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25322
diff changeset
1135 if nbap:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1136 msg.append(b' %i advisory' % nbmp)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1137 msg.append(b')')
25323
21a25fb81d2c bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25322
diff changeset
1138 if not self.data:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1139 msg.append(b' empty payload')
43115
4aa72cdf616f py3: delete b'' prefix from safehasattr arguments
Martin von Zweigbergk <martinvonz@google.com>
parents: 43080
diff changeset
1140 elif util.safehasattr(self.data, 'next') or util.safehasattr(
50893
0e936b950731 bundle2: use sysstr to check for attribute presence
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50706
diff changeset
1141 self.data, '__next__'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1142 ):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1143 msg.append(b' streamed payload')
25323
21a25fb81d2c bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25322
diff changeset
1144 else:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1145 msg.append(b' %i bytes payload' % len(self.data))
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1146 msg.append(b'\n')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1147 ui.debug(b''.join(msg))
25323
21a25fb81d2c bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25322
diff changeset
1148
20877
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1149 #### header
23590
4440c7cc3728 bundle2.bundlepart: make mandatory part flag explicit in API
Eric Sumner <ericsumner@fb.com>
parents: 23586
diff changeset
1150 if self.mandatory:
4440c7cc3728 bundle2.bundlepart: make mandatory part flag explicit in API
Eric Sumner <ericsumner@fb.com>
parents: 23586
diff changeset
1151 parttype = self.type.upper()
4440c7cc3728 bundle2.bundlepart: make mandatory part flag explicit in API
Eric Sumner <ericsumner@fb.com>
parents: 23586
diff changeset
1152 else:
4440c7cc3728 bundle2.bundlepart: make mandatory part flag explicit in API
Eric Sumner <ericsumner@fb.com>
parents: 23586
diff changeset
1153 parttype = self.type.lower()
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1154 outdebug(ui, b'part %s: "%s"' % (pycompat.bytestr(self.id), parttype))
20877
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1155 ## parttype
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1156 header = [
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1157 _pack(_fparttypesize, len(parttype)),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1158 parttype,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1159 _pack(_fpartid, self.id),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1160 ]
20877
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1161 ## parameters
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1162 # count
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1163 manpar = self.mandatoryparams
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1164 advpar = self.advisoryparams
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1165 header.append(_pack(_fpartparamcount, len(manpar), len(advpar)))
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1166 # size
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1167 parsizes = []
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1168 for key, value in manpar:
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1169 parsizes.append(len(key))
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1170 parsizes.append(len(value))
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1171 for key, value in advpar:
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1172 parsizes.append(len(key))
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1173 parsizes.append(len(value))
33672
da7c285ec6da bundle2: use modern Python division
Augie Fackler <augie@google.com>
parents: 33665
diff changeset
1174 paramsizes = _pack(_makefpartparamsizes(len(parsizes) // 2), *parsizes)
20877
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1175 header.append(paramsizes)
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1176 # key, value
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1177 for key, value in manpar:
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1178 header.append(key)
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1179 header.append(value)
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1180 for key, value in advpar:
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1181 header.append(key)
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1182 header.append(value)
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1183 ## finalize header
34249
ab379eed2e31 bundle2: raise a more helpful error if building a bundle part header fails
Augie Fackler <raf@durin42.com>
parents: 34221
diff changeset
1184 try:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1185 headerchunk = b''.join(header)
34249
ab379eed2e31 bundle2: raise a more helpful error if building a bundle part header fails
Augie Fackler <raf@durin42.com>
parents: 34221
diff changeset
1186 except TypeError:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1187 raise TypeError(
43506
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43495
diff changeset
1188 'Found a non-bytes trying to '
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43495
diff changeset
1189 'build bundle part header: %r' % header
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1190 )
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1191 outdebug(ui, b'header chunk size: %i' % len(headerchunk))
20856
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
1192 yield _pack(_fpartheadersize, len(headerchunk))
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
1193 yield headerchunk
20877
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1194 ## payload
23067
420a051616ce bundle2: transmit exception during part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23066
diff changeset
1195 try:
420a051616ce bundle2: transmit exception during part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23066
diff changeset
1196 for chunk in self._payloadchunks():
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1197 outdebug(ui, b'payload chunk size: %i' % len(chunk))
23067
420a051616ce bundle2: transmit exception during part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23066
diff changeset
1198 yield _pack(_fpayloadsize, len(chunk))
420a051616ce bundle2: transmit exception during part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23066
diff changeset
1199 yield chunk
26144
4bc3707f3e67 bundle2: don't try to recover from a GeneratorExit (issue4785)
Augie Fackler <augie@google.com>
parents: 25919
diff changeset
1200 except GeneratorExit:
4bc3707f3e67 bundle2: don't try to recover from a GeneratorExit (issue4785)
Augie Fackler <augie@google.com>
parents: 25919
diff changeset
1201 # GeneratorExit means that nobody is listening for our
4bc3707f3e67 bundle2: don't try to recover from a GeneratorExit (issue4785)
Augie Fackler <augie@google.com>
parents: 25919
diff changeset
1202 # results anyway, so just bail quickly rather than trying
4bc3707f3e67 bundle2: don't try to recover from a GeneratorExit (issue4785)
Augie Fackler <augie@google.com>
parents: 25919
diff changeset
1203 # to produce an error part.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1204 ui.debug(b'bundle2-generatorexit\n')
26144
4bc3707f3e67 bundle2: don't try to recover from a GeneratorExit (issue4785)
Augie Fackler <augie@google.com>
parents: 25919
diff changeset
1205 raise
25660
328739ea70c3 global: mass rewrite to use modern exception syntax
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25641
diff changeset
1206 except BaseException as exc:
37084
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module
Yuya Nishihara <yuya@tcha.org>
parents: 37005
diff changeset
1207 bexc = stringutil.forcebytestr(exc)
23067
420a051616ce bundle2: transmit exception during part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23066
diff changeset
1208 # backup exception data for later
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1209 ui.debug(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1210 b'bundle2-input-stream-interrupt: encoding exception %s' % bexc
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1211 )
32186
76f9a0009b4b pycompat: extract helper to raise exception with traceback
Yuya Nishihara <yuya@tcha.org>
parents: 32024
diff changeset
1212 tb = sys.exc_info()[2]
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1213 msg = b'unexpected error: %s' % bexc
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1214 interpart = bundlepart(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1215 b'error:abort', [(b'message', msg)], mandatory=False
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1216 )
23067
420a051616ce bundle2: transmit exception during part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23066
diff changeset
1217 interpart.id = 0
420a051616ce bundle2: transmit exception during part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23066
diff changeset
1218 yield _pack(_fpayloadsize, -1)
25321
b44ee346211f bundle2: add debug output for part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25320
diff changeset
1219 for chunk in interpart.getchunks(ui=ui):
23067
420a051616ce bundle2: transmit exception during part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23066
diff changeset
1220 yield chunk
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1221 outdebug(ui, b'closing payload chunk')
23067
420a051616ce bundle2: transmit exception during part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23066
diff changeset
1222 # abort current part payload
420a051616ce bundle2: transmit exception during part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23066
diff changeset
1223 yield _pack(_fpayloadsize, 0)
32186
76f9a0009b4b pycompat: extract helper to raise exception with traceback
Yuya Nishihara <yuya@tcha.org>
parents: 32024
diff changeset
1224 pycompat.raisewithtb(exc, tb)
21000
4cae06ae1562 bundle2: extract a _payloadchunks method for part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20998
diff changeset
1225 # end of payload
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1226 outdebug(ui, b'closing payload chunk')
21000
4cae06ae1562 bundle2: extract a _payloadchunks method for part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20998
diff changeset
1227 yield _pack(_fpayloadsize, 0)
21601
7ff01befc7ec bundle2: track life cycle of parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21600
diff changeset
1228 self._generated = True
21000
4cae06ae1562 bundle2: extract a _payloadchunks method for part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20998
diff changeset
1229
4cae06ae1562 bundle2: extract a _payloadchunks method for part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20998
diff changeset
1230 def _payloadchunks(self):
4cae06ae1562 bundle2: extract a _payloadchunks method for part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20998
diff changeset
1231 """yield chunks of a the part payload
4cae06ae1562 bundle2: extract a _payloadchunks method for part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20998
diff changeset
1232
4cae06ae1562 bundle2: extract a _payloadchunks method for part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20998
diff changeset
1233 Exists to handle the different methods to provide data to a part."""
20876
ddd56f3eb786 bundle2: support for bundling and unbundling payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20864
diff changeset
1234 # we only support fixed size data now.
ddd56f3eb786 bundle2: support for bundling and unbundling payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20864
diff changeset
1235 # This will be improved in the future.
43115
4aa72cdf616f py3: delete b'' prefix from safehasattr arguments
Martin von Zweigbergk <martinvonz@google.com>
parents: 43080
diff changeset
1236 if util.safehasattr(self.data, 'next') or util.safehasattr(
50552
037890ecaff5 safehasattr: pass attribute name as string instead of bytes
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50533
diff changeset
1237 self.data, '__next__'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1238 ):
21001
c93bb6a08fa1 bundle2: support chunk iterator as part data
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21000
diff changeset
1239 buff = util.chunkbuffer(self.data)
c93bb6a08fa1 bundle2: support chunk iterator as part data
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21000
diff changeset
1240 chunk = buff.read(preferedchunksize)
c93bb6a08fa1 bundle2: support chunk iterator as part data
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21000
diff changeset
1241 while chunk:
c93bb6a08fa1 bundle2: support chunk iterator as part data
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21000
diff changeset
1242 yield chunk
c93bb6a08fa1 bundle2: support chunk iterator as part data
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21000
diff changeset
1243 chunk = buff.read(preferedchunksize)
c93bb6a08fa1 bundle2: support chunk iterator as part data
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21000
diff changeset
1244 elif len(self.data):
20876
ddd56f3eb786 bundle2: support for bundling and unbundling payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20864
diff changeset
1245 yield self.data
20802
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
1246
23066
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1247
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1248 flaginterrupt = -1
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1249
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1250
23066
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1251 class interrupthandler(unpackermixin):
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1252 """read one part and process it with restricted capability
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1253
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1254 This allows to transmit exception raised on the producer size during part
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1255 iteration while the consumer is reading a part.
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1256
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1257 Part processed in this manner only have access to a ui object,"""
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1258
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1259 def __init__(self, ui, fp):
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1260 super(interrupthandler, self).__init__(fp)
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1261 self.ui = ui
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1262
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1263 def _readpartheader(self):
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1264 """reads a part header size and return the bytes blob
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1265
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1266 returns None if empty"""
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1267 headersize = self._unpack(_fpartheadersize)[0]
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1268 if headersize < 0:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1269 raise error.BundleValueError(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1270 b'negative part header size: %i' % headersize
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1271 )
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1272 indebug(self.ui, b'part header size: %i\n' % headersize)
23066
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1273 if headersize:
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1274 return self._readexact(headersize)
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1275 return None
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1276
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1277 def __call__(self):
25335
8f7137a85a0e bundle2: add generic debug output regarding processed interruption
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25334
diff changeset
1278
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1279 self.ui.debug(
43117
8ff1ecfadcd1 cleanup: join string literals that are already on one line
Martin von Zweigbergk <martinvonz@google.com>
parents: 43115
diff changeset
1280 b'bundle2-input-stream-interrupt: opening out of band context\n'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1281 )
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1282 indebug(self.ui, b'bundle2 stream interruption, looking for a part.')
23066
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1283 headerblock = self._readpartheader()
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1284 if headerblock is None:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1285 indebug(self.ui, b'no part found during interruption.')
23066
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1286 return
35116
da91e7309daf bundle2: don't use seekable bundle2 parts by default (issue5691)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35115
diff changeset
1287 part = unbundlepart(self.ui, headerblock, self._fp)
23066
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1288 op = interruptoperation(self.ui)
34258
e71890f27767 bundle2: move processpart stream maintenance into part iterator
Durham Goode <durham@fb.com>
parents: 34249
diff changeset
1289 hardabort = False
e71890f27767 bundle2: move processpart stream maintenance into part iterator
Durham Goode <durham@fb.com>
parents: 34249
diff changeset
1290 try:
e71890f27767 bundle2: move processpart stream maintenance into part iterator
Durham Goode <durham@fb.com>
parents: 34249
diff changeset
1291 _processpart(op, part)
e71890f27767 bundle2: move processpart stream maintenance into part iterator
Durham Goode <durham@fb.com>
parents: 34249
diff changeset
1292 except (SystemExit, KeyboardInterrupt):
e71890f27767 bundle2: move processpart stream maintenance into part iterator
Durham Goode <durham@fb.com>
parents: 34249
diff changeset
1293 hardabort = True
e71890f27767 bundle2: move processpart stream maintenance into part iterator
Durham Goode <durham@fb.com>
parents: 34249
diff changeset
1294 raise
e71890f27767 bundle2: move processpart stream maintenance into part iterator
Durham Goode <durham@fb.com>
parents: 34249
diff changeset
1295 finally:
e71890f27767 bundle2: move processpart stream maintenance into part iterator
Durham Goode <durham@fb.com>
parents: 34249
diff changeset
1296 if not hardabort:
35114
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35113
diff changeset
1297 part.consume()
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1298 self.ui.debug(
43117
8ff1ecfadcd1 cleanup: join string literals that are already on one line
Martin von Zweigbergk <martinvonz@google.com>
parents: 43115
diff changeset
1299 b'bundle2-input-stream-interrupt: closing out of band context\n'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1300 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1301
23066
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1302
48946
642e31cb55f0 py3: use class X: instead of class X(object):
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48913
diff changeset
1303 class interruptoperation:
23066
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1304 """A limited operation to be use by part handler during interruption
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1305
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1306 It only have access to an ui object.
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1307 """
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1308
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1309 def __init__(self, ui):
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1310 self.ui = ui
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1311 self.reply = None
24878
e530cde6d115 bundle2: disable ouput capture unless we use http (issue4613 issue4615)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24851
diff changeset
1312 self.captureoutput = False
23066
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1313
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1314 @property
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1315 def repo(self):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1316 raise error.ProgrammingError(b'no repo access from stream interruption')
23066
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1317
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1318 def gettransaction(self):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1319 raise TransactionUnavailable(b'no repo access from stream interruption')
23066
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1320
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1321
35113
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35112
diff changeset
1322 def decodepayloadchunks(ui, fh):
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35112
diff changeset
1323 """Reads bundle2 part payload data into chunks.
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35112
diff changeset
1324
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35112
diff changeset
1325 Part payload data consists of framed chunks. This function takes
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35112
diff changeset
1326 a file handle and emits those chunks.
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35112
diff changeset
1327 """
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1328 dolog = ui.configbool(b'devel', b'bundle2.debug')
35117
589eed45a452 bundle2: inline debug logging
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35116
diff changeset
1329 debug = ui.debug
589eed45a452 bundle2: inline debug logging
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35116
diff changeset
1330
35119
764e3ad1cf54 bundle2: inline struct operations
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35118
diff changeset
1331 headerstruct = struct.Struct(_fpayloadsize)
764e3ad1cf54 bundle2: inline struct operations
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35118
diff changeset
1332 headersize = headerstruct.size
764e3ad1cf54 bundle2: inline struct operations
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35118
diff changeset
1333 unpack = headerstruct.unpack
764e3ad1cf54 bundle2: inline struct operations
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35118
diff changeset
1334
35113
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35112
diff changeset
1335 readexactly = changegroup.readexactly
35118
1fb0846ad792 bundle2: inline changegroup.readexactly()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35117
diff changeset
1336 read = fh.read
35113
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35112
diff changeset
1337
35119
764e3ad1cf54 bundle2: inline struct operations
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35118
diff changeset
1338 chunksize = unpack(readexactly(fh, headersize))[0]
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1339 indebug(ui, b'payload chunk size: %i' % chunksize)
35113
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35112
diff changeset
1340
35118
1fb0846ad792 bundle2: inline changegroup.readexactly()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35117
diff changeset
1341 # changegroup.readexactly() is inlined below for performance.
35113
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35112
diff changeset
1342 while chunksize:
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35112
diff changeset
1343 if chunksize >= 0:
35118
1fb0846ad792 bundle2: inline changegroup.readexactly()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35117
diff changeset
1344 s = read(chunksize)
1fb0846ad792 bundle2: inline changegroup.readexactly()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35117
diff changeset
1345 if len(s) < chunksize:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1346 raise error.Abort(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1347 _(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1348 b'stream ended unexpectedly '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1349 b' (got %d bytes, expected %d)'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1350 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1351 % (len(s), chunksize)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1352 )
35118
1fb0846ad792 bundle2: inline changegroup.readexactly()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35117
diff changeset
1353
1fb0846ad792 bundle2: inline changegroup.readexactly()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35117
diff changeset
1354 yield s
35113
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35112
diff changeset
1355 elif chunksize == flaginterrupt:
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35112
diff changeset
1356 # Interrupt "signal" detected. The regular stream is interrupted
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35112
diff changeset
1357 # and a bundle2 part follows. Consume it.
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35112
diff changeset
1358 interrupthandler(ui, fh)()
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35112
diff changeset
1359 else:
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35112
diff changeset
1360 raise error.BundleValueError(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1361 b'negative payload chunk size: %s' % chunksize
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1362 )
35113
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35112
diff changeset
1363
35118
1fb0846ad792 bundle2: inline changegroup.readexactly()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35117
diff changeset
1364 s = read(headersize)
1fb0846ad792 bundle2: inline changegroup.readexactly()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35117
diff changeset
1365 if len(s) < headersize:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1366 raise error.Abort(
43117
8ff1ecfadcd1 cleanup: join string literals that are already on one line
Martin von Zweigbergk <martinvonz@google.com>
parents: 43115
diff changeset
1367 _(b'stream ended unexpectedly (got %d bytes, expected %d)')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1368 % (len(s), chunksize)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1369 )
35118
1fb0846ad792 bundle2: inline changegroup.readexactly()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35117
diff changeset
1370
35119
764e3ad1cf54 bundle2: inline struct operations
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35118
diff changeset
1371 chunksize = unpack(s)[0]
35117
589eed45a452 bundle2: inline debug logging
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35116
diff changeset
1372
589eed45a452 bundle2: inline debug logging
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35116
diff changeset
1373 # indebug() inlined for performance.
589eed45a452 bundle2: inline debug logging
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35116
diff changeset
1374 if dolog:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1375 debug(b'bundle2-input: payload chunk size: %i\n' % chunksize)
35113
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35112
diff changeset
1376
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1377
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1378 class unbundlepart(unpackermixin):
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1379 """a bundle part read from a bundle"""
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1380
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1381 def __init__(self, ui, header, fp):
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1382 super(unbundlepart, self).__init__(fp)
43115
4aa72cdf616f py3: delete b'' prefix from safehasattr arguments
Martin von Zweigbergk <martinvonz@google.com>
parents: 43080
diff changeset
1383 self._seekable = util.safehasattr(fp, 'seek') and util.safehasattr(
50616
7200a9d4ea04 safehasattr: pass attribute name as string instead of bytes
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50552
diff changeset
1384 fp, 'tell'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1385 )
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1386 self.ui = ui
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1387 # unbundle state attr
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1388 self._headerdata = header
21015
14dd49260246 bundle2: move the fromheader closure into the class itself
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21014
diff changeset
1389 self._headeroffset = 0
21019
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
1390 self._initialized = False
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
1391 self.consumed = False
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1392 # part data
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1393 self.id = None
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1394 self.type = None
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1395 self.mandatoryparams = None
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1396 self.advisoryparams = None
21610
d6056805f8f4 bundle2: introduce a ``params`` dictionary on unbundled parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21609
diff changeset
1397 self.params = None
21612
f221eb0531d9 bundle2: expose mandatory params in a mandatorykeys attribute
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21611
diff changeset
1398 self.mandatorykeys = ()
21019
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
1399 self._readheader()
23585
94b25d71dd0f bundle2.unbundlepart: decouple mandatory from parttype
Eric Sumner <ericsumner@fb.com>
parents: 23439
diff changeset
1400 self._mandatory = None
24036
c7601086338a bundle2.unbundlepart: tell() implementation
Eric Sumner <ericsumner@fb.com>
parents: 24035
diff changeset
1401 self._pos = 0
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1402
21015
14dd49260246 bundle2: move the fromheader closure into the class itself
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21014
diff changeset
1403 def _fromheader(self, size):
14dd49260246 bundle2: move the fromheader closure into the class itself
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21014
diff changeset
1404 """return the next <size> byte from the header"""
14dd49260246 bundle2: move the fromheader closure into the class itself
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21014
diff changeset
1405 offset = self._headeroffset
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1406 data = self._headerdata[offset : (offset + size)]
21019
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
1407 self._headeroffset = offset + size
21015
14dd49260246 bundle2: move the fromheader closure into the class itself
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21014
diff changeset
1408 return data
14dd49260246 bundle2: move the fromheader closure into the class itself
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21014
diff changeset
1409
21016
b477afb1c81e bundle2: move unpackheader closure into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21015
diff changeset
1410 def _unpackheader(self, format):
b477afb1c81e bundle2: move unpackheader closure into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21015
diff changeset
1411 """read given format from header
b477afb1c81e bundle2: move unpackheader closure into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21015
diff changeset
1412
b477afb1c81e bundle2: move unpackheader closure into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21015
diff changeset
1413 This automatically compute the size of the format to read."""
b477afb1c81e bundle2: move unpackheader closure into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21015
diff changeset
1414 data = self._fromheader(struct.calcsize(format))
b477afb1c81e bundle2: move unpackheader closure into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21015
diff changeset
1415 return _unpack(format, data)
b477afb1c81e bundle2: move unpackheader closure into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21015
diff changeset
1416
21608
3cb96ca90c17 bundle2: introduce an ``_initparams`` method
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21607
diff changeset
1417 def _initparams(self, mandatoryparams, advisoryparams):
3cb96ca90c17 bundle2: introduce an ``_initparams`` method
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21607
diff changeset
1418 """internal function to setup all logic related parameters"""
21609
63cc2594ef8a bundle2: make sure unbundled part param are read-only
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21608
diff changeset
1419 # make it read only to prevent people touching it by mistake.
63cc2594ef8a bundle2: make sure unbundled part param are read-only
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21608
diff changeset
1420 self.mandatoryparams = tuple(mandatoryparams)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1421 self.advisoryparams = tuple(advisoryparams)
21610
d6056805f8f4 bundle2: introduce a ``params`` dictionary on unbundled parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21609
diff changeset
1422 # user friendly UI
29591
6215b5537ba5 bundle2: use a sorted dict for holding parameters
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29294
diff changeset
1423 self.params = util.sortdict(self.mandatoryparams)
6215b5537ba5 bundle2: use a sorted dict for holding parameters
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29294
diff changeset
1424 self.params.update(self.advisoryparams)
21612
f221eb0531d9 bundle2: expose mandatory params in a mandatorykeys attribute
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21611
diff changeset
1425 self.mandatorykeys = frozenset(p[0] for p in mandatoryparams)
21608
3cb96ca90c17 bundle2: introduce an ``_initparams`` method
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21607
diff changeset
1426
21019
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
1427 def _readheader(self):
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1428 """read the header and setup the object"""
21016
b477afb1c81e bundle2: move unpackheader closure into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21015
diff changeset
1429 typesize = self._unpackheader(_fparttypesize)[0]
21015
14dd49260246 bundle2: move the fromheader closure into the class itself
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21014
diff changeset
1430 self.type = self._fromheader(typesize)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1431 indebug(self.ui, b'part type: "%s"' % self.type)
21016
b477afb1c81e bundle2: move unpackheader closure into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21015
diff changeset
1432 self.id = self._unpackheader(_fpartid)[0]
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1433 indebug(self.ui, b'part id: "%s"' % pycompat.bytestr(self.id))
23585
94b25d71dd0f bundle2.unbundlepart: decouple mandatory from parttype
Eric Sumner <ericsumner@fb.com>
parents: 23439
diff changeset
1434 # extract mandatory bit from type
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1435 self.mandatory = self.type != self.type.lower()
23585
94b25d71dd0f bundle2.unbundlepart: decouple mandatory from parttype
Eric Sumner <ericsumner@fb.com>
parents: 23439
diff changeset
1436 self.type = self.type.lower()
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1437 ## reading parameters
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1438 # param count
21016
b477afb1c81e bundle2: move unpackheader closure into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21015
diff changeset
1439 mancount, advcount = self._unpackheader(_fpartparamcount)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1440 indebug(self.ui, b'part parameters: %i' % (mancount + advcount))
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1441 # param size
21016
b477afb1c81e bundle2: move unpackheader closure into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21015
diff changeset
1442 fparamsizes = _makefpartparamsizes(mancount + advcount)
b477afb1c81e bundle2: move unpackheader closure into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21015
diff changeset
1443 paramsizes = self._unpackheader(fparamsizes)
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1444 # make it a list of couple again
33674
9a323773216c bundle2: work around zip() being lazy in Python 3
Augie Fackler <augie@google.com>
parents: 33673
diff changeset
1445 paramsizes = list(zip(paramsizes[::2], paramsizes[1::2]))
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1446 # split mandatory from advisory
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1447 mansizes = paramsizes[:mancount]
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1448 advsizes = paramsizes[mancount:]
23139
e53f6b72a0e4 spelling: fixes from proofreading of spell checker issues
Mads Kiilerich <madski@unity3d.com>
parents: 23067
diff changeset
1449 # retrieve param value
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1450 manparams = []
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1451 for key, value in mansizes:
21015
14dd49260246 bundle2: move the fromheader closure into the class itself
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21014
diff changeset
1452 manparams.append((self._fromheader(key), self._fromheader(value)))
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1453 advparams = []
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1454 for key, value in advsizes:
21015
14dd49260246 bundle2: move the fromheader closure into the class itself
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21014
diff changeset
1455 advparams.append((self._fromheader(key), self._fromheader(value)))
21608
3cb96ca90c17 bundle2: introduce an ``_initparams`` method
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21607
diff changeset
1456 self._initparams(manparams, advparams)
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1457 ## part payload
24034
9881a1437799 bundle2.unbundlepart: raise payloadchunks from a closure to a method
Eric Sumner <ericsumner@fb.com>
parents: 24026
diff changeset
1458 self._payloadstream = util.chunkbuffer(self._payloadchunks())
21019
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
1459 # we read the data, tell it
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
1460 self._initialized = True
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
1461
35113
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35112
diff changeset
1462 def _payloadchunks(self):
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35112
diff changeset
1463 """Generator of decoded chunks in the payload."""
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35112
diff changeset
1464 return decodepayloadchunks(self.ui, self._fp)
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35112
diff changeset
1465
35114
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35113
diff changeset
1466 def consume(self):
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35113
diff changeset
1467 """Read the part payload until completion.
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35113
diff changeset
1468
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35113
diff changeset
1469 By consuming the part data, the underlying stream read offset will
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35113
diff changeset
1470 be advanced to the next part (or end of stream).
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35113
diff changeset
1471 """
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35113
diff changeset
1472 if self.consumed:
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35113
diff changeset
1473 return
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35113
diff changeset
1474
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35113
diff changeset
1475 chunk = self.read(32768)
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35113
diff changeset
1476 while chunk:
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35113
diff changeset
1477 self._pos += len(chunk)
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35113
diff changeset
1478 chunk = self.read(32768)
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35113
diff changeset
1479
21019
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
1480 def read(self, size=None):
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
1481 """read payload data"""
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
1482 if not self._initialized:
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
1483 self._readheader()
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
1484 if size is None:
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
1485 data = self._payloadstream.read()
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
1486 else:
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
1487 data = self._payloadstream.read(size)
25334
76cba1ecf55f bundle2: add generic debug output regarding processed part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25333
diff changeset
1488 self._pos += len(data)
21019
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
1489 if size is None or len(data) < size:
25334
76cba1ecf55f bundle2: add generic debug output regarding processed part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25333
diff changeset
1490 if not self.consumed and self._pos:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1491 self.ui.debug(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1492 b'bundle2-input-part: total payload size %i\n' % self._pos
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1493 )
21019
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
1494 self.consumed = True
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
1495 return data
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
1496
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1497
35112
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1498 class seekableunbundlepart(unbundlepart):
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1499 """A bundle2 part in a bundle that is seekable.
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1500
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1501 Regular ``unbundlepart`` instances can only be read once. This class
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1502 extends ``unbundlepart`` to enable bi-directional seeking within the
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1503 part.
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1504
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1505 Bundle2 part data consists of framed chunks. Offsets when seeking
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1506 refer to the decoded data, not the offsets in the underlying bundle2
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1507 stream.
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1508
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1509 To facilitate quickly seeking within the decoded data, instances of this
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1510 class maintain a mapping between offsets in the underlying stream and
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1511 the decoded payload. This mapping will consume memory in proportion
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1512 to the number of chunks within the payload (which almost certainly
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1513 increases in proportion with the size of the part).
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1514 """
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1515
35112
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1516 def __init__(self, ui, header, fp):
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1517 # (payload, file) offsets for chunk starts.
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1518 self._chunkindex = []
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1519
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1520 super(seekableunbundlepart, self).__init__(ui, header, fp)
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1521
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1522 def _payloadchunks(self, chunknum=0):
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1523 '''seek to specified chunk and start yielding data'''
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1524 if len(self._chunkindex) == 0:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1525 assert chunknum == 0, b'Must start with chunk 0'
35112
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1526 self._chunkindex.append((0, self._tellfp()))
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1527 else:
41759
aaad36b88298 cleanup: use () to wrap long lines instead of \
Augie Fackler <augie@google.com>
parents: 40429
diff changeset
1528 assert chunknum < len(self._chunkindex), (
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1529 b'Unknown chunk %d' % chunknum
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1530 )
35112
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1531 self._seekfp(self._chunkindex[chunknum][1])
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1532
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1533 pos = self._chunkindex[chunknum][0]
35113
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35112
diff changeset
1534
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35112
diff changeset
1535 for chunk in decodepayloadchunks(self.ui, self._fp):
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35112
diff changeset
1536 chunknum += 1
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35112
diff changeset
1537 pos += len(chunk)
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35112
diff changeset
1538 if chunknum == len(self._chunkindex):
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35112
diff changeset
1539 self._chunkindex.append((pos, self._tellfp()))
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35112
diff changeset
1540
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35112
diff changeset
1541 yield chunk
35112
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1542
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1543 def _findchunk(self, pos):
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1544 '''for a given payload position, return a chunk number and offset'''
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1545 for chunk, (ppos, fpos) in enumerate(self._chunkindex):
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1546 if ppos == pos:
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1547 return chunk, 0
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1548 elif ppos > pos:
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1549 return chunk - 1, pos - self._chunkindex[chunk - 1][0]
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1550 raise ValueError(b'Unknown chunk')
35112
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35046
diff changeset
1551
24036
c7601086338a bundle2.unbundlepart: tell() implementation
Eric Sumner <ericsumner@fb.com>
parents: 24035
diff changeset
1552 def tell(self):
c7601086338a bundle2.unbundlepart: tell() implementation
Eric Sumner <ericsumner@fb.com>
parents: 24035
diff changeset
1553 return self._pos
c7601086338a bundle2.unbundlepart: tell() implementation
Eric Sumner <ericsumner@fb.com>
parents: 24035
diff changeset
1554
35046
241d9caca11e bundle2: use os.SEEK_* constants
Gregory Szorc <gregory.szorc@gmail.com>
parents: 34820
diff changeset
1555 def seek(self, offset, whence=os.SEEK_SET):
241d9caca11e bundle2: use os.SEEK_* constants
Gregory Szorc <gregory.szorc@gmail.com>
parents: 34820
diff changeset
1556 if whence == os.SEEK_SET:
24037
f0b498cfc5c8 bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents: 24036
diff changeset
1557 newpos = offset
35046
241d9caca11e bundle2: use os.SEEK_* constants
Gregory Szorc <gregory.szorc@gmail.com>
parents: 34820
diff changeset
1558 elif whence == os.SEEK_CUR:
24037
f0b498cfc5c8 bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents: 24036
diff changeset
1559 newpos = self._pos + offset
35046
241d9caca11e bundle2: use os.SEEK_* constants
Gregory Szorc <gregory.szorc@gmail.com>
parents: 34820
diff changeset
1560 elif whence == os.SEEK_END:
24037
f0b498cfc5c8 bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents: 24036
diff changeset
1561 if not self.consumed:
35120
699b2a759319 bundle2: avoid unbound read when seeking
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35119
diff changeset
1562 # Can't use self.consume() here because it advances self._pos.
699b2a759319 bundle2: avoid unbound read when seeking
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35119
diff changeset
1563 chunk = self.read(32768)
699b2a759319 bundle2: avoid unbound read when seeking
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35119
diff changeset
1564 while chunk:
699b2a759319 bundle2: avoid unbound read when seeking
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35119
diff changeset
1565 chunk = self.read(32768)
24037
f0b498cfc5c8 bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents: 24036
diff changeset
1566 newpos = self._chunkindex[-1][0] - offset
f0b498cfc5c8 bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents: 24036
diff changeset
1567 else:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1568 raise ValueError(b'Unknown whence value: %r' % (whence,))
24037
f0b498cfc5c8 bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents: 24036
diff changeset
1569
f0b498cfc5c8 bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents: 24036
diff changeset
1570 if newpos > self._chunkindex[-1][0] and not self.consumed:
35120
699b2a759319 bundle2: avoid unbound read when seeking
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35119
diff changeset
1571 # Can't use self.consume() here because it advances self._pos.
699b2a759319 bundle2: avoid unbound read when seeking
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35119
diff changeset
1572 chunk = self.read(32768)
699b2a759319 bundle2: avoid unbound read when seeking
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35119
diff changeset
1573 while chunk:
699b2a759319 bundle2: avoid unbound read when seeking
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35119
diff changeset
1574 chunk = self.read(32668)
699b2a759319 bundle2: avoid unbound read when seeking
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35119
diff changeset
1575
24037
f0b498cfc5c8 bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents: 24036
diff changeset
1576 if not 0 <= newpos <= self._chunkindex[-1][0]:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1577 raise ValueError(b'Offset out of range')
24037
f0b498cfc5c8 bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents: 24036
diff changeset
1578
f0b498cfc5c8 bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents: 24036
diff changeset
1579 if self._pos != newpos:
f0b498cfc5c8 bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents: 24036
diff changeset
1580 chunk, internaloffset = self._findchunk(newpos)
f0b498cfc5c8 bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents: 24036
diff changeset
1581 self._payloadstream = util.chunkbuffer(self._payloadchunks(chunk))
f0b498cfc5c8 bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents: 24036
diff changeset
1582 adjust = self.read(internaloffset)
f0b498cfc5c8 bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents: 24036
diff changeset
1583 if len(adjust) != internaloffset:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1584 raise error.Abort(_(b'Seek failed\n'))
24037
f0b498cfc5c8 bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents: 24036
diff changeset
1585 self._pos = newpos
f0b498cfc5c8 bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents: 24036
diff changeset
1586
31889
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1587 def _seekfp(self, offset, whence=0):
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1588 """move the underlying file pointer
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1589
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1590 This method is meant for internal usage by the bundle2 protocol only.
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1591 They directly manipulate the low level stream including bundle2 level
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1592 instruction.
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1593
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1594 Do not use it to implement higher-level logic or methods."""
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1595 if self._seekable:
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1596 return self._fp.seek(offset, whence)
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1597 else:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1598 raise NotImplementedError(_(b'File pointer is not seekable'))
31889
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1599
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1600 def _tellfp(self):
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1601 """return the file offset, or None if file is not seekable
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1602
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1603 This method is meant for internal usage by the bundle2 protocol only.
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1604 They directly manipulate the low level stream including bundle2 level
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1605 instruction.
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1606
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1607 Do not use it to implement higher-level logic or methods."""
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1608 if self._seekable:
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1609 try:
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1610 return self._fp.tell()
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1611 except IOError as e:
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1612 if e.errno == errno.ESPIPE:
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1613 self._seekable = False
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1614 else:
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1615 raise
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1616 return None
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1617
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1618
25317
5a5b7046d00e bundle2: add an informative comment to the capability dict
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25315
diff changeset
1619 # These are only the static capabilities.
5a5b7046d00e bundle2: add an informative comment to the capability dict
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25315
diff changeset
1620 # Check the 'getrepocaps' function for the rest.
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1621 capabilities = {
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1622 b'HG20': (),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1623 b'bookmarks': (),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1624 b'error': (b'abort', b'unsupportedcontent', b'pushraced', b'pushkey'),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1625 b'listkeys': (),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1626 b'pushkey': (),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1627 b'digests': tuple(sorted(util.DIGESTS.keys())),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1628 b'remote-changegroup': (b'http', b'https'),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1629 b'hgtagsfnodes': (),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1630 b'phases': (b'heads',),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1631 b'stream': (b'v2',),
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1632 }
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1633
22341
2d16b39601b5 obsmarker: move bundle2caps from the localrepo class to the bundle2 module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22340
diff changeset
1634
35783
c97639ad6874 bundle2: specify what capabilities will be used for
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35763
diff changeset
1635 def getrepocaps(repo, allowpushback=False, role=None):
22342
262c5cc126c1 bundle2: introduce a `getrepocaps` to retrieve the bundle2 caps of a repo
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22341
diff changeset
1636 """return the bundle2 capabilities for a given repo
262c5cc126c1 bundle2: introduce a `getrepocaps` to retrieve the bundle2 caps of a repo
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22341
diff changeset
1637
22343
76677a2c1cfd bundle2: advertise the obsmarker part in bundle2 capabilities
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22342
diff changeset
1638 Exists to allow extensions (like evolution) to mutate the capabilities.
35783
c97639ad6874 bundle2: specify what capabilities will be used for
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35763
diff changeset
1639
c97639ad6874 bundle2: specify what capabilities will be used for
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35763
diff changeset
1640 The returned value is used for servers advertising their capabilities as
c97639ad6874 bundle2: specify what capabilities will be used for
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35763
diff changeset
1641 well as clients advertising their capabilities to servers as part of
c97639ad6874 bundle2: specify what capabilities will be used for
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35763
diff changeset
1642 bundle2 requests. The ``role`` argument specifies which is which.
22342
262c5cc126c1 bundle2: introduce a `getrepocaps` to retrieve the bundle2 caps of a repo
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22341
diff changeset
1643 """
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1644 if role not in (b'client', b'server'):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1645 raise error.ProgrammingError(b'role argument must be client or server')
35783
c97639ad6874 bundle2: specify what capabilities will be used for
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35763
diff changeset
1646
22343
76677a2c1cfd bundle2: advertise the obsmarker part in bundle2 capabilities
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22342
diff changeset
1647 caps = capabilities.copy()
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1648 caps[b'changegroup'] = tuple(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1649 sorted(changegroup.supportedincomingversions(repo))
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1650 )
22953
b1d694d3975e obsolete: add exchange option
Durham Goode <durham@fb.com>
parents: 22661
diff changeset
1651 if obsolete.isenabled(repo, obsolete.exchangeopt):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1652 supportedformat = tuple(b'V%i' % v for v in obsolete.formats)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1653 caps[b'obsmarkers'] = supportedformat
23439
743736fc7c41 bundle2-push: provide transaction to reply unbundler
Eric Sumner <ericsumner@fb.com>
parents: 23438
diff changeset
1654 if allowpushback:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1655 caps[b'pushback'] = ()
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1656 cpmode = repo.ui.config(b'server', b'concurrent-push-mode')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1657 if cpmode == b'check-related':
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1658 caps[b'checkheads'] = (b'related',)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1659 if b'phases' in repo.ui.configlist(b'devel', b'legacy.exchange'):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1660 caps.pop(b'phases')
35790
c1b9eb15a51c bundle2: don't advertise stream bundle2 capability when feature disabled
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35788
diff changeset
1661
c1b9eb15a51c bundle2: don't advertise stream bundle2 capability when feature disabled
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35788
diff changeset
1662 # Don't advertise stream clone support in server mode if not configured.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1663 if role == b'server':
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1664 streamsupported = repo.ui.configbool(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1665 b'server', b'uncompressed', untrusted=True
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1666 )
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1667 featuresupported = repo.ui.configbool(b'server', b'bundle2.stream')
35790
c1b9eb15a51c bundle2: don't advertise stream bundle2 capability when feature disabled
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35788
diff changeset
1668
c1b9eb15a51c bundle2: don't advertise stream bundle2 capability when feature disabled
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35788
diff changeset
1669 if not streamsupported or not featuresupported:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1670 caps.pop(b'stream')
35792
1d118f9f4f57 bundle2: always advertise client support for stream parts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35790
diff changeset
1671 # Else always advertise support on client, because payload support
1d118f9f4f57 bundle2: always advertise client support for stream parts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35790
diff changeset
1672 # should always be advertised.
35790
c1b9eb15a51c bundle2: don't advertise stream bundle2 capability when feature disabled
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35788
diff changeset
1673
50524
58adcabc295f stream-clone: introduce the notion of an experimental "v3" version
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50519
diff changeset
1674 if repo.ui.configbool(b'experimental', b'stream-v3'):
58adcabc295f stream-clone: introduce the notion of an experimental "v3" version
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50519
diff changeset
1675 if b'stream' in caps:
58adcabc295f stream-clone: introduce the notion of an experimental "v3" version
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50519
diff changeset
1676 caps[b'stream'] += (b'v3-exp',)
58adcabc295f stream-clone: introduce the notion of an experimental "v3" version
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50519
diff changeset
1677
46633
7015b0232c5e exchange: stop advertising rev-branch-cache bundle capability
Joerg Sonnenberger <joerg@bec.de>
parents: 46519
diff changeset
1678 # b'rev-branch-cache is no longer advertised, but still supported
7015b0232c5e exchange: stop advertising rev-branch-cache bundle capability
Joerg Sonnenberger <joerg@bec.de>
parents: 46519
diff changeset
1679 # for legacy clients.
7015b0232c5e exchange: stop advertising rev-branch-cache bundle capability
Joerg Sonnenberger <joerg@bec.de>
parents: 46519
diff changeset
1680
22343
76677a2c1cfd bundle2: advertise the obsmarker part in bundle2 capabilities
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22342
diff changeset
1681 return caps
22342
262c5cc126c1 bundle2: introduce a `getrepocaps` to retrieve the bundle2 caps of a repo
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22341
diff changeset
1682
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1683
21644
17755dd8c509 bundle2: introduce a bundle2caps function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21628
diff changeset
1684 def bundle2caps(remote):
23139
e53f6b72a0e4 spelling: fixes from proofreading of spell checker issues
Mads Kiilerich <madski@unity3d.com>
parents: 23067
diff changeset
1685 """return the bundle capabilities of a peer as dict"""
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1686 raw = remote.capable(b'bundle2')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1687 if not raw and raw != b'':
21644
17755dd8c509 bundle2: introduce a bundle2caps function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21628
diff changeset
1688 return {}
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1689 capsblob = urlreq.unquote(remote.capable(b'bundle2'))
21644
17755dd8c509 bundle2: introduce a bundle2caps function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21628
diff changeset
1690 return decodecaps(capsblob)
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1691
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1692
22344
9829b7948100 bundle2: add a `obsmarkersversion` function to extract supported version
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22343
diff changeset
1693 def obsmarkersversion(caps):
45942
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45682
diff changeset
1694 """extract the list of supported obsmarkers versions from a bundle2caps dict"""
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1695 obscaps = caps.get(b'obsmarkers', ())
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1696 return [int(c[1:]) for c in obscaps if c.startswith(b'V')]
22344
9829b7948100 bundle2: add a `obsmarkersversion` function to extract supported version
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22343
diff changeset
1697
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1698
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1699 def writenewbundle(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1700 ui,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1701 repo,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1702 source,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1703 filename,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1704 bundletype,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1705 outgoing,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1706 opts,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1707 vfs=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1708 compression=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1709 compopts=None,
50399
bcf54837241d bundle: prevent implicite bundling of internal changeset
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50392
diff changeset
1710 allow_internal=False,
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1711 ):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1712 if bundletype.startswith(b'HG10'):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1713 cg = changegroup.makechangegroup(repo, outgoing, b'01', source)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1714 return writebundle(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1715 ui,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1716 cg,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1717 filename,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1718 bundletype,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1719 vfs=vfs,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1720 compression=compression,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1721 compopts=compopts,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1722 )
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1723 elif not bundletype.startswith(b'HG20'):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1724 raise error.ProgrammingError(b'unknown bundle type: %s' % bundletype)
32221
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32186
diff changeset
1725
50399
bcf54837241d bundle: prevent implicite bundling of internal changeset
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50392
diff changeset
1726 # enforce that no internal phase are to be bundled
bcf54837241d bundle: prevent implicite bundling of internal changeset
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50392
diff changeset
1727 bundled_internal = repo.revs(b"%ln and _internal()", outgoing.ancestorsof)
bcf54837241d bundle: prevent implicite bundling of internal changeset
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50392
diff changeset
1728 if bundled_internal and not allow_internal:
bcf54837241d bundle: prevent implicite bundling of internal changeset
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50392
diff changeset
1729 count = len(repo.revs(b'%ln and _internal()', outgoing.missing))
bcf54837241d bundle: prevent implicite bundling of internal changeset
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50392
diff changeset
1730 msg = "backup bundle would contains %d internal changesets"
bcf54837241d bundle: prevent implicite bundling of internal changeset
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50392
diff changeset
1731 msg %= count
bcf54837241d bundle: prevent implicite bundling of internal changeset
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50392
diff changeset
1732 raise error.ProgrammingError(msg)
bcf54837241d bundle: prevent implicite bundling of internal changeset
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50392
diff changeset
1733
32516
37d70ba1d9d1 bundle: add an experimental knob to include obsmarkers in bundle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32515
diff changeset
1734 caps = {}
49336
290c29df1915 bundlespec: check the `obsolescence` value before adding the caps
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48992
diff changeset
1735 if opts.get(b'obsolescence', False):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1736 caps[b'obsmarkers'] = (b'V1',)
50518
f2bcb56a1d39 stream-clone: make sure the `stream` capability is set when bundling
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50517
diff changeset
1737 if opts.get(b'streamv2'):
f2bcb56a1d39 stream-clone: make sure the `stream` capability is set when bundling
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50517
diff changeset
1738 caps[b'stream'] = [b'v2']
50533
f4a540c203d7 stream-clone: support streamv3 on the cli [hg bundle]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 50532
diff changeset
1739 elif opts.get(b'streamv3-exp'):
f4a540c203d7 stream-clone: support streamv3 on the cli [hg bundle]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 50532
diff changeset
1740 caps[b'stream'] = [b'v3-exp']
32516
37d70ba1d9d1 bundle: add an experimental knob to include obsmarkers in bundle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32515
diff changeset
1741 bundle = bundle20(ui, caps)
32221
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32186
diff changeset
1742 bundle.setcompression(compression, compopts)
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32186
diff changeset
1743 _addpartsfromopts(ui, repo, bundle, source, outgoing, opts)
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32186
diff changeset
1744 chunkiter = bundle.getchunks()
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32186
diff changeset
1745
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32186
diff changeset
1746 return changegroup.writechunks(ui, chunkiter, filename, vfs=vfs)
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32186
diff changeset
1747
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1748
32221
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32186
diff changeset
1749 def _addpartsfromopts(ui, repo, bundler, source, outgoing, opts):
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32186
diff changeset
1750 # We should eventually reconcile this logic with the one behind
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32186
diff changeset
1751 # 'exchange.getbundle2partsgenerator'.
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32186
diff changeset
1752 #
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32186
diff changeset
1753 # The type of input from 'getbundle' and 'writenewbundle' are a bit
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32186
diff changeset
1754 # different right now. So we keep them separated for now for the sake of
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32186
diff changeset
1755 # simplicity.
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32186
diff changeset
1756
37005
66c0ff381cfc bundle: condition the changegroup part when creating a new bundle
Boris Feld <boris.feld@octobus.net>
parents: 36968
diff changeset
1757 # we might not always want a changegroup in such bundle, for example in
66c0ff381cfc bundle: condition the changegroup part when creating a new bundle
Boris Feld <boris.feld@octobus.net>
parents: 36968
diff changeset
1758 # stream bundles
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1759 if opts.get(b'changegroup', True):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1760 cgversion = opts.get(b'cg.version')
37005
66c0ff381cfc bundle: condition the changegroup part when creating a new bundle
Boris Feld <boris.feld@octobus.net>
parents: 36968
diff changeset
1761 if cgversion is None:
66c0ff381cfc bundle: condition the changegroup part when creating a new bundle
Boris Feld <boris.feld@octobus.net>
parents: 36968
diff changeset
1762 cgversion = changegroup.safeversion(repo)
66c0ff381cfc bundle: condition the changegroup part when creating a new bundle
Boris Feld <boris.feld@octobus.net>
parents: 36968
diff changeset
1763 cg = changegroup.makechangegroup(repo, outgoing, cgversion, source)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1764 part = bundler.newpart(b'changegroup', data=cg.getchunks())
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1765 part.addparam(b'version', cg.version)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1766 if b'clcount' in cg.extras:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1767 part.addparam(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1768 b'nbchanges', b'%d' % cg.extras[b'clcount'], mandatory=False
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1769 )
50392
385a4f8056e5 bundle: include required phases when saving a bundle (issue6794)
Jason R. Coombs <jaraco@jaraco.com>, Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49767
diff changeset
1770 if opts.get(b'phases'):
385a4f8056e5 bundle: include required phases when saving a bundle (issue6794)
Jason R. Coombs <jaraco@jaraco.com>, Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49767
diff changeset
1771 target_phase = phases.draft
385a4f8056e5 bundle: include required phases when saving a bundle (issue6794)
Jason R. Coombs <jaraco@jaraco.com>, Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49767
diff changeset
1772 for head in outgoing.ancestorsof:
385a4f8056e5 bundle: include required phases when saving a bundle (issue6794)
Jason R. Coombs <jaraco@jaraco.com>, Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49767
diff changeset
1773 target_phase = max(target_phase, repo[head].phase())
385a4f8056e5 bundle: include required phases when saving a bundle (issue6794)
Jason R. Coombs <jaraco@jaraco.com>, Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49767
diff changeset
1774 if target_phase > phases.draft:
385a4f8056e5 bundle: include required phases when saving a bundle (issue6794)
Jason R. Coombs <jaraco@jaraco.com>, Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49767
diff changeset
1775 part.addparam(
385a4f8056e5 bundle: include required phases when saving a bundle (issue6794)
Jason R. Coombs <jaraco@jaraco.com>, Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49767
diff changeset
1776 b'targetphase',
385a4f8056e5 bundle: include required phases when saving a bundle (issue6794)
Jason R. Coombs <jaraco@jaraco.com>, Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49767
diff changeset
1777 b'%d' % target_phase,
385a4f8056e5 bundle: include required phases when saving a bundle (issue6794)
Jason R. Coombs <jaraco@jaraco.com>, Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49767
diff changeset
1778 mandatory=False,
385a4f8056e5 bundle: include required phases when saving a bundle (issue6794)
Jason R. Coombs <jaraco@jaraco.com>, Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49767
diff changeset
1779 )
47226
19d4802cb304 sidedata: add a 'side-data' repository feature and use it
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47128
diff changeset
1780 if repository.REPO_FEATURE_SIDE_DATA in repo.features:
19d4802cb304 sidedata: add a 'side-data' repository feature and use it
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47128
diff changeset
1781 part.addparam(b'exp-sidedata', b'1')
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1782
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1783 if opts.get(b'streamv2', False):
37167
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part
Boris Feld <boris.feld@octobus.net>
parents: 37165
diff changeset
1784 addpartbundlestream2(bundler, repo, stream=True)
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part
Boris Feld <boris.feld@octobus.net>
parents: 37165
diff changeset
1785
50533
f4a540c203d7 stream-clone: support streamv3 on the cli [hg bundle]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 50532
diff changeset
1786 if opts.get(b'streamv3-exp', False):
f4a540c203d7 stream-clone: support streamv3 on the cli [hg bundle]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 50532
diff changeset
1787 addpartbundlestream2(bundler, repo, stream=True)
f4a540c203d7 stream-clone: support streamv3 on the cli [hg bundle]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 50532
diff changeset
1788
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1789 if opts.get(b'tagsfnodescache', True):
37165
6c7a6b04b274 bundlespec: move computing the bundle contentops in parsebundlespec
Boris Feld <boris.feld@octobus.net>
parents: 37084
diff changeset
1790 addparttagsfnodescache(repo, bundler, outgoing)
6c7a6b04b274 bundlespec: move computing the bundle contentops in parsebundlespec
Boris Feld <boris.feld@octobus.net>
parents: 37084
diff changeset
1791
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1792 if opts.get(b'revbranchcache', True):
37165
6c7a6b04b274 bundlespec: move computing the bundle contentops in parsebundlespec
Boris Feld <boris.feld@octobus.net>
parents: 37084
diff changeset
1793 addpartrevbranchcache(repo, bundler, outgoing)
32223
d7f93ebbbbdf bundle: add optional 'tagsfnodecache' data to on disk bundle (issue5543)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32222
diff changeset
1794
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1795 if opts.get(b'obsolescence', False):
32516
37d70ba1d9d1 bundle: add an experimental knob to include obsmarkers in bundle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32515
diff changeset
1796 obsmarkers = repo.obsstore.relevantmarkers(outgoing.missing)
46165
41d695a08e90 bundle: optional advisory obsolescence parts
Joerg Sonnenberger <joerg@bec.de>
parents: 46113
diff changeset
1797 buildobsmarkerspart(
41d695a08e90 bundle: optional advisory obsolescence parts
Joerg Sonnenberger <joerg@bec.de>
parents: 46113
diff changeset
1798 bundler,
41d695a08e90 bundle: optional advisory obsolescence parts
Joerg Sonnenberger <joerg@bec.de>
parents: 46113
diff changeset
1799 obsmarkers,
41d695a08e90 bundle: optional advisory obsolescence parts
Joerg Sonnenberger <joerg@bec.de>
parents: 46113
diff changeset
1800 mandatory=opts.get(b'obsolescence-mandatory', True),
41d695a08e90 bundle: optional advisory obsolescence parts
Joerg Sonnenberger <joerg@bec.de>
parents: 46113
diff changeset
1801 )
32516
37d70ba1d9d1 bundle: add an experimental knob to include obsmarkers in bundle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32515
diff changeset
1802
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1803 if opts.get(b'phases', False):
33031
e8c8d81eb864 bundle: add config option to include phases
Martin von Zweigbergk <martinvonz@google.com>
parents: 33030
diff changeset
1804 headsbyphase = phases.subsetphaseheads(repo, outgoing.missing)
34319
5779d096a696 phases: move binary encoding into a reusable function
Boris Feld <boris.feld@octobus.net>
parents: 34318
diff changeset
1805 phasedata = phases.binaryencode(headsbyphase)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1806 bundler.newpart(b'phase-heads', data=phasedata)
33031
e8c8d81eb864 bundle: add config option to include phases
Martin von Zweigbergk <martinvonz@google.com>
parents: 33030
diff changeset
1807
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1808
32222
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32221
diff changeset
1809 def addparttagsfnodescache(repo, bundler, outgoing):
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32221
diff changeset
1810 # we include the tags fnode cache for the bundle changeset
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32221
diff changeset
1811 # (as an optional parts)
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32221
diff changeset
1812 cache = tags.hgtagsfnodescache(repo.unfiltered())
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32221
diff changeset
1813 chunks = []
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32221
diff changeset
1814
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32221
diff changeset
1815 # .hgtags fnodes are only relevant for head changesets. While we could
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32221
diff changeset
1816 # transfer values for all known nodes, there will likely be little to
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32221
diff changeset
1817 # no benefit.
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32221
diff changeset
1818 #
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32221
diff changeset
1819 # We don't bother using a generator to produce output data because
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32221
diff changeset
1820 # a) we only have 40 bytes per head and even esoteric numbers of heads
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32221
diff changeset
1821 # consume little memory (1M heads is 40MB) b) we don't want to send the
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32221
diff changeset
1822 # part if we don't have entries and knowing if we have entries requires
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32221
diff changeset
1823 # cache lookups.
45144
c93dd9d9f1e6 discovery: change users of `outgoing.missingheads` to `outgoing.ancestorsof`
Manuel Jacob <me@manueljacob.de>
parents: 45117
diff changeset
1824 for node in outgoing.ancestorsof:
32222
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32221
diff changeset
1825 # Don't compute missing, as this may slow down serving.
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32221
diff changeset
1826 fnode = cache.getfnode(node, computemissing=False)
46495
5aac1a1a5beb tagcache: distinguish between invalid and missing entries
Matt Harbison <matt_harbison@yahoo.com>
parents: 46409
diff changeset
1827 if fnode:
32222
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32221
diff changeset
1828 chunks.extend([node, fnode])
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32221
diff changeset
1829
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32221
diff changeset
1830 if chunks:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1831 bundler.newpart(b'hgtagsfnodes', data=b''.join(chunks))
32222
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32221
diff changeset
1832
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1833
36964
79b73be4dda5 rev-branch-cache: add a function to generate a part
Boris Feld <boris.feld@octobus.net>
parents: 36963
diff changeset
1834 def addpartrevbranchcache(repo, bundler, outgoing):
79b73be4dda5 rev-branch-cache: add a function to generate a part
Boris Feld <boris.feld@octobus.net>
parents: 36963
diff changeset
1835 # we include the rev branch cache for the bundle changeset
79b73be4dda5 rev-branch-cache: add a function to generate a part
Boris Feld <boris.feld@octobus.net>
parents: 36963
diff changeset
1836 # (as an optional parts)
79b73be4dda5 rev-branch-cache: add a function to generate a part
Boris Feld <boris.feld@octobus.net>
parents: 36963
diff changeset
1837 cache = repo.revbranchcache()
79b73be4dda5 rev-branch-cache: add a function to generate a part
Boris Feld <boris.feld@octobus.net>
parents: 36963
diff changeset
1838 cl = repo.unfiltered().changelog
79b73be4dda5 rev-branch-cache: add a function to generate a part
Boris Feld <boris.feld@octobus.net>
parents: 36963
diff changeset
1839 branchesdata = collections.defaultdict(lambda: (set(), set()))
79b73be4dda5 rev-branch-cache: add a function to generate a part
Boris Feld <boris.feld@octobus.net>
parents: 36963
diff changeset
1840 for node in outgoing.missing:
79b73be4dda5 rev-branch-cache: add a function to generate a part
Boris Feld <boris.feld@octobus.net>
parents: 36963
diff changeset
1841 branch, close = cache.branchinfo(cl.rev(node))
79b73be4dda5 rev-branch-cache: add a function to generate a part
Boris Feld <boris.feld@octobus.net>
parents: 36963
diff changeset
1842 branchesdata[branch][close].add(node)
79b73be4dda5 rev-branch-cache: add a function to generate a part
Boris Feld <boris.feld@octobus.net>
parents: 36963
diff changeset
1843
79b73be4dda5 rev-branch-cache: add a function to generate a part
Boris Feld <boris.feld@octobus.net>
parents: 36963
diff changeset
1844 def generate():
79b73be4dda5 rev-branch-cache: add a function to generate a part
Boris Feld <boris.feld@octobus.net>
parents: 36963
diff changeset
1845 for branch, (nodes, closed) in sorted(branchesdata.items()):
79b73be4dda5 rev-branch-cache: add a function to generate a part
Boris Feld <boris.feld@octobus.net>
parents: 36963
diff changeset
1846 utf8branch = encoding.fromlocal(branch)
79b73be4dda5 rev-branch-cache: add a function to generate a part
Boris Feld <boris.feld@octobus.net>
parents: 36963
diff changeset
1847 yield rbcstruct.pack(len(utf8branch), len(nodes), len(closed))
79b73be4dda5 rev-branch-cache: add a function to generate a part
Boris Feld <boris.feld@octobus.net>
parents: 36963
diff changeset
1848 yield utf8branch
79b73be4dda5 rev-branch-cache: add a function to generate a part
Boris Feld <boris.feld@octobus.net>
parents: 36963
diff changeset
1849 for n in sorted(nodes):
79b73be4dda5 rev-branch-cache: add a function to generate a part
Boris Feld <boris.feld@octobus.net>
parents: 36963
diff changeset
1850 yield n
79b73be4dda5 rev-branch-cache: add a function to generate a part
Boris Feld <boris.feld@octobus.net>
parents: 36963
diff changeset
1851 for n in sorted(closed):
79b73be4dda5 rev-branch-cache: add a function to generate a part
Boris Feld <boris.feld@octobus.net>
parents: 36963
diff changeset
1852 yield n
79b73be4dda5 rev-branch-cache: add a function to generate a part
Boris Feld <boris.feld@octobus.net>
parents: 36963
diff changeset
1853
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1854 bundler.newpart(b'cache:rev-branch-cache', data=generate(), mandatory=False)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1855
36964
79b73be4dda5 rev-branch-cache: add a function to generate a part
Boris Feld <boris.feld@octobus.net>
parents: 36963
diff changeset
1856
37167
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part
Boris Feld <boris.feld@octobus.net>
parents: 37165
diff changeset
1857 def _formatrequirementsspec(requirements):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1858 requirements = [req for req in requirements if req != b"shared"]
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1859 return urlreq.quote(b','.join(sorted(requirements)))
37167
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part
Boris Feld <boris.feld@octobus.net>
parents: 37165
diff changeset
1860
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1861
37167
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part
Boris Feld <boris.feld@octobus.net>
parents: 37165
diff changeset
1862 def _formatrequirementsparams(requirements):
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part
Boris Feld <boris.feld@octobus.net>
parents: 37165
diff changeset
1863 requirements = _formatrequirementsspec(requirements)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1864 params = b"%s%s" % (urlreq.quote(b"requirements="), requirements)
37167
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part
Boris Feld <boris.feld@octobus.net>
parents: 37165
diff changeset
1865 return params
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part
Boris Feld <boris.feld@octobus.net>
parents: 37165
diff changeset
1866
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1867
46713
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos
Raphaël Gomès <rgomes@octobus.net>
parents: 46633
diff changeset
1868 def format_remote_wanted_sidedata(repo):
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos
Raphaël Gomès <rgomes@octobus.net>
parents: 46633
diff changeset
1869 """Formats a repo's wanted sidedata categories into a bytestring for
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos
Raphaël Gomès <rgomes@octobus.net>
parents: 46633
diff changeset
1870 capabilities exchange."""
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos
Raphaël Gomès <rgomes@octobus.net>
parents: 46633
diff changeset
1871 wanted = b""
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos
Raphaël Gomès <rgomes@octobus.net>
parents: 46633
diff changeset
1872 if repo._wanted_sidedata:
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos
Raphaël Gomès <rgomes@octobus.net>
parents: 46633
diff changeset
1873 wanted = b','.join(
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos
Raphaël Gomès <rgomes@octobus.net>
parents: 46633
diff changeset
1874 pycompat.bytestr(c) for c in sorted(repo._wanted_sidedata)
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos
Raphaël Gomès <rgomes@octobus.net>
parents: 46633
diff changeset
1875 )
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos
Raphaël Gomès <rgomes@octobus.net>
parents: 46633
diff changeset
1876 return wanted
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos
Raphaël Gomès <rgomes@octobus.net>
parents: 46633
diff changeset
1877
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos
Raphaël Gomès <rgomes@octobus.net>
parents: 46633
diff changeset
1878
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos
Raphaël Gomès <rgomes@octobus.net>
parents: 46633
diff changeset
1879 def read_remote_wanted_sidedata(remote):
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos
Raphaël Gomès <rgomes@octobus.net>
parents: 46633
diff changeset
1880 sidedata_categories = remote.capable(b'exp-wanted-sidedata')
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos
Raphaël Gomès <rgomes@octobus.net>
parents: 46633
diff changeset
1881 return read_wanted_sidedata(sidedata_categories)
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos
Raphaël Gomès <rgomes@octobus.net>
parents: 46633
diff changeset
1882
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos
Raphaël Gomès <rgomes@octobus.net>
parents: 46633
diff changeset
1883
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos
Raphaël Gomès <rgomes@octobus.net>
parents: 46633
diff changeset
1884 def read_wanted_sidedata(formatted):
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos
Raphaël Gomès <rgomes@octobus.net>
parents: 46633
diff changeset
1885 if formatted:
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos
Raphaël Gomès <rgomes@octobus.net>
parents: 46633
diff changeset
1886 return set(formatted.split(b','))
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos
Raphaël Gomès <rgomes@octobus.net>
parents: 46633
diff changeset
1887 return set()
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos
Raphaël Gomès <rgomes@octobus.net>
parents: 46633
diff changeset
1888
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos
Raphaël Gomès <rgomes@octobus.net>
parents: 46633
diff changeset
1889
37167
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part
Boris Feld <boris.feld@octobus.net>
parents: 37165
diff changeset
1890 def addpartbundlestream2(bundler, repo, **kwargs):
43506
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43495
diff changeset
1891 if not kwargs.get('stream', False):
37167
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part
Boris Feld <boris.feld@octobus.net>
parents: 37165
diff changeset
1892 return
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part
Boris Feld <boris.feld@octobus.net>
parents: 37165
diff changeset
1893
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part
Boris Feld <boris.feld@octobus.net>
parents: 37165
diff changeset
1894 if not streamclone.allowservergeneration(repo):
50517
cd2a2963b982 stream-clone: upgrade the error message for bad stream request
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 50399
diff changeset
1895 msg = _(b'stream data requested but server does not allow this feature')
cd2a2963b982 stream-clone: upgrade the error message for bad stream request
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 50399
diff changeset
1896 hint = _(b'the client seems buggy')
cd2a2963b982 stream-clone: upgrade the error message for bad stream request
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 50399
diff changeset
1897 raise error.Abort(msg, hint=hint)
50519
3c0da0fa0eed stream-clone: check the version of streaming clone supported by the client
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 50518
diff changeset
1898 if not (b'stream' in bundler.capabilities):
3c0da0fa0eed stream-clone: check the version of streaming clone supported by the client
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 50518
diff changeset
1899 msg = _(
3c0da0fa0eed stream-clone: check the version of streaming clone supported by the client
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 50518
diff changeset
1900 b'stream data requested but supported streaming clone versions were not specified'
3c0da0fa0eed stream-clone: check the version of streaming clone supported by the client
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 50518
diff changeset
1901 )
3c0da0fa0eed stream-clone: check the version of streaming clone supported by the client
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 50518
diff changeset
1902 hint = _(b'the client seems buggy')
3c0da0fa0eed stream-clone: check the version of streaming clone supported by the client
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 50518
diff changeset
1903 raise error.Abort(msg, hint=hint)
50524
58adcabc295f stream-clone: introduce the notion of an experimental "v3" version
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50519
diff changeset
1904 client_supported = set(bundler.capabilities[b'stream'])
58adcabc295f stream-clone: introduce the notion of an experimental "v3" version
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50519
diff changeset
1905 server_supported = set(getrepocaps(repo, role=b'client').get(b'stream', []))
58adcabc295f stream-clone: introduce the notion of an experimental "v3" version
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50519
diff changeset
1906 common_supported = client_supported & server_supported
58adcabc295f stream-clone: introduce the notion of an experimental "v3" version
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50519
diff changeset
1907 if not common_supported:
58adcabc295f stream-clone: introduce the notion of an experimental "v3" version
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50519
diff changeset
1908 msg = _(b'no common supported version with the client: %s; %s')
58adcabc295f stream-clone: introduce the notion of an experimental "v3" version
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50519
diff changeset
1909 str_server = b','.join(sorted(server_supported))
58adcabc295f stream-clone: introduce the notion of an experimental "v3" version
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50519
diff changeset
1910 str_client = b','.join(sorted(client_supported))
58adcabc295f stream-clone: introduce the notion of an experimental "v3" version
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50519
diff changeset
1911 msg %= (str_server, str_client)
58adcabc295f stream-clone: introduce the notion of an experimental "v3" version
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50519
diff changeset
1912 raise error.Abort(msg)
58adcabc295f stream-clone: introduce the notion of an experimental "v3" version
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50519
diff changeset
1913 version = max(common_supported)
37167
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part
Boris Feld <boris.feld@octobus.net>
parents: 37165
diff changeset
1914
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part
Boris Feld <boris.feld@octobus.net>
parents: 37165
diff changeset
1915 # Stream clones don't compress well. And compression undermines a
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part
Boris Feld <boris.feld@octobus.net>
parents: 37165
diff changeset
1916 # goal of stream clones, which is to be fast. Communicate the desire
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part
Boris Feld <boris.feld@octobus.net>
parents: 37165
diff changeset
1917 # to avoid compression to consumers of the bundle.
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part
Boris Feld <boris.feld@octobus.net>
parents: 37165
diff changeset
1918 bundler.prefercompressed = False
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part
Boris Feld <boris.feld@octobus.net>
parents: 37165
diff changeset
1919
40338
af62936c2508 streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40147
diff changeset
1920 # get the includes and excludes
43506
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43495
diff changeset
1921 includepats = kwargs.get('includepats')
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43495
diff changeset
1922 excludepats = kwargs.get('excludepats')
40338
af62936c2508 streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40147
diff changeset
1923
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1924 narrowstream = repo.ui.configbool(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1925 b'experimental', b'server.stream-narrow-clones'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1926 )
40338
af62936c2508 streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40147
diff changeset
1927
af62936c2508 streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40147
diff changeset
1928 if (includepats or excludepats) and not narrowstream:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1929 raise error.Abort(_(b'server does not support narrow stream clones'))
40338
af62936c2508 streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40147
diff changeset
1930
40398
0ac794e0e285 streamclone: include obsstore file into stream bundle if client can read it
Anton Shestakov <av6@dwimlabs.net>
parents: 40372
diff changeset
1931 includeobsmarkers = False
0ac794e0e285 streamclone: include obsstore file into stream bundle if client can read it
Anton Shestakov <av6@dwimlabs.net>
parents: 40372
diff changeset
1932 if repo.obsstore:
0ac794e0e285 streamclone: include obsstore file into stream bundle if client can read it
Anton Shestakov <av6@dwimlabs.net>
parents: 40372
diff changeset
1933 remoteversions = obsmarkersversion(bundler.capabilities)
40399
4ab6e7b4fe8a streamclone: abort when client needs to handle obsmarkers, but doesn't
Anton Shestakov <av6@dwimlabs.net>
parents: 40398
diff changeset
1934 if not remoteversions:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1935 raise error.Abort(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1936 _(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1937 b'server has obsolescence markers, but client '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1938 b'cannot receive them via stream clone'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1939 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1940 )
40399
4ab6e7b4fe8a streamclone: abort when client needs to handle obsmarkers, but doesn't
Anton Shestakov <av6@dwimlabs.net>
parents: 40398
diff changeset
1941 elif repo.obsstore._version in remoteversions:
40398
0ac794e0e285 streamclone: include obsstore file into stream bundle if client can read it
Anton Shestakov <av6@dwimlabs.net>
parents: 40372
diff changeset
1942 includeobsmarkers = True
0ac794e0e285 streamclone: include obsstore file into stream bundle if client can read it
Anton Shestakov <av6@dwimlabs.net>
parents: 40372
diff changeset
1943
50524
58adcabc295f stream-clone: introduce the notion of an experimental "v3" version
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50519
diff changeset
1944 if version == b"v2":
58adcabc295f stream-clone: introduce the notion of an experimental "v3" version
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50519
diff changeset
1945 filecount, bytecount, it = streamclone.generatev2(
58adcabc295f stream-clone: introduce the notion of an experimental "v3" version
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50519
diff changeset
1946 repo, includepats, excludepats, includeobsmarkers
58adcabc295f stream-clone: introduce the notion of an experimental "v3" version
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50519
diff changeset
1947 )
58adcabc295f stream-clone: introduce the notion of an experimental "v3" version
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50519
diff changeset
1948 requirements = streamclone.streamed_requirements(repo)
58adcabc295f stream-clone: introduce the notion of an experimental "v3" version
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50519
diff changeset
1949 requirements = _formatrequirementsspec(requirements)
58adcabc295f stream-clone: introduce the notion of an experimental "v3" version
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50519
diff changeset
1950 part = bundler.newpart(b'stream2', data=it)
58adcabc295f stream-clone: introduce the notion of an experimental "v3" version
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50519
diff changeset
1951 part.addparam(b'bytecount', b'%d' % bytecount, mandatory=True)
58adcabc295f stream-clone: introduce the notion of an experimental "v3" version
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50519
diff changeset
1952 part.addparam(b'filecount', b'%d' % filecount, mandatory=True)
58adcabc295f stream-clone: introduce the notion of an experimental "v3" version
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50519
diff changeset
1953 part.addparam(b'requirements', requirements, mandatory=True)
58adcabc295f stream-clone: introduce the notion of an experimental "v3" version
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50519
diff changeset
1954 elif version == b"v3-exp":
50706
0452af304808 stream-clone: add a v3 version of the protocol
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 50616
diff changeset
1955 it = streamclone.generatev3(
50524
58adcabc295f stream-clone: introduce the notion of an experimental "v3" version
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50519
diff changeset
1956 repo, includepats, excludepats, includeobsmarkers
58adcabc295f stream-clone: introduce the notion of an experimental "v3" version
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50519
diff changeset
1957 )
58adcabc295f stream-clone: introduce the notion of an experimental "v3" version
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50519
diff changeset
1958 requirements = streamclone.streamed_requirements(repo)
58adcabc295f stream-clone: introduce the notion of an experimental "v3" version
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50519
diff changeset
1959 requirements = _formatrequirementsspec(requirements)
50532
116da6bb7e3b stream-clone: add the `-exp` prefix to the bundle part
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50524
diff changeset
1960 part = bundler.newpart(b'stream3-exp', data=it)
50524
58adcabc295f stream-clone: introduce the notion of an experimental "v3" version
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50519
diff changeset
1961 part.addparam(b'requirements', requirements, mandatory=True)
37167
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part
Boris Feld <boris.feld@octobus.net>
parents: 37165
diff changeset
1962
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1963
46165
41d695a08e90 bundle: optional advisory obsolescence parts
Joerg Sonnenberger <joerg@bec.de>
parents: 46113
diff changeset
1964 def buildobsmarkerspart(bundler, markers, mandatory=True):
32515
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32387
diff changeset
1965 """add an obsmarker part to the bundler with <markers>
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32387
diff changeset
1966
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32387
diff changeset
1967 No part is created if markers is empty.
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32387
diff changeset
1968 Raises ValueError if the bundler doesn't support any known obsmarker format.
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32387
diff changeset
1969 """
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32387
diff changeset
1970 if not markers:
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32387
diff changeset
1971 return None
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32387
diff changeset
1972
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32387
diff changeset
1973 remoteversions = obsmarkersversion(bundler.capabilities)
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32387
diff changeset
1974 version = obsolete.commonversion(remoteversions)
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32387
diff changeset
1975 if version is None:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1976 raise ValueError(b'bundler does not support common obsmarker format')
32515
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32387
diff changeset
1977 stream = obsolete.encodemarkers(markers, True, version=version)
46165
41d695a08e90 bundle: optional advisory obsolescence parts
Joerg Sonnenberger <joerg@bec.de>
parents: 46113
diff changeset
1978 return bundler.newpart(b'obsmarkers', data=stream, mandatory=mandatory)
32515
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32387
diff changeset
1979
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1980
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1981 def writebundle(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1982 ui, cg, filename, bundletype, vfs=None, compression=None, compopts=None
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1983 ):
28666
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1984 """Write a bundle file and return its filename.
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1985
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1986 Existing files will not be overwritten.
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1987 If no filename is specified, a temporary file is created.
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1988 bz2 compression can be turned off.
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1989 The bundle file will be deleted in case of errors.
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1990 """
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1991
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1992 if bundletype == b"HG20":
28666
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1993 bundle = bundle20(ui)
30757
511a4bf52754 bundle2: allow compression options to be passed to compressor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30439
diff changeset
1994 bundle.setcompression(compression, compopts)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1995 part = bundle.newpart(b'changegroup', data=cg.getchunks())
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1996 part.addparam(b'version', cg.version)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1997 if b'clcount' in cg.extras:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
1998 part.addparam(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1999 b'nbchanges', b'%d' % cg.extras[b'clcount'], mandatory=False
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2000 )
28666
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
2001 chunkiter = bundle.getchunks()
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
2002 else:
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
2003 # compression argument is only for the bundle2 case
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
2004 assert compression is None
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2005 if cg.version != b'01':
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2006 raise error.Abort(
43117
8ff1ecfadcd1 cleanup: join string literals that are already on one line
Martin von Zweigbergk <martinvonz@google.com>
parents: 43115
diff changeset
2007 _(b'old bundle types only supports v1 changegroups')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2008 )
49648
9be765b82a90 typing: minor tweaks to allow updating to pytype 2022.11.18
Matt Harbison <matt_harbison@yahoo.com>
parents: 49336
diff changeset
2009
9be765b82a90 typing: minor tweaks to allow updating to pytype 2022.11.18
Matt Harbison <matt_harbison@yahoo.com>
parents: 49336
diff changeset
2010 # HG20 is the case without 2 values to unpack, but is handled above.
9be765b82a90 typing: minor tweaks to allow updating to pytype 2022.11.18
Matt Harbison <matt_harbison@yahoo.com>
parents: 49336
diff changeset
2011 # pytype: disable=bad-unpacking
28666
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
2012 header, comp = bundletypes[bundletype]
49648
9be765b82a90 typing: minor tweaks to allow updating to pytype 2022.11.18
Matt Harbison <matt_harbison@yahoo.com>
parents: 49336
diff changeset
2013 # pytype: enable=bad-unpacking
9be765b82a90 typing: minor tweaks to allow updating to pytype 2022.11.18
Matt Harbison <matt_harbison@yahoo.com>
parents: 49336
diff changeset
2014
30351
f81002f736d7 bundle2: use new compression engine API for compression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30177
diff changeset
2015 if comp not in util.compengines.supportedbundletypes:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2016 raise error.Abort(_(b'unknown stream compression type: %s') % comp)
30351
f81002f736d7 bundle2: use new compression engine API for compression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30177
diff changeset
2017 compengine = util.compengines.forbundletype(comp)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2018
28666
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
2019 def chunkiter():
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
2020 yield header
30757
511a4bf52754 bundle2: allow compression options to be passed to compressor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30439
diff changeset
2021 for chunk in compengine.compressstream(cg.getchunks(), compopts):
30357
5925bda42dbd bundle2: use compressstream compression engine API
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30353
diff changeset
2022 yield chunk
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2023
28666
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
2024 chunkiter = chunkiter()
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
2025
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
2026 # parse the changegroup data, otherwise we will block
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
2027 # in case of sshrepo because we don't know the end of the stream
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
2028 return changegroup.writechunks(ui, chunkiter, filename, vfs=vfs)
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
2029
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2030
33037
d765ad56081f bundle: make combinechangegroupresults() take a bundleoperation
Martin von Zweigbergk <martinvonz@google.com>
parents: 33036
diff changeset
2031 def combinechangegroupresults(op):
33036
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33035
diff changeset
2032 """logic to combine 0 or more addchangegroup results into one"""
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2033 results = [r.get(b'return', 0) for r in op.records[b'changegroup']]
33036
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33035
diff changeset
2034 changedheads = 0
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33035
diff changeset
2035 result = 1
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33035
diff changeset
2036 for ret in results:
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33035
diff changeset
2037 # If any changegroup result is 0, return 0
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33035
diff changeset
2038 if ret == 0:
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33035
diff changeset
2039 result = 0
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33035
diff changeset
2040 break
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33035
diff changeset
2041 if ret < -1:
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33035
diff changeset
2042 changedheads += ret + 1
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33035
diff changeset
2043 elif ret > 1:
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33035
diff changeset
2044 changedheads += ret - 1
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33035
diff changeset
2045 if changedheads > 0:
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33035
diff changeset
2046 result = 1 + changedheads
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33035
diff changeset
2047 elif changedheads < 0:
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33035
diff changeset
2048 result = -1 + changedheads
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33035
diff changeset
2049 return result
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33035
diff changeset
2050
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2051
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2052 @parthandler(
43131
c17a63eb5d4c sidedata: apply basic but tight security around exchange
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43117
diff changeset
2053 b'changegroup',
c17a63eb5d4c sidedata: apply basic but tight security around exchange
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43117
diff changeset
2054 (
c17a63eb5d4c sidedata: apply basic but tight security around exchange
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43117
diff changeset
2055 b'version',
c17a63eb5d4c sidedata: apply basic but tight security around exchange
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43117
diff changeset
2056 b'nbchanges',
c17a63eb5d4c sidedata: apply basic but tight security around exchange
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43117
diff changeset
2057 b'exp-sidedata',
46713
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos
Raphaël Gomès <rgomes@octobus.net>
parents: 46633
diff changeset
2058 b'exp-wanted-sidedata',
43131
c17a63eb5d4c sidedata: apply basic but tight security around exchange
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43117
diff changeset
2059 b'treemanifest',
c17a63eb5d4c sidedata: apply basic but tight security around exchange
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43117
diff changeset
2060 b'targetphase',
c17a63eb5d4c sidedata: apply basic but tight security around exchange
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43117
diff changeset
2061 ),
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2062 )
20998
93a3c5b58635 bundle2: use reply part to return result of addchangegroup
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20997
diff changeset
2063 def handlechangegroup(op, inpart):
46166
c511fef30290 bundle2: drop some outdated comment
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46165
diff changeset
2064 """apply a changegroup part on the repo"""
39700
b10d145837bc localrepo: extract resolving of opener options to standalone functions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38783
diff changeset
2065 from . import localrepo
b10d145837bc localrepo: extract resolving of opener options to standalone functions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38783
diff changeset
2066
32930
af31d531dda0 changegroup: let callers pass in transaction to apply() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 32892
diff changeset
2067 tr = op.gettransaction()
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2068 unpackerversion = inpart.params.get(b'version', b'01')
23170
02e8f9b60052 bundle2: support a "version" argument in `changegroup` part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23169
diff changeset
2069 # We should raise an appropriate exception here
27751
a40e2f7fe49d changegroup: hide packermap behind methods
Martin von Zweigbergk <martinvonz@google.com>
parents: 27734
diff changeset
2070 cg = changegroup.getunbundler(unpackerversion, inpart, None)
23001
4df9b5e62f70 bundle2: add a comment about addchangegroup source and url
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22953
diff changeset
2071 # the source and url passed here are overwritten by the one contained in
4df9b5e62f70 bundle2: add a comment about addchangegroup source and url
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22953
diff changeset
2072 # the transaction.hookargs argument. So 'bundle2' is a placeholder
25518
ca656f3dffd7 bundle2: provide number of changesets information to 'addchangegroup'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25507
diff changeset
2073 nbchangesets = None
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2074 if b'nbchanges' in inpart.params:
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2075 nbchangesets = int(inpart.params.get(b'nbchanges'))
45552
10284ce3d5ed scmutil: introduce function to check whether repo uses treemanifest or not
Pulkit Goyal <7895pulkit@gmail.com>
parents: 45372
diff changeset
2076 if b'treemanifest' in inpart.params and not scmutil.istreemanifest(op.repo):
27734
5c0fd878779c treemanifests: set bundle2 part parameter indicating treemanifest
Martin von Zweigbergk <martinvonz@google.com>
parents: 26829
diff changeset
2077 if len(op.repo.changelog) != 0:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2078 raise error.Abort(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2079 _(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2080 b"bundle contains tree manifests, but local repo is "
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2081 b"non-empty and does not use tree manifests"
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2082 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2083 )
45372
77b8588dd84e requirements: introduce new requirements related module
Pulkit Goyal <7895pulkit@gmail.com>
parents: 45371
diff changeset
2084 op.repo.requirements.add(requirements.TREEMANIFEST_REQUIREMENT)
39700
b10d145837bc localrepo: extract resolving of opener options to standalone functions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38783
diff changeset
2085 op.repo.svfs.options = localrepo.resolvestorevfsoptions(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2086 op.repo.ui, op.repo.requirements, op.repo.features
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2087 )
45106
a03c177a4679 scmutil: add writereporequirements() and route requires writing through it
Pulkit Goyal <7895pulkit@gmail.com>
parents: 44372
diff changeset
2088 scmutil.writereporequirements(op.repo)
43131
c17a63eb5d4c sidedata: apply basic but tight security around exchange
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43117
diff changeset
2089
33407
39d4e5a66f5a bundle2: support the 'targetphase' parameter for the changegroup part
Boris Feld <boris.feld@octobus.net>
parents: 33252
diff changeset
2090 extrakwargs = {}
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2091 targetphase = inpart.params.get(b'targetphase')
33407
39d4e5a66f5a bundle2: support the 'targetphase' parameter for the changegroup part
Boris Feld <boris.feld@octobus.net>
parents: 33252
diff changeset
2092 if targetphase is not None:
43506
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43495
diff changeset
2093 extrakwargs['targetphase'] = int(targetphase)
46713
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos
Raphaël Gomès <rgomes@octobus.net>
parents: 46633
diff changeset
2094
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos
Raphaël Gomès <rgomes@octobus.net>
parents: 46633
diff changeset
2095 remote_sidedata = inpart.params.get(b'exp-wanted-sidedata')
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos
Raphaël Gomès <rgomes@octobus.net>
parents: 46633
diff changeset
2096 extrakwargs['sidedata_categories'] = read_wanted_sidedata(remote_sidedata)
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos
Raphaël Gomès <rgomes@octobus.net>
parents: 46633
diff changeset
2097
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2098 ret = _processchangegroup(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2099 op,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2100 cg,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2101 tr,
46519
83ac7c91e172 bundle2: pass the operation source down to the changegroup
Raphaël Gomès <rgomes@octobus.net>
parents: 46495
diff changeset
2102 op.source,
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2103 b'bundle2',
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2104 expectedtotal=nbchangesets,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2105 **extrakwargs
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2106 )
20998
93a3c5b58635 bundle2: use reply part to return result of addchangegroup
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20997
diff changeset
2107 if op.reply is not None:
23139
e53f6b72a0e4 spelling: fixes from proofreading of spell checker issues
Mads Kiilerich <madski@unity3d.com>
parents: 23067
diff changeset
2108 # This is definitely not the final form of this
20998
93a3c5b58635 bundle2: use reply part to return result of addchangegroup
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20997
diff changeset
2109 # return. But one need to start somewhere.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2110 part = op.reply.newpart(b'reply:changegroup', mandatory=False)
33675
24b7c75c6441 bundle2: use bytestr() instead of str() to convert part id to bytes
Augie Fackler <augie@google.com>
parents: 33674
diff changeset
2111 part.addparam(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2112 b'in-reply-to', pycompat.bytestr(inpart.id), mandatory=False
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2113 )
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2114 part.addparam(b'return', b'%i' % ret, mandatory=False)
21019
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
2115 assert not inpart.read()
20950
c7ceae0faf69 bundle2: first crude version of bundling changeset with bundle2
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20949
diff changeset
2116
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2117
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2118 _remotechangegroupparams = tuple(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2119 [b'url', b'size', b'digests']
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2120 + [b'digest:%s' % k for k in util.DIGESTS.keys()]
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2121 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2122
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2123
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2124 @parthandler(b'remote-changegroup', _remotechangegroupparams)
23029
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2125 def handleremotechangegroup(op, inpart):
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2126 """apply a bundle10 on the repo, given an url and validation information
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2127
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2128 All the information about the remote bundle to import are given as
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2129 parameters. The parameters include:
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2130 - url: the url to the bundle10.
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2131 - size: the bundle10 file size. It is used to validate what was
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2132 retrieved by the client matches the server knowledge about the bundle.
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2133 - digests: a space separated list of the digest types provided as
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2134 parameters.
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2135 - digest:<digest-type>: the hexadecimal representation of the digest with
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2136 that name. Like the size, it is used to validate what was retrieved by
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2137 the client matches what the server knows about the bundle.
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2138
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2139 When multiple digest types are given, all of them are checked.
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2140 """
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2141 try:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2142 raw_url = inpart.params[b'url']
23029
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2143 except KeyError:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2144 raise error.Abort(_(b'remote-changegroup: missing "%s" param') % b'url')
46907
ffd3e823a7e5 urlutil: extract `url` related code from `util` into the new module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46780
diff changeset
2145 parsed_url = urlutil.url(raw_url)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2146 if parsed_url.scheme not in capabilities[b'remote-changegroup']:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2147 raise error.Abort(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2148 _(b'remote-changegroup does not support %s urls')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2149 % parsed_url.scheme
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2150 )
23029
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2151
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2152 try:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2153 size = int(inpart.params[b'size'])
23029
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2154 except ValueError:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2155 raise error.Abort(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2156 _(b'remote-changegroup: invalid value for param "%s"') % b'size'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2157 )
23029
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2158 except KeyError:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2159 raise error.Abort(
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2160 _(b'remote-changegroup: missing "%s" param') % b'size'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2161 )
23029
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2162
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2163 digests = {}
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2164 for typ in inpart.params.get(b'digests', b'').split():
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2165 param = b'digest:%s' % typ
23029
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2166 try:
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2167 value = inpart.params[param]
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2168 except KeyError:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2169 raise error.Abort(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2170 _(b'remote-changegroup: missing "%s" param') % param
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2171 )
23029
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2172 digests[typ] = value
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2173
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2174 real_part = util.digestchecker(url.open(op.ui, raw_url), size, digests)
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2175
32930
af31d531dda0 changegroup: let callers pass in transaction to apply() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 32892
diff changeset
2176 tr = op.gettransaction()
25919
8221fefaea08 bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
2177 from . import exchange
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2178
23029
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2179 cg = exchange.readbundle(op.repo.ui, real_part, raw_url)
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2180 if not isinstance(cg, changegroup.cg1unpacker):
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2181 raise error.Abort(
46907
ffd3e823a7e5 urlutil: extract `url` related code from `util` into the new module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46780
diff changeset
2182 _(b'%s: not a bundle version 1.0') % urlutil.hidepassword(raw_url)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2183 )
46519
83ac7c91e172 bundle2: pass the operation source down to the changegroup
Raphaël Gomès <rgomes@octobus.net>
parents: 46495
diff changeset
2184 ret = _processchangegroup(op, cg, tr, op.source, b'bundle2')
23029
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2185 if op.reply is not None:
23139
e53f6b72a0e4 spelling: fixes from proofreading of spell checker issues
Mads Kiilerich <madski@unity3d.com>
parents: 23067
diff changeset
2186 # This is definitely not the final form of this
23029
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2187 # return. But one need to start somewhere.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2188 part = op.reply.newpart(b'reply:changegroup')
33675
24b7c75c6441 bundle2: use bytestr() instead of str() to convert part id to bytes
Augie Fackler <augie@google.com>
parents: 33674
diff changeset
2189 part.addparam(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2190 b'in-reply-to', pycompat.bytestr(inpart.id), mandatory=False
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2191 )
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2192 part.addparam(b'return', b'%i' % ret, mandatory=False)
23029
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2193 try:
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2194 real_part.validate()
26587
56b2bcea2529 error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26565
diff changeset
2195 except error.Abort as e:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2196 raise error.Abort(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2197 _(b'bundle at %s is corrupted:\n%s')
46907
ffd3e823a7e5 urlutil: extract `url` related code from `util` into the new module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46780
diff changeset
2198 % (urlutil.hidepassword(raw_url), e.message)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2199 )
23029
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2200 assert not inpart.read()
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
2201
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2202
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2203 @parthandler(b'reply:changegroup', (b'return', b'in-reply-to'))
22548
8a1ae133770a bundle2: rename functions that have the same name
Mike Hommey <mh@glandium.org>
parents: 22390
diff changeset
2204 def handlereplychangegroup(op, inpart):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2205 ret = int(inpart.params[b'return'])
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2206 replyto = int(inpart.params[b'in-reply-to'])
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2207 op.records.add(b'changegroup', {b'return': ret}, replyto)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2208
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2209
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2210 @parthandler(b'check:bookmarks')
35258
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35120
diff changeset
2211 def handlecheckbookmarks(op, inpart):
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35120
diff changeset
2212 """check location of bookmarks
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35120
diff changeset
2213
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35120
diff changeset
2214 This part is to be used to detect push race regarding bookmark, it
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35120
diff changeset
2215 contains binary encoded (bookmark, node) tuple. If the local state does
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35120
diff changeset
2216 not marks the one in the part, a PushRaced exception is raised
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35120
diff changeset
2217 """
46780
6266d19556ad node: introduce nodeconstants class
Joerg Sonnenberger <joerg@bec.de>
parents: 46713
diff changeset
2218 bookdata = bookmarks.binarydecode(op.repo, inpart)
35258
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35120
diff changeset
2219
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2220 msgstandard = (
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2221 b'remote repository changed while pushing - please try again '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2222 b'(bookmark "%s" move from %s to %s)'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2223 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2224 msgmissing = (
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2225 b'remote repository changed while pushing - please try again '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2226 b'(bookmark "%s" is missing, expected %s)'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2227 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2228 msgexist = (
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2229 b'remote repository changed while pushing - please try again '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2230 b'(bookmark "%s" set on %s, expected missing)'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2231 )
35258
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35120
diff changeset
2232 for book, node in bookdata:
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35120
diff changeset
2233 currentnode = op.repo._bookmarks.get(book)
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35120
diff changeset
2234 if currentnode != node:
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35120
diff changeset
2235 if node is None:
46113
59fa3890d40a node: import symbols explicitly
Joerg Sonnenberger <joerg@bec.de>
parents: 45942
diff changeset
2236 finalmsg = msgexist % (book, short(currentnode))
35258
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35120
diff changeset
2237 elif currentnode is None:
46113
59fa3890d40a node: import symbols explicitly
Joerg Sonnenberger <joerg@bec.de>
parents: 45942
diff changeset
2238 finalmsg = msgmissing % (book, short(node))
35258
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35120
diff changeset
2239 else:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2240 finalmsg = msgstandard % (
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2241 book,
46113
59fa3890d40a node: import symbols explicitly
Joerg Sonnenberger <joerg@bec.de>
parents: 45942
diff changeset
2242 short(node),
59fa3890d40a node: import symbols explicitly
Joerg Sonnenberger <joerg@bec.de>
parents: 45942
diff changeset
2243 short(currentnode),
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2244 )
35258
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35120
diff changeset
2245 raise error.PushRaced(finalmsg)
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35120
diff changeset
2246
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2247
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2248 @parthandler(b'check:heads')
22548
8a1ae133770a bundle2: rename functions that have the same name
Mike Hommey <mh@glandium.org>
parents: 22390
diff changeset
2249 def handlecheckheads(op, inpart):
21060
0bea9db7543b bundle2: add a "check:heads" handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21024
diff changeset
2250 """check that head of the repo did not change
0bea9db7543b bundle2: add a "check:heads" handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21024
diff changeset
2251
0bea9db7543b bundle2: add a "check:heads" handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21024
diff changeset
2252 This is used to detect a push race when using unbundle.
0bea9db7543b bundle2: add a "check:heads" handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21024
diff changeset
2253 This replaces the "heads" argument of unbundle."""
0bea9db7543b bundle2: add a "check:heads" handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21024
diff changeset
2254 h = inpart.read(20)
0bea9db7543b bundle2: add a "check:heads" handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21024
diff changeset
2255 heads = []
0bea9db7543b bundle2: add a "check:heads" handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21024
diff changeset
2256 while len(h) == 20:
0bea9db7543b bundle2: add a "check:heads" handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21024
diff changeset
2257 heads.append(h)
0bea9db7543b bundle2: add a "check:heads" handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21024
diff changeset
2258 h = inpart.read(20)
0bea9db7543b bundle2: add a "check:heads" handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21024
diff changeset
2259 assert not h
26565
ee1bcb9aa6e4 bundle2: add op.gettransaction() to handlers that need the lock
Durham Goode <durham@fb.com>
parents: 26542
diff changeset
2260 # Trigger a transaction so that we are guaranteed to have the lock now.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2261 if op.ui.configbool(b'experimental', b'bundle2lazylocking'):
26565
ee1bcb9aa6e4 bundle2: add op.gettransaction() to handlers that need the lock
Durham Goode <durham@fb.com>
parents: 26542
diff changeset
2262 op.gettransaction()
29294
077d0535f51f bundle2: don't assume ordering of heads checked after push
Mads Kiilerich <madski@unity3d.com>
parents: 28883
diff changeset
2263 if sorted(heads) != sorted(op.repo.heads()):
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2264 raise error.PushRaced(
43117
8ff1ecfadcd1 cleanup: join string literals that are already on one line
Martin von Zweigbergk <martinvonz@google.com>
parents: 43115
diff changeset
2265 b'remote repository changed while pushing - please try again'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2266 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2267
21130
1ff06386217f bundle2: introduce `replycaps` part for on-demand reply
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21129
diff changeset
2268
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2269 @parthandler(b'check:updated-heads')
32709
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32516
diff changeset
2270 def handlecheckupdatedheads(op, inpart):
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32516
diff changeset
2271 """check for race on the heads touched by a push
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32516
diff changeset
2272
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32516
diff changeset
2273 This is similar to 'check:heads' but focus on the heads actually updated
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32516
diff changeset
2274 during the push. If other activities happen on unrelated heads, it is
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32516
diff changeset
2275 ignored.
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32516
diff changeset
2276
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32516
diff changeset
2277 This allow server with high traffic to avoid push contention as long as
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32516
diff changeset
2278 unrelated parts of the graph are involved."""
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32516
diff changeset
2279 h = inpart.read(20)
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32516
diff changeset
2280 heads = []
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32516
diff changeset
2281 while len(h) == 20:
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32516
diff changeset
2282 heads.append(h)
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32516
diff changeset
2283 h = inpart.read(20)
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32516
diff changeset
2284 assert not h
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32516
diff changeset
2285 # trigger a transaction so that we are guaranteed to have the lock now.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2286 if op.ui.configbool(b'experimental', b'bundle2lazylocking'):
32709
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32516
diff changeset
2287 op.gettransaction()
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32516
diff changeset
2288
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32516
diff changeset
2289 currentheads = set()
42002
662ffdde5adf branchcache: rename itervalues() to iterheads()
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41792
diff changeset
2290 for ls in op.repo.branchmap().iterheads():
32709
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32516
diff changeset
2291 currentheads.update(ls)
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32516
diff changeset
2292
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32516
diff changeset
2293 for h in heads:
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32516
diff changeset
2294 if h not in currentheads:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2295 raise error.PushRaced(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2296 b'remote repository changed while pushing - '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2297 b'please try again'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2298 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2299
32709
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32516
diff changeset
2300
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2301 @parthandler(b'check:phases')
34820
a95067b1dca6 phase: introduce a new 'check:phases' part
Boris Feld <boris.feld@octobus.net>
parents: 34637
diff changeset
2302 def handlecheckphases(op, inpart):
a95067b1dca6 phase: introduce a new 'check:phases' part
Boris Feld <boris.feld@octobus.net>
parents: 34637
diff changeset
2303 """check that phase boundaries of the repository did not change
a95067b1dca6 phase: introduce a new 'check:phases' part
Boris Feld <boris.feld@octobus.net>
parents: 34637
diff changeset
2304
a95067b1dca6 phase: introduce a new 'check:phases' part
Boris Feld <boris.feld@octobus.net>
parents: 34637
diff changeset
2305 This is used to detect a push race.
a95067b1dca6 phase: introduce a new 'check:phases' part
Boris Feld <boris.feld@octobus.net>
parents: 34637
diff changeset
2306 """
a95067b1dca6 phase: introduce a new 'check:phases' part
Boris Feld <boris.feld@octobus.net>
parents: 34637
diff changeset
2307 phasetonodes = phases.binarydecode(inpart)
a95067b1dca6 phase: introduce a new 'check:phases' part
Boris Feld <boris.feld@octobus.net>
parents: 34637
diff changeset
2308 unfi = op.repo.unfiltered()
a95067b1dca6 phase: introduce a new 'check:phases' part
Boris Feld <boris.feld@octobus.net>
parents: 34637
diff changeset
2309 cl = unfi.changelog
a95067b1dca6 phase: introduce a new 'check:phases' part
Boris Feld <boris.feld@octobus.net>
parents: 34637
diff changeset
2310 phasecache = unfi._phasecache
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2311 msg = (
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2312 b'remote repository changed while pushing - please try again '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2313 b'(%s is %s expected %s)'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2314 )
48913
f254fc73d956 global: bulk replace simple pycompat.iteritems(x) with x.items()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48875
diff changeset
2315 for expectedphase, nodes in phasetonodes.items():
34820
a95067b1dca6 phase: introduce a new 'check:phases' part
Boris Feld <boris.feld@octobus.net>
parents: 34637
diff changeset
2316 for n in nodes:
a95067b1dca6 phase: introduce a new 'check:phases' part
Boris Feld <boris.feld@octobus.net>
parents: 34637
diff changeset
2317 actualphase = phasecache.phase(unfi, cl.rev(n))
a95067b1dca6 phase: introduce a new 'check:phases' part
Boris Feld <boris.feld@octobus.net>
parents: 34637
diff changeset
2318 if actualphase != expectedphase:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2319 finalmsg = msg % (
46113
59fa3890d40a node: import symbols explicitly
Joerg Sonnenberger <joerg@bec.de>
parents: 45942
diff changeset
2320 short(n),
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2321 phases.phasenames[actualphase],
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2322 phases.phasenames[expectedphase],
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2323 )
34820
a95067b1dca6 phase: introduce a new 'check:phases' part
Boris Feld <boris.feld@octobus.net>
parents: 34637
diff changeset
2324 raise error.PushRaced(finalmsg)
a95067b1dca6 phase: introduce a new 'check:phases' part
Boris Feld <boris.feld@octobus.net>
parents: 34637
diff changeset
2325
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2326
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2327 @parthandler(b'output')
21131
b7435117d951 bundle2: capture remote stdout while unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21130
diff changeset
2328 def handleoutput(op, inpart):
b7435117d951 bundle2: capture remote stdout while unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21130
diff changeset
2329 """forward output captured on the server to the client"""
b7435117d951 bundle2: capture remote stdout while unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21130
diff changeset
2330 for line in inpart.read().splitlines():
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2331 op.ui.status(_(b'remote: %s\n') % line)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2332
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2333
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2334 @parthandler(b'replycaps')
21130
1ff06386217f bundle2: introduce `replycaps` part for on-demand reply
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21129
diff changeset
2335 def handlereplycaps(op, inpart):
1ff06386217f bundle2: introduce `replycaps` part for on-demand reply
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21129
diff changeset
2336 """Notify that a reply bundle should be created
1ff06386217f bundle2: introduce `replycaps` part for on-demand reply
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21129
diff changeset
2337
21138
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
2338 The payload contains the capabilities information for the reply"""
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
2339 caps = decodecaps(inpart.read())
21130
1ff06386217f bundle2: introduce `replycaps` part for on-demand reply
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21129
diff changeset
2340 if op.reply is None:
21135
98fbf3adfd83 bundle2: add capabilities support in `replycaps` part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21134
diff changeset
2341 op.reply = bundle20(op.ui, caps)
21131
b7435117d951 bundle2: capture remote stdout while unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21130
diff changeset
2342
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2343
26829
58f1645f72c3 bundle2: attribute remote failures to remote (issue4788)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26802
diff changeset
2344 class AbortFromPart(error.Abort):
58f1645f72c3 bundle2: attribute remote failures to remote (issue4788)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26802
diff changeset
2345 """Sub-class of Abort that denotes an error from a bundle2 part."""
58f1645f72c3 bundle2: attribute remote failures to remote (issue4788)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26802
diff changeset
2346
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2347
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2348 @parthandler(b'error:abort', (b'message', b'hint'))
24741
bb67e52362d6 bundle2: fix names for error part handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24733
diff changeset
2349 def handleerrorabort(op, inpart):
21177
952af771bc17 bundle2: gracefully handle abort during unbundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21176
diff changeset
2350 """Used to transmit abort error over the wire"""
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2351 raise AbortFromPart(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2352 inpart.params[b'message'], hint=inpart.params.get(b'hint')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2353 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2354
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2355
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2356 @parthandler(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2357 b'error:pushkey',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2358 (b'namespace', b'key', b'new', b'old', b'ret', b'in-reply-to'),
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2359 )
25493
d8e7b0781ad7 bundle2: convey PushkeyFailed error over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25492
diff changeset
2360 def handleerrorpushkey(op, inpart):
d8e7b0781ad7 bundle2: convey PushkeyFailed error over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25492
diff changeset
2361 """Used to transmit failure of a mandatory pushkey over the wire"""
d8e7b0781ad7 bundle2: convey PushkeyFailed error over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25492
diff changeset
2362 kwargs = {}
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2363 for name in (b'namespace', b'key', b'new', b'old', b'ret'):
25493
d8e7b0781ad7 bundle2: convey PushkeyFailed error over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25492
diff changeset
2364 value = inpart.params.get(name)
d8e7b0781ad7 bundle2: convey PushkeyFailed error over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25492
diff changeset
2365 if value is not None:
d8e7b0781ad7 bundle2: convey PushkeyFailed error over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25492
diff changeset
2366 kwargs[name] = value
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2367 raise error.PushkeyFailed(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2368 inpart.params[b'in-reply-to'], **pycompat.strkwargs(kwargs)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2369 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2370
25493
d8e7b0781ad7 bundle2: convey PushkeyFailed error over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25492
diff changeset
2371
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2372 @parthandler(b'error:unsupportedcontent', (b'parttype', b'params'))
24741
bb67e52362d6 bundle2: fix names for error part handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24733
diff changeset
2373 def handleerrorunsupportedcontent(op, inpart):
21619
292331e906d7 bundle2: rename b2x:error:unknownpart to b2x:error:unsupportedcontent
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21618
diff changeset
2374 """Used to transmit unknown content error over the wire"""
21622
457492741007 bundle2: support transmission of params error over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21620
diff changeset
2375 kwargs = {}
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2376 parttype = inpart.params.get(b'parttype')
21627
3e8bcc90f07c bundle2: support None parttype in BundleValueError
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21626
diff changeset
2377 if parttype is not None:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2378 kwargs[b'parttype'] = parttype
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2379 params = inpart.params.get(b'params')
21622
457492741007 bundle2: support transmission of params error over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21620
diff changeset
2380 if params is not None:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2381 kwargs[b'params'] = params.split(b'\0')
21622
457492741007 bundle2: support transmission of params error over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21620
diff changeset
2382
36427
247b473f408e bundle2: **strkwargs love on various kwargs constructions
Augie Fackler <augie@google.com>
parents: 36273
diff changeset
2383 raise error.BundleUnknownFeatureError(**pycompat.strkwargs(kwargs))
21186
9f3652e851f8 bundle2: gracefully handle PushRaced error during unbundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21185
diff changeset
2384
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2385
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2386 @parthandler(b'error:pushraced', (b'message',))
24741
bb67e52362d6 bundle2: fix names for error part handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24733
diff changeset
2387 def handleerrorpushraced(op, inpart):
21186
9f3652e851f8 bundle2: gracefully handle PushRaced error during unbundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21185
diff changeset
2388 """Used to transmit push race error over the wire"""
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2389 raise error.ResponseError(_(b'push failed:'), inpart.params[b'message'])
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2390
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2391
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2392 @parthandler(b'listkeys', (b'namespace',))
21655
35095f332846 bundle: introduce a listkey handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21644
diff changeset
2393 def handlelistkeys(op, inpart):
35095f332846 bundle: introduce a listkey handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21644
diff changeset
2394 """retrieve pushkey namespace content stored in a bundle2"""
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2395 namespace = inpart.params[b'namespace']
21655
35095f332846 bundle: introduce a listkey handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21644
diff changeset
2396 r = pushkey.decodekeys(inpart.read())
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2397 op.records.add(b'listkeys', (namespace, r))
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2398
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2399
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2400 @parthandler(b'pushkey', (b'namespace', b'key', b'old', b'new'))
21660
e87d2a12d41b bundle2: add ``pushkey`` support
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21655
diff changeset
2401 def handlepushkey(op, inpart):
e87d2a12d41b bundle2: add ``pushkey`` support
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21655
diff changeset
2402 """process a pushkey request"""
e87d2a12d41b bundle2: add ``pushkey`` support
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21655
diff changeset
2403 dec = pushkey.decode
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2404 namespace = dec(inpart.params[b'namespace'])
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2405 key = dec(inpart.params[b'key'])
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2406 old = dec(inpart.params[b'old'])
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2407 new = dec(inpart.params[b'new'])
26565
ee1bcb9aa6e4 bundle2: add op.gettransaction() to handlers that need the lock
Durham Goode <durham@fb.com>
parents: 26542
diff changeset
2408 # Grab the transaction to ensure that we have the lock before performing the
ee1bcb9aa6e4 bundle2: add op.gettransaction() to handlers that need the lock
Durham Goode <durham@fb.com>
parents: 26542
diff changeset
2409 # pushkey.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2410 if op.ui.configbool(b'experimental', b'bundle2lazylocking'):
26565
ee1bcb9aa6e4 bundle2: add op.gettransaction() to handlers that need the lock
Durham Goode <durham@fb.com>
parents: 26542
diff changeset
2411 op.gettransaction()
21660
e87d2a12d41b bundle2: add ``pushkey`` support
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21655
diff changeset
2412 ret = op.repo.pushkey(namespace, key, old, new)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2413 record = {b'namespace': namespace, b'key': key, b'old': old, b'new': new}
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2414 op.records.add(b'pushkey', record)
21660
e87d2a12d41b bundle2: add ``pushkey`` support
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21655
diff changeset
2415 if op.reply is not None:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2416 rpart = op.reply.newpart(b'reply:pushkey')
33675
24b7c75c6441 bundle2: use bytestr() instead of str() to convert part id to bytes
Augie Fackler <augie@google.com>
parents: 33674
diff changeset
2417 rpart.addparam(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2418 b'in-reply-to', pycompat.bytestr(inpart.id), mandatory=False
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2419 )
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2420 rpart.addparam(b'return', b'%i' % ret, mandatory=False)
25481
6de96cb31176 bundle2: abort when a mandatory pushkey part fails
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25401
diff changeset
2421 if inpart.mandatory and not ret:
25484
a5192774e925 bundle2: introduce a PushkeyFail error to abort unbundle on pushkey error
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25481
diff changeset
2422 kwargs = {}
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2423 for key in (b'namespace', b'key', b'new', b'old', b'ret'):
25484
a5192774e925 bundle2: introduce a PushkeyFail error to abort unbundle on pushkey error
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25481
diff changeset
2424 if key in inpart.params:
a5192774e925 bundle2: introduce a PushkeyFail error to abort unbundle on pushkey error
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25481
diff changeset
2425 kwargs[key] = inpart.params[key]
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2426 raise error.PushkeyFailed(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2427 partid=b'%d' % inpart.id, **pycompat.strkwargs(kwargs)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2428 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2429
21660
e87d2a12d41b bundle2: add ``pushkey`` support
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21655
diff changeset
2430
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2431 @parthandler(b'bookmarks')
35260
af5507203d01 bookmark: introduce a 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35259
diff changeset
2432 def handlebookmark(op, inpart):
af5507203d01 bookmark: introduce a 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35259
diff changeset
2433 """transmit bookmark information
af5507203d01 bookmark: introduce a 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35259
diff changeset
2434
35266
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35265
diff changeset
2435 The part contains binary encoded bookmark information.
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35265
diff changeset
2436
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35265
diff changeset
2437 The exact behavior of this part can be controlled by the 'bookmarks' mode
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35265
diff changeset
2438 on the bundle operation.
35260
af5507203d01 bookmark: introduce a 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35259
diff changeset
2439
35266
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35265
diff changeset
2440 When mode is 'apply' (the default) the bookmark information is applied as
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35265
diff changeset
2441 is to the unbundling repository. Make sure a 'check:bookmarks' part is
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35265
diff changeset
2442 issued earlier to check for push races in such update. This behavior is
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35265
diff changeset
2443 suitable for pushing.
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35265
diff changeset
2444
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35265
diff changeset
2445 When mode is 'records', the information is recorded into the 'bookmarks'
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35265
diff changeset
2446 records of the bundle operation. This behavior is suitable for pulling.
35260
af5507203d01 bookmark: introduce a 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35259
diff changeset
2447 """
46780
6266d19556ad node: introduce nodeconstants class
Joerg Sonnenberger <joerg@bec.de>
parents: 46713
diff changeset
2448 changes = bookmarks.binarydecode(op.repo, inpart)
35261
f392066d127c bookmark: add pushkey hook compatiblity to the bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35260
diff changeset
2449
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2450 pushkeycompat = op.repo.ui.configbool(
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2451 b'server', b'bookmarks-pushkey-compat'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2452 )
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2453 bookmarksmode = op.modes.get(b'bookmarks', b'apply')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2454
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2455 if bookmarksmode == b'apply':
35266
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35265
diff changeset
2456 tr = op.gettransaction()
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35265
diff changeset
2457 bookstore = op.repo._bookmarks
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35265
diff changeset
2458 if pushkeycompat:
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35265
diff changeset
2459 allhooks = []
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35265
diff changeset
2460 for book, node in changes:
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35265
diff changeset
2461 hookargs = tr.hookargs.copy()
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2462 hookargs[b'pushkeycompat'] = b'1'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2463 hookargs[b'namespace'] = b'bookmarks'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2464 hookargs[b'key'] = book
46113
59fa3890d40a node: import symbols explicitly
Joerg Sonnenberger <joerg@bec.de>
parents: 45942
diff changeset
2465 hookargs[b'old'] = hex(bookstore.get(book, b''))
59fa3890d40a node: import symbols explicitly
Joerg Sonnenberger <joerg@bec.de>
parents: 45942
diff changeset
2466 hookargs[b'new'] = hex(node if node is not None else b'')
35266
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35265
diff changeset
2467 allhooks.append(hookargs)
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35265
diff changeset
2468
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35265
diff changeset
2469 for hookargs in allhooks:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2470 op.repo.hook(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2471 b'prepushkey', throw=True, **pycompat.strkwargs(hookargs)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2472 )
35266
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35265
diff changeset
2473
44372
8407031f195f bookmarks: prevent pushes of divergent bookmarks (foo@remote)
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 44256
diff changeset
2474 for book, node in changes:
8407031f195f bookmarks: prevent pushes of divergent bookmarks (foo@remote)
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 44256
diff changeset
2475 if bookmarks.isdivergent(book):
8407031f195f bookmarks: prevent pushes of divergent bookmarks (foo@remote)
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 44256
diff changeset
2476 msg = _(b'cannot accept divergent bookmark %s!') % book
8407031f195f bookmarks: prevent pushes of divergent bookmarks (foo@remote)
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 44256
diff changeset
2477 raise error.Abort(msg)
8407031f195f bookmarks: prevent pushes of divergent bookmarks (foo@remote)
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 44256
diff changeset
2478
35266
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35265
diff changeset
2479 bookstore.applychanges(op.repo, op.gettransaction(), changes)
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35265
diff changeset
2480
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35265
diff changeset
2481 if pushkeycompat:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2482
43778
888bd39ed555 lock: pass "success" boolean to _afterlock callbacks
Kyle Lippincott <spectral@google.com>
parents: 43506
diff changeset
2483 def runhook(unused_success):
35266
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35265
diff changeset
2484 for hookargs in allhooks:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2485 op.repo.hook(b'pushkey', **pycompat.strkwargs(hookargs))
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2486
35266
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35265
diff changeset
2487 op.repo._afterlock(runhook)
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35265
diff changeset
2488
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2489 elif bookmarksmode == b'records':
35261
f392066d127c bookmark: add pushkey hook compatiblity to the bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35260
diff changeset
2490 for book, node in changes:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2491 record = {b'bookmark': book, b'node': node}
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2492 op.records.add(b'bookmarks', record)
35266
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35265
diff changeset
2493 else:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2494 raise error.ProgrammingError(
48423
fff5942d445f typo: s/unkown/unknown across the codebase
Raphaël Gomès <rgomes@octobus.net>
parents: 47226
diff changeset
2495 b'unknown bookmark mode: %s' % bookmarksmode
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2496 )
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2497
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2498
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2499 @parthandler(b'phase-heads')
33031
e8c8d81eb864 bundle: add config option to include phases
Martin von Zweigbergk <martinvonz@google.com>
parents: 33030
diff changeset
2500 def handlephases(op, inpart):
e8c8d81eb864 bundle: add config option to include phases
Martin von Zweigbergk <martinvonz@google.com>
parents: 33030
diff changeset
2501 """apply phases from bundle part to repo"""
34320
12c42bcd4133 phases: move the binary decoding function in the phases module
Boris Feld <boris.feld@octobus.net>
parents: 34319
diff changeset
2502 headsbyphase = phases.binarydecode(inpart)
34321
4ef472b975ff bundle2: only grab a transaction when 'phase-heads' affect the repository
Boris Feld <boris.feld@octobus.net>
parents: 34320
diff changeset
2503 phases.updatephases(op.repo.unfiltered(), op.gettransaction, headsbyphase)
33031
e8c8d81eb864 bundle: add config option to include phases
Martin von Zweigbergk <martinvonz@google.com>
parents: 33030
diff changeset
2504
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2505
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2506 @parthandler(b'reply:pushkey', (b'return', b'in-reply-to'))
21660
e87d2a12d41b bundle2: add ``pushkey`` support
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21655
diff changeset
2507 def handlepushkeyreply(op, inpart):
e87d2a12d41b bundle2: add ``pushkey`` support
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21655
diff changeset
2508 """retrieve the result of a pushkey request"""
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2509 ret = int(inpart.params[b'return'])
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2510 partid = int(inpart.params[b'in-reply-to'])
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2511 op.records.add(b'pushkey', {b'return': ret}, partid)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2512
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2513
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2514 @parthandler(b'obsmarkers')
22336
60786c8a2f70 bundle2: add an obsmarkers part handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21900
diff changeset
2515 def handleobsmarker(op, inpart):
60786c8a2f70 bundle2: add an obsmarkers part handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21900
diff changeset
2516 """add a stream of obsmarkers to the repo"""
60786c8a2f70 bundle2: add an obsmarkers part handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21900
diff changeset
2517 tr = op.gettransaction()
24733
c00e4338fa4b obsolete: experimental flag to get debug about obsmarkers exchange
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24686
diff changeset
2518 markerdata = inpart.read()
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2519 if op.ui.config(b'experimental', b'obsmarkers-exchange-debug'):
43080
86e4daa2d54c cleanup: mark some ui.(status|note|warn|write) calls as not needing i18n
Augie Fackler <augie@google.com>
parents: 43077
diff changeset
2520 op.ui.writenoi18n(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2521 b'obsmarker-exchange: %i bytes received\n' % len(markerdata)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2522 )
26685
ea390d889d3a bundle2: gracefully skip 'obsmarkers' part if evolution is disabled
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26587
diff changeset
2523 # The mergemarkers call will crash if marker creation is not enabled.
ea390d889d3a bundle2: gracefully skip 'obsmarkers' part if evolution is disabled
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26587
diff changeset
2524 # we want to avoid this if the part is advisory.
ea390d889d3a bundle2: gracefully skip 'obsmarkers' part if evolution is disabled
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26587
diff changeset
2525 if not inpart.mandatory and op.repo.obsstore.readonly:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2526 op.repo.ui.debug(
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2527 b'ignoring obsolescence markers, feature not enabled\n'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2528 )
26685
ea390d889d3a bundle2: gracefully skip 'obsmarkers' part if evolution is disabled
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26587
diff changeset
2529 return
24733
c00e4338fa4b obsolete: experimental flag to get debug about obsmarkers exchange
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24686
diff changeset
2530 new = op.repo.obsstore.mergemarkers(tr, markerdata)
32326
99515353c72a obsolete: invalidate "volatile" set cache after merging marker
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32024
diff changeset
2531 op.repo.invalidatevolatilesets()
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2532 op.records.add(b'obsmarkers', {b'new': new})
22340
394a17de6a2d obsmarker: produce a reply part for markers received through bundle2
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22338
diff changeset
2533 if op.reply is not None:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2534 rpart = op.reply.newpart(b'reply:obsmarkers')
33675
24b7c75c6441 bundle2: use bytestr() instead of str() to convert part id to bytes
Augie Fackler <augie@google.com>
parents: 33674
diff changeset
2535 rpart.addparam(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2536 b'in-reply-to', pycompat.bytestr(inpart.id), mandatory=False
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2537 )
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2538 rpart.addparam(b'new', b'%i' % new, mandatory=False)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2539
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2540
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2541 @parthandler(b'reply:obsmarkers', (b'new', b'in-reply-to'))
25506
86472038dfd2 bundle2: rename duplicate handlepushkeyreply to handleobsmarkerreply
Martin von Zweigbergk <martinvonz@google.com>
parents: 25493
diff changeset
2542 def handleobsmarkerreply(op, inpart):
22340
394a17de6a2d obsmarker: produce a reply part for markers received through bundle2
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22338
diff changeset
2543 """retrieve the result of a pushkey request"""
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2544 ret = int(inpart.params[b'new'])
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2545 partid = int(inpart.params[b'in-reply-to'])
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2546 op.records.add(b'obsmarkers', {b'new': ret}, partid)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2547
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2548
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2549 @parthandler(b'hgtagsfnodes')
25401
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
2550 def handlehgtagsfnodes(op, inpart):
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
2551 """Applies .hgtags fnodes cache entries to the local repo.
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
2552
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
2553 Payload is pairs of 20 byte changeset nodes and filenodes.
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
2554 """
26565
ee1bcb9aa6e4 bundle2: add op.gettransaction() to handlers that need the lock
Durham Goode <durham@fb.com>
parents: 26542
diff changeset
2555 # Grab the transaction so we ensure that we have the lock at this point.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2556 if op.ui.configbool(b'experimental', b'bundle2lazylocking'):
26565
ee1bcb9aa6e4 bundle2: add op.gettransaction() to handlers that need the lock
Durham Goode <durham@fb.com>
parents: 26542
diff changeset
2557 op.gettransaction()
25401
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
2558 cache = tags.hgtagsfnodescache(op.repo.unfiltered())
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
2559
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
2560 count = 0
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
2561 while True:
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
2562 node = inpart.read(20)
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
2563 fnode = inpart.read(20)
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
2564 if len(node) < 20 or len(fnode) < 20:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2565 op.ui.debug(b'ignoring incomplete received .hgtags fnodes data\n')
25401
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
2566 break
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
2567 cache.setfnode(node, fnode)
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
2568 count += 1
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
2569
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
2570 cache.write()
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2571 op.ui.debug(b'applied %i hgtags fnodes cache entries\n' % count)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2572
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2573
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2574 rbcstruct = struct.Struct(b'>III')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2575
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2576
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2577 @parthandler(b'cache:rev-branch-cache')
36963
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part
Boris Feld <boris.feld@octobus.net>
parents: 36428
diff changeset
2578 def handlerbc(op, inpart):
46372
3e91d9978bec branchmap: update rev-branch-cache incrementally
Joerg Sonnenberger <joerg@bec.de>
parents: 46166
diff changeset
2579 """Legacy part, ignored for compatibility with bundles from or
3e91d9978bec branchmap: update rev-branch-cache incrementally
Joerg Sonnenberger <joerg@bec.de>
parents: 46166
diff changeset
2580 for Mercurial before 5.7. Newer Mercurial computes the cache
3e91d9978bec branchmap: update rev-branch-cache incrementally
Joerg Sonnenberger <joerg@bec.de>
parents: 46166
diff changeset
2581 efficiently enough during unbundling that the additional transfer
3e91d9978bec branchmap: update rev-branch-cache incrementally
Joerg Sonnenberger <joerg@bec.de>
parents: 46166
diff changeset
2582 is unnecessary."""
36963
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part
Boris Feld <boris.feld@octobus.net>
parents: 36428
diff changeset
2583
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2584
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2585 @parthandler(b'pushvars')
33693
db3dc11356ed pushvars: move fb extension pushvars to core
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33682
diff changeset
2586 def bundle2getvars(op, part):
db3dc11356ed pushvars: move fb extension pushvars to core
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33682
diff changeset
2587 '''unbundle a bundle2 containing shellvars on the server'''
db3dc11356ed pushvars: move fb extension pushvars to core
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33682
diff changeset
2588 # An option to disable unbundling on server-side for security reasons
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2589 if op.ui.configbool(b'push', b'pushvars.server'):
33693
db3dc11356ed pushvars: move fb extension pushvars to core
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33682
diff changeset
2590 hookargs = {}
db3dc11356ed pushvars: move fb extension pushvars to core
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33682
diff changeset
2591 for key, value in part.advisoryparams:
db3dc11356ed pushvars: move fb extension pushvars to core
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33682
diff changeset
2592 key = key.upper()
db3dc11356ed pushvars: move fb extension pushvars to core
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33682
diff changeset
2593 # We want pushed variables to have USERVAR_ prepended so we know
db3dc11356ed pushvars: move fb extension pushvars to core
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33682
diff changeset
2594 # they came from the --pushvar flag.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2595 key = b"USERVAR_" + key
33693
db3dc11356ed pushvars: move fb extension pushvars to core
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33682
diff changeset
2596 hookargs[key] = value
db3dc11356ed pushvars: move fb extension pushvars to core
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33682
diff changeset
2597 op.addhookargs(hookargs)
35758
b996ddf5963d bundle2: add a 'stream' part handler for stream cloning
Boris Feld <boris.feld@octobus.net>
parents: 35655
diff changeset
2598
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2599
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2600 @parthandler(b'stream2', (b'requirements', b'filecount', b'bytecount'))
35788
b116a66bcc44 bundle2: move version of stream clone into part name
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35787
diff changeset
2601 def handlestreamv2bundle(op, part):
35758
b996ddf5963d bundle2: add a 'stream' part handler for stream cloning
Boris Feld <boris.feld@octobus.net>
parents: 35655
diff changeset
2602
48990
d9ed7c5e915d streamclone: avoid some obscure error in a corner case
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents: 48653
diff changeset
2603 requirements = urlreq.unquote(part.params[b'requirements'])
d9ed7c5e915d streamclone: avoid some obscure error in a corner case
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents: 48653
diff changeset
2604 requirements = requirements.split(b',') if requirements else []
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2605 filecount = int(part.params[b'filecount'])
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2606 bytecount = int(part.params[b'bytecount'])
35758
b996ddf5963d bundle2: add a 'stream' part handler for stream cloning
Boris Feld <boris.feld@octobus.net>
parents: 35655
diff changeset
2607
b996ddf5963d bundle2: add a 'stream' part handler for stream cloning
Boris Feld <boris.feld@octobus.net>
parents: 35655
diff changeset
2608 repo = op.repo
b996ddf5963d bundle2: add a 'stream' part handler for stream cloning
Boris Feld <boris.feld@octobus.net>
parents: 35655
diff changeset
2609 if len(repo):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2610 msg = _(b'cannot apply stream clone to non empty repository')
35758
b996ddf5963d bundle2: add a 'stream' part handler for stream cloning
Boris Feld <boris.feld@octobus.net>
parents: 35655
diff changeset
2611 raise error.Abort(msg)
b996ddf5963d bundle2: add a 'stream' part handler for stream cloning
Boris Feld <boris.feld@octobus.net>
parents: 35655
diff changeset
2612
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2613 repo.ui.debug(b'applying stream bundle\n')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2614 streamclone.applybundlev2(repo, part, filecount, bytecount, requirements)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2615
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2616
50706
0452af304808 stream-clone: add a v3 version of the protocol
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 50616
diff changeset
2617 @parthandler(b'stream3-exp', (b'requirements',))
50524
58adcabc295f stream-clone: introduce the notion of an experimental "v3" version
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50519
diff changeset
2618 def handlestreamv3bundle(op, part):
50706
0452af304808 stream-clone: add a v3 version of the protocol
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 50616
diff changeset
2619 requirements = urlreq.unquote(part.params[b'requirements'])
0452af304808 stream-clone: add a v3 version of the protocol
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 50616
diff changeset
2620 requirements = requirements.split(b',') if requirements else []
0452af304808 stream-clone: add a v3 version of the protocol
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 50616
diff changeset
2621
0452af304808 stream-clone: add a v3 version of the protocol
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 50616
diff changeset
2622 repo = op.repo
0452af304808 stream-clone: add a v3 version of the protocol
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 50616
diff changeset
2623 if len(repo):
0452af304808 stream-clone: add a v3 version of the protocol
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 50616
diff changeset
2624 msg = _(b'cannot apply stream clone to non empty repository')
0452af304808 stream-clone: add a v3 version of the protocol
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 50616
diff changeset
2625 raise error.Abort(msg)
0452af304808 stream-clone: add a v3 version of the protocol
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 50616
diff changeset
2626
0452af304808 stream-clone: add a v3 version of the protocol
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 50616
diff changeset
2627 repo.ui.debug(b'applying stream bundle\n')
0452af304808 stream-clone: add a v3 version of the protocol
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 50616
diff changeset
2628 streamclone.applybundlev3(repo, part, requirements)
50524
58adcabc295f stream-clone: introduce the notion of an experimental "v3" version
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50519
diff changeset
2629
58adcabc295f stream-clone: introduce the notion of an experimental "v3" version
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50519
diff changeset
2630
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2631 def widen_bundle(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2632 bundler, repo, oldmatcher, newmatcher, common, known, cgversion, ellipses
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2633 ):
40072
1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39850
diff changeset
2634 """generates bundle2 for widening a narrow clone
1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39850
diff changeset
2635
42417
a97b12f726e4 narrow: pass the bundle to bundle2.widen_bundle() instead of generating there
Pulkit Goyal <7895pulkit@gmail.com>
parents: 42143
diff changeset
2636 bundler is the bundle to which data should be added
40072
1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39850
diff changeset
2637 repo is the localrepository instance
40344
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40338
diff changeset
2638 oldmatcher matches what the client already has
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40338
diff changeset
2639 newmatcher matches what the client needs (including what it already has)
40072
1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39850
diff changeset
2640 common is set of common heads between server and client
1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39850
diff changeset
2641 known is a set of revs known on the client side (used in ellipses)
1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39850
diff changeset
2642 cgversion is the changegroup version to send
1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39850
diff changeset
2643 ellipses is boolean value telling whether to send ellipses data or not
1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39850
diff changeset
2644
1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39850
diff changeset
2645 returns bundle2 of the data required for extending
1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39850
diff changeset
2646 """
1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39850
diff changeset
2647 commonnodes = set()
1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39850
diff changeset
2648 cl = repo.changelog
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2649 for r in repo.revs(b"::%ln", common):
40072
1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39850
diff changeset
2650 commonnodes.add(cl.node(r))
1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39850
diff changeset
2651 if commonnodes:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2652 packer = changegroup.getbundler(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2653 cgversion,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2654 repo,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2655 oldmatcher=oldmatcher,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2656 matcher=newmatcher,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2657 fullnodes=commonnodes,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2658 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2659 cgdata = packer.generate(
47012
d55b71393907 node: replace nullid and friends with nodeconstants class
Joerg Sonnenberger <joerg@bec.de>
parents: 46907
diff changeset
2660 {repo.nullid},
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2661 list(commonnodes),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2662 False,
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2663 b'narrow_widen',
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2664 changelog=False,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42931
diff changeset
2665 )
40072
1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39850
diff changeset
2666
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2667 part = bundler.newpart(b'changegroup', data=cgdata)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2668 part.addparam(b'version', cgversion)
45552
10284ce3d5ed scmutil: introduce function to check whether repo uses treemanifest or not
Pulkit Goyal <7895pulkit@gmail.com>
parents: 45372
diff changeset
2669 if scmutil.istreemanifest(repo):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
2670 part.addparam(b'treemanifest', b'1')
47226
19d4802cb304 sidedata: add a 'side-data' repository feature and use it
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47128
diff changeset
2671 if repository.REPO_FEATURE_SIDE_DATA in repo.features:
19d4802cb304 sidedata: add a 'side-data' repository feature and use it
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47128
diff changeset
2672 part.addparam(b'exp-sidedata', b'1')
19d4802cb304 sidedata: add a 'side-data' repository feature and use it
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47128
diff changeset
2673 wanted = format_remote_wanted_sidedata(repo)
19d4802cb304 sidedata: add a 'side-data' repository feature and use it
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47128
diff changeset
2674 part.addparam(b'exp-wanted-sidedata', wanted)
40072
1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39850
diff changeset
2675
1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39850
diff changeset
2676 return bundler