Mercurial > hg
annotate mercurial/bundle2.py @ 29268:f200b58497f1
sslutil: reference appropriate config section in messaging
Error messages reference the config section defining the host
fingerprint. Now that we have multiple sections where this config
setting could live, we need to point the user at the appropriate
one.
We default to the new "hostsecurity" section. But we will still
refer them to the "hostfingerprint" section if a value is defined
there.
There are some corner cases where the messaging might be off. e.g.
they could define a SHA-1 fingerprint in both sections. IMO the
messaging needs a massive overhaul. I plan to do this as part
of future refactoring to security settings.
author | Gregory Szorc <gregory.szorc@gmail.com> |
---|---|
date | Sat, 28 May 2016 12:58:46 -0700 |
parents | 032c4c2f802a |
children | 077d0535f51f |
rev | line source |
---|---|
20801
9c5183cb9bca
bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff
changeset
|
1 # bundle2.py - generic container format to transmit arbitrary data. |
9c5183cb9bca
bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff
changeset
|
2 # |
9c5183cb9bca
bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff
changeset
|
3 # Copyright 2013 Facebook, Inc. |
9c5183cb9bca
bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff
changeset
|
4 # |
9c5183cb9bca
bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff
changeset
|
5 # This software may be used and distributed according to the terms of the |
9c5183cb9bca
bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff
changeset
|
6 # GNU General Public License version 2 or any later version. |
9c5183cb9bca
bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff
changeset
|
7 """Handling of the new bundle2 format |
9c5183cb9bca
bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff
changeset
|
8 |
9c5183cb9bca
bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff
changeset
|
9 The goal of bundle2 is to act as an atomically packet to transmit a set of |
9c5183cb9bca
bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff
changeset
|
10 payloads in an application agnostic way. It consist in a sequence of "parts" |
9c5183cb9bca
bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff
changeset
|
11 that will be handed to and processed by the application layer. |
9c5183cb9bca
bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff
changeset
|
12 |
9c5183cb9bca
bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff
changeset
|
13 |
9c5183cb9bca
bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff
changeset
|
14 General format architecture |
9c5183cb9bca
bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff
changeset
|
15 =========================== |
9c5183cb9bca
bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff
changeset
|
16 |
9c5183cb9bca
bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff
changeset
|
17 The format is architectured as follow |
9c5183cb9bca
bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff
changeset
|
18 |
9c5183cb9bca
bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff
changeset
|
19 - magic string |
9c5183cb9bca
bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff
changeset
|
20 - stream level parameters |
9c5183cb9bca
bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff
changeset
|
21 - payload parts (any number) |
9c5183cb9bca
bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff
changeset
|
22 - end of stream marker. |
9c5183cb9bca
bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff
changeset
|
23 |
20856
8a6a86c9a5b5
bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20844
diff
changeset
|
24 the Binary format |
20801
9c5183cb9bca
bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff
changeset
|
25 ============================ |
9c5183cb9bca
bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff
changeset
|
26 |
21024
7731a2281cf0
spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents:
21020
diff
changeset
|
27 All numbers are unsigned and big-endian. |
20801
9c5183cb9bca
bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff
changeset
|
28 |
9c5183cb9bca
bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff
changeset
|
29 stream level parameters |
9c5183cb9bca
bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff
changeset
|
30 ------------------------ |
9c5183cb9bca
bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff
changeset
|
31 |
9c5183cb9bca
bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff
changeset
|
32 Binary format is as follow |
9c5183cb9bca
bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff
changeset
|
33 |
23009
90f86ad3d4ff
bundle2: change header size and make them signed (new format)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23008
diff
changeset
|
34 :params size: int32 |
20801
9c5183cb9bca
bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff
changeset
|
35 |
9c5183cb9bca
bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff
changeset
|
36 The total number of Bytes used by the parameters |
9c5183cb9bca
bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff
changeset
|
37 |
9c5183cb9bca
bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff
changeset
|
38 :params value: arbitrary number of Bytes |
9c5183cb9bca
bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff
changeset
|
39 |
9c5183cb9bca
bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff
changeset
|
40 A blob of `params size` containing the serialized version of all stream level |
9c5183cb9bca
bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff
changeset
|
41 parameters. |
9c5183cb9bca
bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff
changeset
|
42 |
21024
7731a2281cf0
spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents:
21020
diff
changeset
|
43 The blob contains a space separated list of parameters. Parameters with value |
20811
9785c3f8f598
bundle2: urlquote stream parameter name and value
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20810
diff
changeset
|
44 are stored in the form `<name>=<value>`. Both name and value are urlquoted. |
20804
db9d3991d2c6
bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20803
diff
changeset
|
45 |
20813
8c74b3ce5b70
bundle2: refuse empty parameter name
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20812
diff
changeset
|
46 Empty name are obviously forbidden. |
8c74b3ce5b70
bundle2: refuse empty parameter name
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20812
diff
changeset
|
47 |
20844
2631204d7305
bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20843
diff
changeset
|
48 Name MUST start with a letter. If this first letter is lower case, the |
21024
7731a2281cf0
spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents:
21020
diff
changeset
|
49 parameter is advisory and can be safely ignored. However when the first |
20844
2631204d7305
bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20843
diff
changeset
|
50 letter is capital, the parameter is mandatory and the bundling process MUST |
2631204d7305
bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20843
diff
changeset
|
51 stop if he is not able to proceed it. |
20814
8532f5e1b9df
bundle2: force the first char of parameter to be an letter.
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20813
diff
changeset
|
52 |
20808
4c9130c7a29f
bundle2: clarify stream parameter design in the documentation
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20805
diff
changeset
|
53 Stream parameters use a simple textual format for two main reasons: |
20804
db9d3991d2c6
bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20803
diff
changeset
|
54 |
21024
7731a2281cf0
spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents:
21020
diff
changeset
|
55 - Stream level parameters should remain simple and we want to discourage any |
20808
4c9130c7a29f
bundle2: clarify stream parameter design in the documentation
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20805
diff
changeset
|
56 crazy usage. |
21024
7731a2281cf0
spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents:
21020
diff
changeset
|
57 - Textual data allow easy human inspection of a bundle2 header in case of |
20808
4c9130c7a29f
bundle2: clarify stream parameter design in the documentation
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20805
diff
changeset
|
58 troubles. |
4c9130c7a29f
bundle2: clarify stream parameter design in the documentation
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20805
diff
changeset
|
59 |
4c9130c7a29f
bundle2: clarify stream parameter design in the documentation
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20805
diff
changeset
|
60 Any Applicative level options MUST go into a bundle2 part instead. |
20801
9c5183cb9bca
bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff
changeset
|
61 |
9c5183cb9bca
bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff
changeset
|
62 Payload part |
9c5183cb9bca
bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff
changeset
|
63 ------------------------ |
9c5183cb9bca
bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff
changeset
|
64 |
9c5183cb9bca
bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff
changeset
|
65 Binary format is as follow |
9c5183cb9bca
bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff
changeset
|
66 |
23009
90f86ad3d4ff
bundle2: change header size and make them signed (new format)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23008
diff
changeset
|
67 :header size: int32 |
20801
9c5183cb9bca
bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff
changeset
|
68 |
25507
5bee483705e8
bundle2: clarify in docstring that header size is for a single header
Martin von Zweigbergk <martinvonz@google.com>
parents:
25506
diff
changeset
|
69 The total number of Bytes used by the part header. When the header is empty |
20801
9c5183cb9bca
bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff
changeset
|
70 (size = 0) this is interpreted as the end of stream marker. |
9c5183cb9bca
bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff
changeset
|
71 |
20856
8a6a86c9a5b5
bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20844
diff
changeset
|
72 :header: |
8a6a86c9a5b5
bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20844
diff
changeset
|
73 |
8a6a86c9a5b5
bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20844
diff
changeset
|
74 The header defines how to interpret the part. It contains two piece of |
8a6a86c9a5b5
bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20844
diff
changeset
|
75 data: the part type, and the part parameters. |
8a6a86c9a5b5
bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20844
diff
changeset
|
76 |
8a6a86c9a5b5
bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20844
diff
changeset
|
77 The part type is used to route an application level handler, that can |
8a6a86c9a5b5
bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20844
diff
changeset
|
78 interpret payload. |
8a6a86c9a5b5
bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20844
diff
changeset
|
79 |
8a6a86c9a5b5
bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20844
diff
changeset
|
80 Part parameters are passed to the application level handler. They are |
8a6a86c9a5b5
bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20844
diff
changeset
|
81 meant to convey information that will help the application level object to |
8a6a86c9a5b5
bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20844
diff
changeset
|
82 interpret the part payload. |
8a6a86c9a5b5
bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20844
diff
changeset
|
83 |
8a6a86c9a5b5
bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20844
diff
changeset
|
84 The binary format of the header is has follow |
8a6a86c9a5b5
bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20844
diff
changeset
|
85 |
8a6a86c9a5b5
bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20844
diff
changeset
|
86 :typesize: (one byte) |
20877
9e9e3a4e9261
bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20876
diff
changeset
|
87 |
23916
a3f7c781786b
bundle2: fix parttype enforcement
Matt Mackall <mpm@selenic.com>
parents:
23868
diff
changeset
|
88 :parttype: alphanumerical part name (restricted to [a-zA-Z0-9_:-]*) |
20877
9e9e3a4e9261
bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20876
diff
changeset
|
89 |
20995
e995d104c87f
bundle2: add an integer id to part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20952
diff
changeset
|
90 :partid: A 32bits integer (unique in the bundle) that can be used to refer |
e995d104c87f
bundle2: add an integer id to part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20952
diff
changeset
|
91 to this part. |
e995d104c87f
bundle2: add an integer id to part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20952
diff
changeset
|
92 |
20877
9e9e3a4e9261
bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20876
diff
changeset
|
93 :parameters: |
9e9e3a4e9261
bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20876
diff
changeset
|
94 |
21024
7731a2281cf0
spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents:
21020
diff
changeset
|
95 Part's parameter may have arbitrary content, the binary structure is:: |
20877
9e9e3a4e9261
bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20876
diff
changeset
|
96 |
9e9e3a4e9261
bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20876
diff
changeset
|
97 <mandatory-count><advisory-count><param-sizes><param-data> |
9e9e3a4e9261
bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20876
diff
changeset
|
98 |
9e9e3a4e9261
bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20876
diff
changeset
|
99 :mandatory-count: 1 byte, number of mandatory parameters |
9e9e3a4e9261
bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20876
diff
changeset
|
100 |
9e9e3a4e9261
bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20876
diff
changeset
|
101 :advisory-count: 1 byte, number of advisory parameters |
9e9e3a4e9261
bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20876
diff
changeset
|
102 |
9e9e3a4e9261
bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20876
diff
changeset
|
103 :param-sizes: |
9e9e3a4e9261
bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20876
diff
changeset
|
104 |
9e9e3a4e9261
bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20876
diff
changeset
|
105 N couple of bytes, where N is the total number of parameters. Each |
9e9e3a4e9261
bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20876
diff
changeset
|
106 couple contains (<size-of-key>, <size-of-value) for one parameter. |
9e9e3a4e9261
bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20876
diff
changeset
|
107 |
9e9e3a4e9261
bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20876
diff
changeset
|
108 :param-data: |
9e9e3a4e9261
bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20876
diff
changeset
|
109 |
9e9e3a4e9261
bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20876
diff
changeset
|
110 A blob of bytes from which each parameter key and value can be |
9e9e3a4e9261
bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20876
diff
changeset
|
111 retrieved using the list of size couples stored in the previous |
9e9e3a4e9261
bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20876
diff
changeset
|
112 field. |
9e9e3a4e9261
bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20876
diff
changeset
|
113 |
9e9e3a4e9261
bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20876
diff
changeset
|
114 Mandatory parameters comes first, then the advisory ones. |
20856
8a6a86c9a5b5
bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20844
diff
changeset
|
115 |
21607
054fa5176fa7
bundle2: forbid duplicate parameter keys
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21606
diff
changeset
|
116 Each parameter's key MUST be unique within the part. |
054fa5176fa7
bundle2: forbid duplicate parameter keys
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21606
diff
changeset
|
117 |
20856
8a6a86c9a5b5
bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20844
diff
changeset
|
118 :payload: |
8a6a86c9a5b5
bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20844
diff
changeset
|
119 |
20876
ddd56f3eb786
bundle2: support for bundling and unbundling payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20864
diff
changeset
|
120 payload is a series of `<chunksize><chunkdata>`. |
ddd56f3eb786
bundle2: support for bundling and unbundling payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20864
diff
changeset
|
121 |
23009
90f86ad3d4ff
bundle2: change header size and make them signed (new format)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23008
diff
changeset
|
122 `chunksize` is an int32, `chunkdata` are plain bytes (as much as |
20876
ddd56f3eb786
bundle2: support for bundling and unbundling payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20864
diff
changeset
|
123 `chunksize` says)` The payload part is concluded by a zero size chunk. |
ddd56f3eb786
bundle2: support for bundling and unbundling payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20864
diff
changeset
|
124 |
ddd56f3eb786
bundle2: support for bundling and unbundling payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20864
diff
changeset
|
125 The current implementation always produces either zero or one chunk. |
21024
7731a2281cf0
spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents:
21020
diff
changeset
|
126 This is an implementation limitation that will ultimately be lifted. |
20891
1c6cd23fc221
bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20890
diff
changeset
|
127 |
23009
90f86ad3d4ff
bundle2: change header size and make them signed (new format)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23008
diff
changeset
|
128 `chunksize` can be negative to trigger special case processing. No such |
90f86ad3d4ff
bundle2: change header size and make them signed (new format)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23008
diff
changeset
|
129 processing is in place yet. |
90f86ad3d4ff
bundle2: change header size and make them signed (new format)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23008
diff
changeset
|
130 |
20891
1c6cd23fc221
bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20890
diff
changeset
|
131 Bundle processing |
1c6cd23fc221
bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20890
diff
changeset
|
132 ============================ |
1c6cd23fc221
bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20890
diff
changeset
|
133 |
1c6cd23fc221
bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20890
diff
changeset
|
134 Each part is processed in order using a "part handler". Handler are registered |
1c6cd23fc221
bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20890
diff
changeset
|
135 for a certain part type. |
1c6cd23fc221
bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20890
diff
changeset
|
136 |
1c6cd23fc221
bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20890
diff
changeset
|
137 The matching of a part to its handler is case insensitive. The case of the |
1c6cd23fc221
bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20890
diff
changeset
|
138 part type is used to know if a part is mandatory or advisory. If the Part type |
1c6cd23fc221
bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20890
diff
changeset
|
139 contains any uppercase char it is considered mandatory. When no handler is |
1c6cd23fc221
bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20890
diff
changeset
|
140 known for a Mandatory part, the process is aborted and an exception is raised. |
20892
6fe95448596d
bundle2: read the whole bundle from stream on abort
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20891
diff
changeset
|
141 If the part is advisory and no handler is known, the part is ignored. When the |
6fe95448596d
bundle2: read the whole bundle from stream on abort
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20891
diff
changeset
|
142 process is aborted, the full bundle is still read from the stream to keep the |
6fe95448596d
bundle2: read the whole bundle from stream on abort
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20891
diff
changeset
|
143 channel usable. But none of the part read from an abort are processed. In the |
6fe95448596d
bundle2: read the whole bundle from stream on abort
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20891
diff
changeset
|
144 future, dropping the stream may become an option for channel we do not care to |
6fe95448596d
bundle2: read the whole bundle from stream on abort
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20891
diff
changeset
|
145 preserve. |
20801
9c5183cb9bca
bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff
changeset
|
146 """ |
9c5183cb9bca
bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff
changeset
|
147 |
25919
8221fefaea08
bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25660
diff
changeset
|
148 from __future__ import absolute_import |
8221fefaea08
bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25660
diff
changeset
|
149 |
24026
3daef83a1873
bundle2.unpackermixin: control for underlying file descriptor
Eric Sumner <ericsumner@fb.com>
parents:
23916
diff
changeset
|
150 import errno |
25919
8221fefaea08
bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25660
diff
changeset
|
151 import re |
20814
8532f5e1b9df
bundle2: force the first char of parameter to be an letter.
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20813
diff
changeset
|
152 import string |
25919
8221fefaea08
bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25660
diff
changeset
|
153 import struct |
8221fefaea08
bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25660
diff
changeset
|
154 import sys |
20804
db9d3991d2c6
bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20803
diff
changeset
|
155 |
25919
8221fefaea08
bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25660
diff
changeset
|
156 from .i18n import _ |
8221fefaea08
bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25660
diff
changeset
|
157 from . import ( |
8221fefaea08
bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25660
diff
changeset
|
158 changegroup, |
8221fefaea08
bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25660
diff
changeset
|
159 error, |
8221fefaea08
bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25660
diff
changeset
|
160 obsolete, |
8221fefaea08
bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25660
diff
changeset
|
161 pushkey, |
8221fefaea08
bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25660
diff
changeset
|
162 tags, |
8221fefaea08
bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25660
diff
changeset
|
163 url, |
8221fefaea08
bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25660
diff
changeset
|
164 util, |
8221fefaea08
bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25660
diff
changeset
|
165 ) |
20802
520df53ad26a
bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20801
diff
changeset
|
166 |
28883
032c4c2f802a
pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents:
28672
diff
changeset
|
167 urlerr = util.urlerr |
032c4c2f802a
pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents:
28672
diff
changeset
|
168 urlreq = util.urlreq |
032c4c2f802a
pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents:
28672
diff
changeset
|
169 |
20804
db9d3991d2c6
bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20803
diff
changeset
|
170 _pack = struct.pack |
db9d3991d2c6
bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20803
diff
changeset
|
171 _unpack = struct.unpack |
db9d3991d2c6
bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20803
diff
changeset
|
172 |
23009
90f86ad3d4ff
bundle2: change header size and make them signed (new format)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23008
diff
changeset
|
173 _fstreamparamsize = '>i' |
90f86ad3d4ff
bundle2: change header size and make them signed (new format)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23008
diff
changeset
|
174 _fpartheadersize = '>i' |
20856
8a6a86c9a5b5
bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20844
diff
changeset
|
175 _fparttypesize = '>B' |
20995
e995d104c87f
bundle2: add an integer id to part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20952
diff
changeset
|
176 _fpartid = '>I' |
23009
90f86ad3d4ff
bundle2: change header size and make them signed (new format)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23008
diff
changeset
|
177 _fpayloadsize = '>i' |
20877
9e9e3a4e9261
bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20876
diff
changeset
|
178 _fpartparamcount = '>BB' |
9e9e3a4e9261
bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20876
diff
changeset
|
179 |
21001
c93bb6a08fa1
bundle2: support chunk iterator as part data
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21000
diff
changeset
|
180 preferedchunksize = 4096 |
c93bb6a08fa1
bundle2: support chunk iterator as part data
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21000
diff
changeset
|
181 |
23868
405eacbecc52
bundle2: enforce parttype as alphanumerical
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23590
diff
changeset
|
182 _parttypeforbidden = re.compile('[^a-zA-Z0-9_:-]') |
405eacbecc52
bundle2: enforce parttype as alphanumerical
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23590
diff
changeset
|
183 |
25313
8f2c362bcb58
bundle2: introduce a specific function for bundling debug message
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25181
diff
changeset
|
184 def outdebug(ui, message): |
8f2c362bcb58
bundle2: introduce a specific function for bundling debug message
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25181
diff
changeset
|
185 """debug regarding output stream (bundling)""" |
25336
7c60a42265fb
bundle2: hide bundle2 stream debug under a config flag
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25335
diff
changeset
|
186 if ui.configbool('devel', 'bundle2.debug', False): |
7c60a42265fb
bundle2: hide bundle2 stream debug under a config flag
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25335
diff
changeset
|
187 ui.debug('bundle2-output: %s\n' % message) |
25313
8f2c362bcb58
bundle2: introduce a specific function for bundling debug message
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25181
diff
changeset
|
188 |
25318
64dd8f25fcf4
bundle2: introduce a specific function for debug messages while unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25317
diff
changeset
|
189 def indebug(ui, message): |
64dd8f25fcf4
bundle2: introduce a specific function for debug messages while unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25317
diff
changeset
|
190 """debug on input stream (unbundling)""" |
25336
7c60a42265fb
bundle2: hide bundle2 stream debug under a config flag
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25335
diff
changeset
|
191 if ui.configbool('devel', 'bundle2.debug', False): |
7c60a42265fb
bundle2: hide bundle2 stream debug under a config flag
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25335
diff
changeset
|
192 ui.debug('bundle2-input: %s\n' % message) |
25318
64dd8f25fcf4
bundle2: introduce a specific function for debug messages while unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25317
diff
changeset
|
193 |
23868
405eacbecc52
bundle2: enforce parttype as alphanumerical
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23590
diff
changeset
|
194 def validateparttype(parttype): |
405eacbecc52
bundle2: enforce parttype as alphanumerical
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23590
diff
changeset
|
195 """raise ValueError if a parttype contains invalid character""" |
23916
a3f7c781786b
bundle2: fix parttype enforcement
Matt Mackall <mpm@selenic.com>
parents:
23868
diff
changeset
|
196 if _parttypeforbidden.search(parttype): |
23868
405eacbecc52
bundle2: enforce parttype as alphanumerical
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23590
diff
changeset
|
197 raise ValueError(parttype) |
405eacbecc52
bundle2: enforce parttype as alphanumerical
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23590
diff
changeset
|
198 |
20877
9e9e3a4e9261
bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20876
diff
changeset
|
199 def _makefpartparamsizes(nbparams): |
9e9e3a4e9261
bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20876
diff
changeset
|
200 """return a struct format to read part parameter sizes |
9e9e3a4e9261
bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20876
diff
changeset
|
201 |
9e9e3a4e9261
bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20876
diff
changeset
|
202 The number parameters is variable so we need to build that format |
9e9e3a4e9261
bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20876
diff
changeset
|
203 dynamically. |
9e9e3a4e9261
bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20876
diff
changeset
|
204 """ |
9e9e3a4e9261
bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20876
diff
changeset
|
205 return '>'+('BB'*nbparams) |
20804
db9d3991d2c6
bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20803
diff
changeset
|
206 |
20890
ec7fc110faee
bundle2: introduce a `parthandler` decorator
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20889
diff
changeset
|
207 parthandlermapping = {} |
20889
deed5edb72de
bundle2: first version of a bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20887
diff
changeset
|
208 |
21623
5b26d82e4e2a
bundle2: make it possible to declare params handled by a part handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21622
diff
changeset
|
209 def parthandler(parttype, params=()): |
20890
ec7fc110faee
bundle2: introduce a `parthandler` decorator
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20889
diff
changeset
|
210 """decorator that register a function as a bundle2 part handler |
ec7fc110faee
bundle2: introduce a `parthandler` decorator
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20889
diff
changeset
|
211 |
ec7fc110faee
bundle2: introduce a `parthandler` decorator
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20889
diff
changeset
|
212 eg:: |
ec7fc110faee
bundle2: introduce a `parthandler` decorator
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20889
diff
changeset
|
213 |
21624
d61066d787c8
bundle2: declare supported parameters for all handlers
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21623
diff
changeset
|
214 @parthandler('myparttype', ('mandatory', 'param', 'handled')) |
20890
ec7fc110faee
bundle2: introduce a `parthandler` decorator
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20889
diff
changeset
|
215 def myparttypehandler(...): |
ec7fc110faee
bundle2: introduce a `parthandler` decorator
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20889
diff
changeset
|
216 '''process a part of type "my part".''' |
ec7fc110faee
bundle2: introduce a `parthandler` decorator
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20889
diff
changeset
|
217 ... |
ec7fc110faee
bundle2: introduce a `parthandler` decorator
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20889
diff
changeset
|
218 """ |
23868
405eacbecc52
bundle2: enforce parttype as alphanumerical
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23590
diff
changeset
|
219 validateparttype(parttype) |
20890
ec7fc110faee
bundle2: introduce a `parthandler` decorator
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20889
diff
changeset
|
220 def _decorator(func): |
20891
1c6cd23fc221
bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20890
diff
changeset
|
221 lparttype = parttype.lower() # enforce lower case matching. |
1c6cd23fc221
bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20890
diff
changeset
|
222 assert lparttype not in parthandlermapping |
1c6cd23fc221
bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20890
diff
changeset
|
223 parthandlermapping[lparttype] = func |
21623
5b26d82e4e2a
bundle2: make it possible to declare params handled by a part handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21622
diff
changeset
|
224 func.params = frozenset(params) |
20890
ec7fc110faee
bundle2: introduce a `parthandler` decorator
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20889
diff
changeset
|
225 return func |
ec7fc110faee
bundle2: introduce a `parthandler` decorator
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20889
diff
changeset
|
226 return _decorator |
20889
deed5edb72de
bundle2: first version of a bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20887
diff
changeset
|
227 |
20949
571f2903ff1e
bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20948
diff
changeset
|
228 class unbundlerecords(object): |
571f2903ff1e
bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20948
diff
changeset
|
229 """keep record of what happens during and unbundle |
571f2903ff1e
bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20948
diff
changeset
|
230 |
571f2903ff1e
bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20948
diff
changeset
|
231 New records are added using `records.add('cat', obj)`. Where 'cat' is a |
21024
7731a2281cf0
spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents:
21020
diff
changeset
|
232 category of record and obj is an arbitrary object. |
20949
571f2903ff1e
bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20948
diff
changeset
|
233 |
571f2903ff1e
bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20948
diff
changeset
|
234 `records['cat']` will return all entries of this category 'cat'. |
571f2903ff1e
bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20948
diff
changeset
|
235 |
571f2903ff1e
bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20948
diff
changeset
|
236 Iterating on the object itself will yield `('category', obj)` tuples |
571f2903ff1e
bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20948
diff
changeset
|
237 for all entries. |
571f2903ff1e
bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20948
diff
changeset
|
238 |
571f2903ff1e
bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20948
diff
changeset
|
239 All iterations happens in chronological order. |
571f2903ff1e
bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20948
diff
changeset
|
240 """ |
571f2903ff1e
bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20948
diff
changeset
|
241 |
571f2903ff1e
bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20948
diff
changeset
|
242 def __init__(self): |
571f2903ff1e
bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20948
diff
changeset
|
243 self._categories = {} |
571f2903ff1e
bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20948
diff
changeset
|
244 self._sequences = [] |
20996
ed3c5e18a047
bundle2: add reply awareness to unbundlerecords
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20995
diff
changeset
|
245 self._replies = {} |
20949
571f2903ff1e
bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20948
diff
changeset
|
246 |
20996
ed3c5e18a047
bundle2: add reply awareness to unbundlerecords
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20995
diff
changeset
|
247 def add(self, category, entry, inreplyto=None): |
20949
571f2903ff1e
bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20948
diff
changeset
|
248 """add a new record of a given category. |
571f2903ff1e
bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20948
diff
changeset
|
249 |
571f2903ff1e
bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20948
diff
changeset
|
250 The entry can then be retrieved in the list returned by |
571f2903ff1e
bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20948
diff
changeset
|
251 self['category'].""" |
571f2903ff1e
bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20948
diff
changeset
|
252 self._categories.setdefault(category, []).append(entry) |
571f2903ff1e
bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20948
diff
changeset
|
253 self._sequences.append((category, entry)) |
20996
ed3c5e18a047
bundle2: add reply awareness to unbundlerecords
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20995
diff
changeset
|
254 if inreplyto is not None: |
ed3c5e18a047
bundle2: add reply awareness to unbundlerecords
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20995
diff
changeset
|
255 self.getreplies(inreplyto).add(category, entry) |
ed3c5e18a047
bundle2: add reply awareness to unbundlerecords
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20995
diff
changeset
|
256 |
ed3c5e18a047
bundle2: add reply awareness to unbundlerecords
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20995
diff
changeset
|
257 def getreplies(self, partid): |
23139
e53f6b72a0e4
spelling: fixes from proofreading of spell checker issues
Mads Kiilerich <madski@unity3d.com>
parents:
23067
diff
changeset
|
258 """get the records that are replies to a specific part""" |
20996
ed3c5e18a047
bundle2: add reply awareness to unbundlerecords
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20995
diff
changeset
|
259 return self._replies.setdefault(partid, unbundlerecords()) |
20949
571f2903ff1e
bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20948
diff
changeset
|
260 |
571f2903ff1e
bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20948
diff
changeset
|
261 def __getitem__(self, cat): |
571f2903ff1e
bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20948
diff
changeset
|
262 return tuple(self._categories.get(cat, ())) |
571f2903ff1e
bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20948
diff
changeset
|
263 |
571f2903ff1e
bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20948
diff
changeset
|
264 def __iter__(self): |
571f2903ff1e
bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20948
diff
changeset
|
265 return iter(self._sequences) |
571f2903ff1e
bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20948
diff
changeset
|
266 |
571f2903ff1e
bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20948
diff
changeset
|
267 def __len__(self): |
571f2903ff1e
bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20948
diff
changeset
|
268 return len(self._sequences) |
571f2903ff1e
bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20948
diff
changeset
|
269 |
571f2903ff1e
bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20948
diff
changeset
|
270 def __nonzero__(self): |
571f2903ff1e
bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20948
diff
changeset
|
271 return bool(self._sequences) |
571f2903ff1e
bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20948
diff
changeset
|
272 |
20948
329cd74b52bd
bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20947
diff
changeset
|
273 class bundleoperation(object): |
329cd74b52bd
bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20947
diff
changeset
|
274 """an object that represents a single bundling process |
329cd74b52bd
bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20947
diff
changeset
|
275 |
329cd74b52bd
bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20947
diff
changeset
|
276 Its purpose is to carry unbundle-related objects and states. |
329cd74b52bd
bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20947
diff
changeset
|
277 |
329cd74b52bd
bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20947
diff
changeset
|
278 A new object should be created at the beginning of each bundle processing. |
329cd74b52bd
bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20947
diff
changeset
|
279 The object is to be returned by the processing function. |
329cd74b52bd
bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20947
diff
changeset
|
280 |
329cd74b52bd
bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20947
diff
changeset
|
281 The object has very little content now it will ultimately contain: |
329cd74b52bd
bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20947
diff
changeset
|
282 * an access to the repo the bundle is applied to, |
329cd74b52bd
bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20947
diff
changeset
|
283 * a ui object, |
329cd74b52bd
bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20947
diff
changeset
|
284 * a way to retrieve a transaction to add changes to the repo, |
329cd74b52bd
bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20947
diff
changeset
|
285 * a way to record the result of processing each part, |
329cd74b52bd
bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20947
diff
changeset
|
286 * a way to construct a bundle response when applicable. |
329cd74b52bd
bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20947
diff
changeset
|
287 """ |
329cd74b52bd
bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20947
diff
changeset
|
288 |
24878
e530cde6d115
bundle2: disable ouput capture unless we use http (issue4613 issue4615)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24851
diff
changeset
|
289 def __init__(self, repo, transactiongetter, captureoutput=True): |
20948
329cd74b52bd
bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20947
diff
changeset
|
290 self.repo = repo |
329cd74b52bd
bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20947
diff
changeset
|
291 self.ui = repo.ui |
20949
571f2903ff1e
bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20948
diff
changeset
|
292 self.records = unbundlerecords() |
20952
b24ee5076b94
bundle2: make it possible have a global transaction for the unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20950
diff
changeset
|
293 self.gettransaction = transactiongetter |
20997
d7df4b7378ae
bundle2: produce a bundle2 reply
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20996
diff
changeset
|
294 self.reply = None |
24878
e530cde6d115
bundle2: disable ouput capture unless we use http (issue4613 issue4615)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24851
diff
changeset
|
295 self.captureoutput = captureoutput |
20948
329cd74b52bd
bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20947
diff
changeset
|
296 |
20952
b24ee5076b94
bundle2: make it possible have a global transaction for the unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20950
diff
changeset
|
297 class TransactionUnavailable(RuntimeError): |
b24ee5076b94
bundle2: make it possible have a global transaction for the unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20950
diff
changeset
|
298 pass |
b24ee5076b94
bundle2: make it possible have a global transaction for the unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20950
diff
changeset
|
299 |
b24ee5076b94
bundle2: make it possible have a global transaction for the unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20950
diff
changeset
|
300 def _notransaction(): |
b24ee5076b94
bundle2: make it possible have a global transaction for the unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20950
diff
changeset
|
301 """default method to get a transaction while processing a bundle |
b24ee5076b94
bundle2: make it possible have a global transaction for the unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20950
diff
changeset
|
302 |
b24ee5076b94
bundle2: make it possible have a global transaction for the unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20950
diff
changeset
|
303 Raise an exception to highlight the fact that no transaction was expected |
b24ee5076b94
bundle2: make it possible have a global transaction for the unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20950
diff
changeset
|
304 to be created""" |
b24ee5076b94
bundle2: make it possible have a global transaction for the unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20950
diff
changeset
|
305 raise TransactionUnavailable() |
b24ee5076b94
bundle2: make it possible have a global transaction for the unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20950
diff
changeset
|
306 |
26795
dff05b3f47d2
applybundle: take url as argument
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26793
diff
changeset
|
307 def applybundle(repo, unbundler, tr, source=None, url=None, op=None): |
26790
28a6c2d72097
bundle2: introduce an "applybundle" function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26698
diff
changeset
|
308 # transform me into unbundler.apply() as soon as the freeze is lifted |
26792
a84e0cacb2dd
applybundle: set 'bundle2=1' env for all transaction
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26790
diff
changeset
|
309 tr.hookargs['bundle2'] = '1' |
26793
f37de5d44aea
applybundle: take source as argument
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26792
diff
changeset
|
310 if source is not None and 'source' not in tr.hookargs: |
f37de5d44aea
applybundle: take source as argument
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26792
diff
changeset
|
311 tr.hookargs['source'] = source |
26795
dff05b3f47d2
applybundle: take url as argument
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26793
diff
changeset
|
312 if url is not None and 'url' not in tr.hookargs: |
dff05b3f47d2
applybundle: take url as argument
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26793
diff
changeset
|
313 tr.hookargs['url'] = url |
26790
28a6c2d72097
bundle2: introduce an "applybundle" function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26698
diff
changeset
|
314 return processbundle(repo, unbundler, lambda: tr, op=op) |
28a6c2d72097
bundle2: introduce an "applybundle" function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26698
diff
changeset
|
315 |
24851
df0ce98c882f
bundle2: also save output when error happens during part processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24849
diff
changeset
|
316 def processbundle(repo, unbundler, transactiongetter=None, op=None): |
20889
deed5edb72de
bundle2: first version of a bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20887
diff
changeset
|
317 """This function process a bundle, apply effect to/from a repo |
deed5edb72de
bundle2: first version of a bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20887
diff
changeset
|
318 |
20947
c33d7bf53812
bundle2: feed a unbundle20 to the `processbundle` function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20892
diff
changeset
|
319 It iterates over each part then searches for and uses the proper handling |
c33d7bf53812
bundle2: feed a unbundle20 to the `processbundle` function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20892
diff
changeset
|
320 code to process the part. Parts are processed in order. |
20889
deed5edb72de
bundle2: first version of a bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20887
diff
changeset
|
321 |
deed5edb72de
bundle2: first version of a bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20887
diff
changeset
|
322 This is very early version of this function that will be strongly reworked |
deed5edb72de
bundle2: first version of a bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20887
diff
changeset
|
323 before final usage. |
deed5edb72de
bundle2: first version of a bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20887
diff
changeset
|
324 |
20891
1c6cd23fc221
bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20890
diff
changeset
|
325 Unknown Mandatory part will abort the process. |
24851
df0ce98c882f
bundle2: also save output when error happens during part processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24849
diff
changeset
|
326 |
df0ce98c882f
bundle2: also save output when error happens during part processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24849
diff
changeset
|
327 It is temporarily possible to provide a prebuilt bundleoperation to the |
df0ce98c882f
bundle2: also save output when error happens during part processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24849
diff
changeset
|
328 function. This is used to ensure output is properly propagated in case of |
df0ce98c882f
bundle2: also save output when error happens during part processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24849
diff
changeset
|
329 an error during the unbundling. This output capturing part will likely be |
df0ce98c882f
bundle2: also save output when error happens during part processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24849
diff
changeset
|
330 reworked and this ability will probably go away in the process. |
20889
deed5edb72de
bundle2: first version of a bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20887
diff
changeset
|
331 """ |
24851
df0ce98c882f
bundle2: also save output when error happens during part processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24849
diff
changeset
|
332 if op is None: |
df0ce98c882f
bundle2: also save output when error happens during part processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24849
diff
changeset
|
333 if transactiongetter is None: |
df0ce98c882f
bundle2: also save output when error happens during part processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24849
diff
changeset
|
334 transactiongetter = _notransaction |
df0ce98c882f
bundle2: also save output when error happens during part processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24849
diff
changeset
|
335 op = bundleoperation(repo, transactiongetter) |
20889
deed5edb72de
bundle2: first version of a bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20887
diff
changeset
|
336 # todo: |
deed5edb72de
bundle2: first version of a bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20887
diff
changeset
|
337 # - replace this is a init function soon. |
deed5edb72de
bundle2: first version of a bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20887
diff
changeset
|
338 # - exception catching |
deed5edb72de
bundle2: first version of a bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20887
diff
changeset
|
339 unbundler.params |
25331
53ca47ef645d
bundle2: add generic debug output regarding processed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25324
diff
changeset
|
340 if repo.ui.debugflag: |
53ca47ef645d
bundle2: add generic debug output regarding processed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25324
diff
changeset
|
341 msg = ['bundle2-input-bundle:'] |
53ca47ef645d
bundle2: add generic debug output regarding processed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25324
diff
changeset
|
342 if unbundler.params: |
53ca47ef645d
bundle2: add generic debug output regarding processed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25324
diff
changeset
|
343 msg.append(' %i params') |
53ca47ef645d
bundle2: add generic debug output regarding processed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25324
diff
changeset
|
344 if op.gettransaction is None: |
53ca47ef645d
bundle2: add generic debug output regarding processed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25324
diff
changeset
|
345 msg.append(' no-transaction') |
53ca47ef645d
bundle2: add generic debug output regarding processed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25324
diff
changeset
|
346 else: |
53ca47ef645d
bundle2: add generic debug output regarding processed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25324
diff
changeset
|
347 msg.append(' with-transaction') |
53ca47ef645d
bundle2: add generic debug output regarding processed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25324
diff
changeset
|
348 msg.append('\n') |
53ca47ef645d
bundle2: add generic debug output regarding processed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25324
diff
changeset
|
349 repo.ui.debug(''.join(msg)) |
25332
0cdd1c811705
bundle2: add generic debug output at the end of bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25331
diff
changeset
|
350 iterparts = enumerate(unbundler.iterparts()) |
21019
3dc09f831a2e
bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21016
diff
changeset
|
351 part = None |
25332
0cdd1c811705
bundle2: add generic debug output at the end of bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25331
diff
changeset
|
352 nbpart = 0 |
20892
6fe95448596d
bundle2: read the whole bundle from stream on abort
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20891
diff
changeset
|
353 try: |
25332
0cdd1c811705
bundle2: add generic debug output at the end of bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25331
diff
changeset
|
354 for nbpart, part in iterparts: |
23008
d3137827016a
bundle2: extract processing of part into its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23001
diff
changeset
|
355 _processpart(op, part) |
25660
328739ea70c3
global: mass rewrite to use modern exception syntax
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25641
diff
changeset
|
356 except BaseException as exc: |
25332
0cdd1c811705
bundle2: add generic debug output at the end of bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25331
diff
changeset
|
357 for nbpart, part in iterparts: |
21019
3dc09f831a2e
bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21016
diff
changeset
|
358 # consume the bundle content |
24047
731fa8e3e580
bundle2: now that we have a seek implementation, use it
Eric Sumner <ericsumner@fb.com>
parents:
24037
diff
changeset
|
359 part.seek(0, 2) |
21176
70fcb0a71445
bundle2: decorate exception raised during bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21146
diff
changeset
|
360 # Small hack to let caller code distinguish exceptions from bundle2 |
23139
e53f6b72a0e4
spelling: fixes from proofreading of spell checker issues
Mads Kiilerich <madski@unity3d.com>
parents:
23067
diff
changeset
|
361 # processing from processing the old format. This is mostly |
21176
70fcb0a71445
bundle2: decorate exception raised during bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21146
diff
changeset
|
362 # needed to handle different return codes to unbundle according to the |
70fcb0a71445
bundle2: decorate exception raised during bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21146
diff
changeset
|
363 # type of bundle. We should probably clean up or drop this return code |
70fcb0a71445
bundle2: decorate exception raised during bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21146
diff
changeset
|
364 # craziness in a future version. |
70fcb0a71445
bundle2: decorate exception raised during bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21146
diff
changeset
|
365 exc.duringunbundle2 = True |
24795
f9aa4cb8f2dd
bundle2: store the salvaged output on the exception object
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24794
diff
changeset
|
366 salvaged = [] |
25492
219b8ab31220
bundle2: also capture reply capability on failure
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25491
diff
changeset
|
367 replycaps = None |
24795
f9aa4cb8f2dd
bundle2: store the salvaged output on the exception object
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24794
diff
changeset
|
368 if op.reply is not None: |
f9aa4cb8f2dd
bundle2: store the salvaged output on the exception object
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24794
diff
changeset
|
369 salvaged = op.reply.salvageoutput() |
25492
219b8ab31220
bundle2: also capture reply capability on failure
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25491
diff
changeset
|
370 replycaps = op.reply.capabilities |
219b8ab31220
bundle2: also capture reply capability on failure
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25491
diff
changeset
|
371 exc._replycaps = replycaps |
24795
f9aa4cb8f2dd
bundle2: store the salvaged output on the exception object
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24794
diff
changeset
|
372 exc._bundle2salvagedoutput = salvaged |
20892
6fe95448596d
bundle2: read the whole bundle from stream on abort
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20891
diff
changeset
|
373 raise |
25332
0cdd1c811705
bundle2: add generic debug output at the end of bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25331
diff
changeset
|
374 finally: |
0cdd1c811705
bundle2: add generic debug output at the end of bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25331
diff
changeset
|
375 repo.ui.debug('bundle2-input-bundle: %i parts total\n' % nbpart) |
0cdd1c811705
bundle2: add generic debug output at the end of bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25331
diff
changeset
|
376 |
20949
571f2903ff1e
bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20948
diff
changeset
|
377 return op |
20889
deed5edb72de
bundle2: first version of a bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20887
diff
changeset
|
378 |
23008
d3137827016a
bundle2: extract processing of part into its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23001
diff
changeset
|
379 def _processpart(op, part): |
d3137827016a
bundle2: extract processing of part into its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23001
diff
changeset
|
380 """process a single part from a bundle |
d3137827016a
bundle2: extract processing of part into its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23001
diff
changeset
|
381 |
d3137827016a
bundle2: extract processing of part into its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23001
diff
changeset
|
382 The part is guaranteed to have been fully consumed when the function exits |
d3137827016a
bundle2: extract processing of part into its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23001
diff
changeset
|
383 (even if an exception is raised).""" |
25333
5bfffdbdc907
bundle2: add generic debug output regarding processed part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25332
diff
changeset
|
384 status = 'unknown' # used by debug output |
23008
d3137827016a
bundle2: extract processing of part into its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23001
diff
changeset
|
385 try: |
d3137827016a
bundle2: extract processing of part into its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23001
diff
changeset
|
386 try: |
23586
112f9c73a0e5
bundle2._processpart: forcing lower-case compare is no longer necessary
Eric Sumner <ericsumner@fb.com>
parents:
23585
diff
changeset
|
387 handler = parthandlermapping.get(part.type) |
23008
d3137827016a
bundle2: extract processing of part into its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23001
diff
changeset
|
388 if handler is None: |
25333
5bfffdbdc907
bundle2: add generic debug output regarding processed part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25332
diff
changeset
|
389 status = 'unsupported-type' |
26393
cff70549a959
bundle2: rename error exception class for unsupported feature
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26269
diff
changeset
|
390 raise error.BundleUnknownFeatureError(parttype=part.type) |
25320
697d8953b04d
bundle2: handle new line in 'indebug' function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25319
diff
changeset
|
391 indebug(op.ui, 'found a handler for part %r' % part.type) |
23008
d3137827016a
bundle2: extract processing of part into its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23001
diff
changeset
|
392 unknownparams = part.mandatorykeys - handler.params |
d3137827016a
bundle2: extract processing of part into its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23001
diff
changeset
|
393 if unknownparams: |
d3137827016a
bundle2: extract processing of part into its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23001
diff
changeset
|
394 unknownparams = list(unknownparams) |
d3137827016a
bundle2: extract processing of part into its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23001
diff
changeset
|
395 unknownparams.sort() |
25333
5bfffdbdc907
bundle2: add generic debug output regarding processed part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25332
diff
changeset
|
396 status = 'unsupported-params (%s)' % unknownparams |
26393
cff70549a959
bundle2: rename error exception class for unsupported feature
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26269
diff
changeset
|
397 raise error.BundleUnknownFeatureError(parttype=part.type, |
cff70549a959
bundle2: rename error exception class for unsupported feature
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26269
diff
changeset
|
398 params=unknownparams) |
25333
5bfffdbdc907
bundle2: add generic debug output regarding processed part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25332
diff
changeset
|
399 status = 'supported' |
26393
cff70549a959
bundle2: rename error exception class for unsupported feature
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26269
diff
changeset
|
400 except error.BundleUnknownFeatureError as exc: |
23585
94b25d71dd0f
bundle2.unbundlepart: decouple mandatory from parttype
Eric Sumner <ericsumner@fb.com>
parents:
23439
diff
changeset
|
401 if part.mandatory: # mandatory parts |
23008
d3137827016a
bundle2: extract processing of part into its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23001
diff
changeset
|
402 raise |
25320
697d8953b04d
bundle2: handle new line in 'indebug' function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25319
diff
changeset
|
403 indebug(op.ui, 'ignoring unsupported advisory part %s' % exc) |
23008
d3137827016a
bundle2: extract processing of part into its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23001
diff
changeset
|
404 return # skip to part processing |
25333
5bfffdbdc907
bundle2: add generic debug output regarding processed part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25332
diff
changeset
|
405 finally: |
5bfffdbdc907
bundle2: add generic debug output regarding processed part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25332
diff
changeset
|
406 if op.ui.debugflag: |
5bfffdbdc907
bundle2: add generic debug output regarding processed part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25332
diff
changeset
|
407 msg = ['bundle2-input-part: "%s"' % part.type] |
5bfffdbdc907
bundle2: add generic debug output regarding processed part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25332
diff
changeset
|
408 if not part.mandatory: |
5bfffdbdc907
bundle2: add generic debug output regarding processed part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25332
diff
changeset
|
409 msg.append(' (advisory)') |
5bfffdbdc907
bundle2: add generic debug output regarding processed part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25332
diff
changeset
|
410 nbmp = len(part.mandatorykeys) |
5bfffdbdc907
bundle2: add generic debug output regarding processed part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25332
diff
changeset
|
411 nbap = len(part.params) - nbmp |
5bfffdbdc907
bundle2: add generic debug output regarding processed part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25332
diff
changeset
|
412 if nbmp or nbap: |
5bfffdbdc907
bundle2: add generic debug output regarding processed part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25332
diff
changeset
|
413 msg.append(' (params:') |
5bfffdbdc907
bundle2: add generic debug output regarding processed part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25332
diff
changeset
|
414 if nbmp: |
5bfffdbdc907
bundle2: add generic debug output regarding processed part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25332
diff
changeset
|
415 msg.append(' %i mandatory' % nbmp) |
5bfffdbdc907
bundle2: add generic debug output regarding processed part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25332
diff
changeset
|
416 if nbap: |
5bfffdbdc907
bundle2: add generic debug output regarding processed part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25332
diff
changeset
|
417 msg.append(' %i advisory' % nbmp) |
5bfffdbdc907
bundle2: add generic debug output regarding processed part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25332
diff
changeset
|
418 msg.append(')') |
5bfffdbdc907
bundle2: add generic debug output regarding processed part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25332
diff
changeset
|
419 msg.append(' %s\n' % status) |
5bfffdbdc907
bundle2: add generic debug output regarding processed part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25332
diff
changeset
|
420 op.ui.debug(''.join(msg)) |
23008
d3137827016a
bundle2: extract processing of part into its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23001
diff
changeset
|
421 |
d3137827016a
bundle2: extract processing of part into its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23001
diff
changeset
|
422 # handler is called outside the above try block so that we don't |
d3137827016a
bundle2: extract processing of part into its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23001
diff
changeset
|
423 # risk catching KeyErrors from anything other than the |
d3137827016a
bundle2: extract processing of part into its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23001
diff
changeset
|
424 # parthandlermapping lookup (any KeyError raised by handler() |
d3137827016a
bundle2: extract processing of part into its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23001
diff
changeset
|
425 # itself represents a defect of a different variety). |
d3137827016a
bundle2: extract processing of part into its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23001
diff
changeset
|
426 output = None |
24878
e530cde6d115
bundle2: disable ouput capture unless we use http (issue4613 issue4615)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24851
diff
changeset
|
427 if op.captureoutput and op.reply is not None: |
24849
aff2aca3420e
bundle2: also capture hook output during processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24846
diff
changeset
|
428 op.ui.pushbuffer(error=True, subproc=True) |
23008
d3137827016a
bundle2: extract processing of part into its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23001
diff
changeset
|
429 output = '' |
d3137827016a
bundle2: extract processing of part into its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23001
diff
changeset
|
430 try: |
d3137827016a
bundle2: extract processing of part into its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23001
diff
changeset
|
431 handler(op, part) |
d3137827016a
bundle2: extract processing of part into its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23001
diff
changeset
|
432 finally: |
d3137827016a
bundle2: extract processing of part into its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23001
diff
changeset
|
433 if output is not None: |
d3137827016a
bundle2: extract processing of part into its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23001
diff
changeset
|
434 output = op.ui.popbuffer() |
24743
a2ef1dc3b59b
bundle2: flush output in a part in all cases
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24741
diff
changeset
|
435 if output: |
a2ef1dc3b59b
bundle2: flush output in a part in all cases
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24741
diff
changeset
|
436 outpart = op.reply.newpart('output', data=output, |
a2ef1dc3b59b
bundle2: flush output in a part in all cases
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24741
diff
changeset
|
437 mandatory=False) |
a2ef1dc3b59b
bundle2: flush output in a part in all cases
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24741
diff
changeset
|
438 outpart.addparam('in-reply-to', str(part.id), mandatory=False) |
23008
d3137827016a
bundle2: extract processing of part into its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23001
diff
changeset
|
439 finally: |
d3137827016a
bundle2: extract processing of part into its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23001
diff
changeset
|
440 # consume the part content to not corrupt the stream. |
24047
731fa8e3e580
bundle2: now that we have a seek implementation, use it
Eric Sumner <ericsumner@fb.com>
parents:
24037
diff
changeset
|
441 part.seek(0, 2) |
23008
d3137827016a
bundle2: extract processing of part into its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23001
diff
changeset
|
442 |
d3137827016a
bundle2: extract processing of part into its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23001
diff
changeset
|
443 |
21138
f469879d27ec
bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21137
diff
changeset
|
444 def decodecaps(blob): |
23139
e53f6b72a0e4
spelling: fixes from proofreading of spell checker issues
Mads Kiilerich <madski@unity3d.com>
parents:
23067
diff
changeset
|
445 """decode a bundle2 caps bytes blob into a dictionary |
21138
f469879d27ec
bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21137
diff
changeset
|
446 |
f469879d27ec
bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21137
diff
changeset
|
447 The blob is a list of capabilities (one per line) |
f469879d27ec
bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21137
diff
changeset
|
448 Capabilities may have values using a line of the form:: |
f469879d27ec
bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21137
diff
changeset
|
449 |
f469879d27ec
bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21137
diff
changeset
|
450 capability=value1,value2,value3 |
f469879d27ec
bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21137
diff
changeset
|
451 |
f469879d27ec
bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21137
diff
changeset
|
452 The values are always a list.""" |
f469879d27ec
bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21137
diff
changeset
|
453 caps = {} |
f469879d27ec
bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21137
diff
changeset
|
454 for line in blob.splitlines(): |
f469879d27ec
bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21137
diff
changeset
|
455 if not line: |
f469879d27ec
bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21137
diff
changeset
|
456 continue |
f469879d27ec
bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21137
diff
changeset
|
457 if '=' not in line: |
f469879d27ec
bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21137
diff
changeset
|
458 key, vals = line, () |
f469879d27ec
bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21137
diff
changeset
|
459 else: |
f469879d27ec
bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21137
diff
changeset
|
460 key, vals = line.split('=', 1) |
f469879d27ec
bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21137
diff
changeset
|
461 vals = vals.split(',') |
28883
032c4c2f802a
pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents:
28672
diff
changeset
|
462 key = urlreq.unquote(key) |
032c4c2f802a
pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents:
28672
diff
changeset
|
463 vals = [urlreq.unquote(v) for v in vals] |
21138
f469879d27ec
bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21137
diff
changeset
|
464 caps[key] = vals |
f469879d27ec
bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21137
diff
changeset
|
465 return caps |
f469879d27ec
bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21137
diff
changeset
|
466 |
21139
2b8c82f7f11d
bundle2: capabilities encoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21138
diff
changeset
|
467 def encodecaps(caps): |
2b8c82f7f11d
bundle2: capabilities encoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21138
diff
changeset
|
468 """encode a bundle2 caps dictionary into a bytes blob""" |
2b8c82f7f11d
bundle2: capabilities encoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21138
diff
changeset
|
469 chunks = [] |
2b8c82f7f11d
bundle2: capabilities encoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21138
diff
changeset
|
470 for ca in sorted(caps): |
2b8c82f7f11d
bundle2: capabilities encoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21138
diff
changeset
|
471 vals = caps[ca] |
28883
032c4c2f802a
pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents:
28672
diff
changeset
|
472 ca = urlreq.quote(ca) |
032c4c2f802a
pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents:
28672
diff
changeset
|
473 vals = [urlreq.quote(v) for v in vals] |
21139
2b8c82f7f11d
bundle2: capabilities encoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21138
diff
changeset
|
474 if vals: |
2b8c82f7f11d
bundle2: capabilities encoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21138
diff
changeset
|
475 ca = "%s=%s" % (ca, ','.join(vals)) |
2b8c82f7f11d
bundle2: capabilities encoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21138
diff
changeset
|
476 chunks.append(ca) |
2b8c82f7f11d
bundle2: capabilities encoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21138
diff
changeset
|
477 return '\n'.join(chunks) |
2b8c82f7f11d
bundle2: capabilities encoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21138
diff
changeset
|
478 |
28666
ae53ecc47414
bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
479 bundletypes = { |
ae53ecc47414
bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
480 "": ("", None), # only when using unbundle on ssh and old http servers |
ae53ecc47414
bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
481 # since the unification ssh accepts a header but there |
ae53ecc47414
bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
482 # is no capability signaling it. |
ae53ecc47414
bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
483 "HG20": (), # special-cased below |
ae53ecc47414
bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
484 "HG10UN": ("HG10UN", None), |
ae53ecc47414
bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
485 "HG10BZ": ("HG10", 'BZ'), |
ae53ecc47414
bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
486 "HG10GZ": ("HG10GZ", 'GZ'), |
ae53ecc47414
bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
487 } |
ae53ecc47414
bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
488 |
ae53ecc47414
bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
489 # hgweb uses this list to communicate its preferred type |
ae53ecc47414
bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
490 bundlepriority = ['HG10GZ', 'HG10BZ', 'HG10UN'] |
ae53ecc47414
bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
491 |
20801
9c5183cb9bca
bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff
changeset
|
492 class bundle20(object): |
9c5183cb9bca
bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff
changeset
|
493 """represent an outgoing bundle2 container |
9c5183cb9bca
bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff
changeset
|
494 |
21599
57cd844d7a5b
bundle2: have ``newpart`` automatically add the part to the bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21598
diff
changeset
|
495 Use the `addparam` method to add stream level parameter. and `newpart` to |
20856
8a6a86c9a5b5
bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20844
diff
changeset
|
496 populate it. Then call `getchunks` to retrieve all the binary chunks of |
21024
7731a2281cf0
spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents:
21020
diff
changeset
|
497 data that compose the bundle2 container.""" |
20801
9c5183cb9bca
bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff
changeset
|
498 |
24686
e0e28e910fa3
bundle2: rename format, parts and config to final names
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24648
diff
changeset
|
499 _magicstring = 'HG20' |
24640
685639f9430d
bundle20: move magic string into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24071
diff
changeset
|
500 |
21134
2f8c4fa237f5
bundle2: adds a capabilities attribute on bundler20
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21133
diff
changeset
|
501 def __init__(self, ui, capabilities=()): |
20842
938718d72624
bundle2: print debug information during bundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20814
diff
changeset
|
502 self.ui = ui |
20801
9c5183cb9bca
bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff
changeset
|
503 self._params = [] |
9c5183cb9bca
bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff
changeset
|
504 self._parts = [] |
21136
b6fd496e5c72
bundle2: support for capabilities with values
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21135
diff
changeset
|
505 self.capabilities = dict(capabilities) |
26404
795f02a24b87
bundle2: allow compressed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26396
diff
changeset
|
506 self._compressor = util.compressors[None]() |
795f02a24b87
bundle2: allow compressed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26396
diff
changeset
|
507 |
795f02a24b87
bundle2: allow compressed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26396
diff
changeset
|
508 def setcompression(self, alg): |
795f02a24b87
bundle2: allow compressed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26396
diff
changeset
|
509 """setup core part compression to <alg>""" |
795f02a24b87
bundle2: allow compressed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26396
diff
changeset
|
510 if alg is None: |
795f02a24b87
bundle2: allow compressed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26396
diff
changeset
|
511 return |
795f02a24b87
bundle2: allow compressed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26396
diff
changeset
|
512 assert not any(n.lower() == 'Compression' for n, v in self._params) |
795f02a24b87
bundle2: allow compressed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26396
diff
changeset
|
513 self.addparam('Compression', alg) |
795f02a24b87
bundle2: allow compressed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26396
diff
changeset
|
514 self._compressor = util.compressors[alg]() |
20801
9c5183cb9bca
bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff
changeset
|
515 |
21900
b8bd97085ec9
bundle2: add a ``bundle20.nbparts`` property
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21660
diff
changeset
|
516 @property |
b8bd97085ec9
bundle2: add a ``bundle20.nbparts`` property
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21660
diff
changeset
|
517 def nbparts(self): |
b8bd97085ec9
bundle2: add a ``bundle20.nbparts`` property
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21660
diff
changeset
|
518 """total number of parts added to the bundler""" |
b8bd97085ec9
bundle2: add a ``bundle20.nbparts`` property
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21660
diff
changeset
|
519 return len(self._parts) |
b8bd97085ec9
bundle2: add a ``bundle20.nbparts`` property
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21660
diff
changeset
|
520 |
21597
1daad9dcdba2
bundle2: small doc update on the bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21186
diff
changeset
|
521 # methods used to defines the bundle2 content |
20804
db9d3991d2c6
bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20803
diff
changeset
|
522 def addparam(self, name, value=None): |
db9d3991d2c6
bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20803
diff
changeset
|
523 """add a stream level parameter""" |
20813
8c74b3ce5b70
bundle2: refuse empty parameter name
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20812
diff
changeset
|
524 if not name: |
8c74b3ce5b70
bundle2: refuse empty parameter name
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20812
diff
changeset
|
525 raise ValueError('empty parameter name') |
20814
8532f5e1b9df
bundle2: force the first char of parameter to be an letter.
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20813
diff
changeset
|
526 if name[0] not in string.letters: |
8532f5e1b9df
bundle2: force the first char of parameter to be an letter.
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20813
diff
changeset
|
527 raise ValueError('non letter first character: %r' % name) |
20804
db9d3991d2c6
bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20803
diff
changeset
|
528 self._params.append((name, value)) |
db9d3991d2c6
bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20803
diff
changeset
|
529 |
20856
8a6a86c9a5b5
bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20844
diff
changeset
|
530 def addpart(self, part): |
8a6a86c9a5b5
bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20844
diff
changeset
|
531 """add a new part to the bundle2 container |
8a6a86c9a5b5
bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20844
diff
changeset
|
532 |
21024
7731a2281cf0
spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents:
21020
diff
changeset
|
533 Parts contains the actual applicative payload.""" |
20995
e995d104c87f
bundle2: add an integer id to part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20952
diff
changeset
|
534 assert part.id is None |
e995d104c87f
bundle2: add an integer id to part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20952
diff
changeset
|
535 part.id = len(self._parts) # very cheap counter |
20856
8a6a86c9a5b5
bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20844
diff
changeset
|
536 self._parts.append(part) |
8a6a86c9a5b5
bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20844
diff
changeset
|
537 |
21598
1b0dbb91de5b
bundle2: add a ``newpart`` method to ``bundle20``
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21597
diff
changeset
|
538 def newpart(self, typeid, *args, **kwargs): |
21602
cc33ae50bab3
bundle2: warn about error during initialization in ``newpart`` docstring
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21601
diff
changeset
|
539 """create a new part and add it to the containers |
cc33ae50bab3
bundle2: warn about error during initialization in ``newpart`` docstring
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21601
diff
changeset
|
540 |
cc33ae50bab3
bundle2: warn about error during initialization in ``newpart`` docstring
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21601
diff
changeset
|
541 As the part is directly added to the containers. For now, this means |
cc33ae50bab3
bundle2: warn about error during initialization in ``newpart`` docstring
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21601
diff
changeset
|
542 that any failure to properly initialize the part after calling |
cc33ae50bab3
bundle2: warn about error during initialization in ``newpart`` docstring
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21601
diff
changeset
|
543 ``newpart`` should result in a failure of the whole bundling process. |
cc33ae50bab3
bundle2: warn about error during initialization in ``newpart`` docstring
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21601
diff
changeset
|
544 |
cc33ae50bab3
bundle2: warn about error during initialization in ``newpart`` docstring
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21601
diff
changeset
|
545 You can still fall back to manually create and add if you need better |
cc33ae50bab3
bundle2: warn about error during initialization in ``newpart`` docstring
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21601
diff
changeset
|
546 control.""" |
21598
1b0dbb91de5b
bundle2: add a ``newpart`` method to ``bundle20``
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21597
diff
changeset
|
547 part = bundlepart(typeid, *args, **kwargs) |
21599
57cd844d7a5b
bundle2: have ``newpart`` automatically add the part to the bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21598
diff
changeset
|
548 self.addpart(part) |
21598
1b0dbb91de5b
bundle2: add a ``newpart`` method to ``bundle20``
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21597
diff
changeset
|
549 return part |
1b0dbb91de5b
bundle2: add a ``newpart`` method to ``bundle20``
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21597
diff
changeset
|
550 |
21597
1daad9dcdba2
bundle2: small doc update on the bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21186
diff
changeset
|
551 # methods used to generate the bundle2 stream |
20801
9c5183cb9bca
bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff
changeset
|
552 def getchunks(self): |
25322
1f7c0403f8be
bundle2: add generic debug output regarding generated bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25321
diff
changeset
|
553 if self.ui.debugflag: |
1f7c0403f8be
bundle2: add generic debug output regarding generated bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25321
diff
changeset
|
554 msg = ['bundle2-output-bundle: "%s",' % self._magicstring] |
1f7c0403f8be
bundle2: add generic debug output regarding generated bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25321
diff
changeset
|
555 if self._params: |
1f7c0403f8be
bundle2: add generic debug output regarding generated bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25321
diff
changeset
|
556 msg.append(' (%i params)' % len(self._params)) |
1f7c0403f8be
bundle2: add generic debug output regarding generated bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25321
diff
changeset
|
557 msg.append(' %i parts total\n' % len(self._parts)) |
1f7c0403f8be
bundle2: add generic debug output regarding generated bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25321
diff
changeset
|
558 self.ui.debug(''.join(msg)) |
25315
e28245b0da56
bundle2: handle new line in 'outdebug' function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25314
diff
changeset
|
559 outdebug(self.ui, 'start emission of %s stream' % self._magicstring) |
24640
685639f9430d
bundle20: move magic string into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24071
diff
changeset
|
560 yield self._magicstring |
20804
db9d3991d2c6
bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20803
diff
changeset
|
561 param = self._paramchunk() |
25315
e28245b0da56
bundle2: handle new line in 'outdebug' function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25314
diff
changeset
|
562 outdebug(self.ui, 'bundle parameter: %s' % param) |
20804
db9d3991d2c6
bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20803
diff
changeset
|
563 yield _pack(_fstreamparamsize, len(param)) |
db9d3991d2c6
bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20803
diff
changeset
|
564 if param: |
db9d3991d2c6
bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20803
diff
changeset
|
565 yield param |
26404
795f02a24b87
bundle2: allow compressed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26396
diff
changeset
|
566 # starting compression |
26396
d90c30801cdf
bundle20: extract core payload generation in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26395
diff
changeset
|
567 for chunk in self._getcorechunk(): |
26404
795f02a24b87
bundle2: allow compressed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26396
diff
changeset
|
568 yield self._compressor.compress(chunk) |
795f02a24b87
bundle2: allow compressed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26396
diff
changeset
|
569 yield self._compressor.flush() |
20802
520df53ad26a
bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20801
diff
changeset
|
570 |
20804
db9d3991d2c6
bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20803
diff
changeset
|
571 def _paramchunk(self): |
db9d3991d2c6
bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20803
diff
changeset
|
572 """return a encoded version of all stream parameters""" |
db9d3991d2c6
bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20803
diff
changeset
|
573 blocks = [] |
20809
b93bb639451a
bundle2: support for bundling parameter value
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20808
diff
changeset
|
574 for par, value in self._params: |
28883
032c4c2f802a
pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents:
28672
diff
changeset
|
575 par = urlreq.quote(par) |
20809
b93bb639451a
bundle2: support for bundling parameter value
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20808
diff
changeset
|
576 if value is not None: |
28883
032c4c2f802a
pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents:
28672
diff
changeset
|
577 value = urlreq.quote(value) |
20809
b93bb639451a
bundle2: support for bundling parameter value
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20808
diff
changeset
|
578 par = '%s=%s' % (par, value) |
b93bb639451a
bundle2: support for bundling parameter value
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20808
diff
changeset
|
579 blocks.append(par) |
20804
db9d3991d2c6
bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20803
diff
changeset
|
580 return ' '.join(blocks) |
db9d3991d2c6
bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20803
diff
changeset
|
581 |
26396
d90c30801cdf
bundle20: extract core payload generation in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26395
diff
changeset
|
582 def _getcorechunk(self): |
d90c30801cdf
bundle20: extract core payload generation in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26395
diff
changeset
|
583 """yield chunk for the core part of the bundle |
d90c30801cdf
bundle20: extract core payload generation in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26395
diff
changeset
|
584 |
d90c30801cdf
bundle20: extract core payload generation in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26395
diff
changeset
|
585 (all but headers and parameters)""" |
d90c30801cdf
bundle20: extract core payload generation in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26395
diff
changeset
|
586 outdebug(self.ui, 'start of parts') |
d90c30801cdf
bundle20: extract core payload generation in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26395
diff
changeset
|
587 for part in self._parts: |
d90c30801cdf
bundle20: extract core payload generation in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26395
diff
changeset
|
588 outdebug(self.ui, 'bundle part: "%s"' % part.type) |
d90c30801cdf
bundle20: extract core payload generation in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26395
diff
changeset
|
589 for chunk in part.getchunks(ui=self.ui): |
d90c30801cdf
bundle20: extract core payload generation in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26395
diff
changeset
|
590 yield chunk |
d90c30801cdf
bundle20: extract core payload generation in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26395
diff
changeset
|
591 outdebug(self.ui, 'end of bundle') |
d90c30801cdf
bundle20: extract core payload generation in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26395
diff
changeset
|
592 yield _pack(_fpartheadersize, 0) |
d90c30801cdf
bundle20: extract core payload generation in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26395
diff
changeset
|
593 |
d90c30801cdf
bundle20: extract core payload generation in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26395
diff
changeset
|
594 |
24794
21f2e8f48ae1
bundle2: add a 'salvageoutput' method on bundle20
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24793
diff
changeset
|
595 def salvageoutput(self): |
21f2e8f48ae1
bundle2: add a 'salvageoutput' method on bundle20
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24793
diff
changeset
|
596 """return a list with a copy of all output parts in the bundle |
21f2e8f48ae1
bundle2: add a 'salvageoutput' method on bundle20
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24793
diff
changeset
|
597 |
21f2e8f48ae1
bundle2: add a 'salvageoutput' method on bundle20
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24793
diff
changeset
|
598 This is meant to be used during error handling to make sure we preserve |
21f2e8f48ae1
bundle2: add a 'salvageoutput' method on bundle20
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24793
diff
changeset
|
599 server output""" |
21f2e8f48ae1
bundle2: add a 'salvageoutput' method on bundle20
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24793
diff
changeset
|
600 salvaged = [] |
21f2e8f48ae1
bundle2: add a 'salvageoutput' method on bundle20
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24793
diff
changeset
|
601 for part in self._parts: |
21f2e8f48ae1
bundle2: add a 'salvageoutput' method on bundle20
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24793
diff
changeset
|
602 if part.type.startswith('output'): |
21f2e8f48ae1
bundle2: add a 'salvageoutput' method on bundle20
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24793
diff
changeset
|
603 salvaged.append(part.copy()) |
21f2e8f48ae1
bundle2: add a 'salvageoutput' method on bundle20
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24793
diff
changeset
|
604 return salvaged |
21f2e8f48ae1
bundle2: add a 'salvageoutput' method on bundle20
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24793
diff
changeset
|
605 |
21f2e8f48ae1
bundle2: add a 'salvageoutput' method on bundle20
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24793
diff
changeset
|
606 |
21013
a813caca89b3
bundle2: extract stream/unpack logic in an unpackermixin
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21005
diff
changeset
|
607 class unpackermixin(object): |
a813caca89b3
bundle2: extract stream/unpack logic in an unpackermixin
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21005
diff
changeset
|
608 """A mixin to extract bytes and struct data from a stream""" |
20802
520df53ad26a
bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20801
diff
changeset
|
609 |
21013
a813caca89b3
bundle2: extract stream/unpack logic in an unpackermixin
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21005
diff
changeset
|
610 def __init__(self, fp): |
20802
520df53ad26a
bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20801
diff
changeset
|
611 self._fp = fp |
24026
3daef83a1873
bundle2.unpackermixin: control for underlying file descriptor
Eric Sumner <ericsumner@fb.com>
parents:
23916
diff
changeset
|
612 self._seekable = (util.safehasattr(fp, 'seek') and |
3daef83a1873
bundle2.unpackermixin: control for underlying file descriptor
Eric Sumner <ericsumner@fb.com>
parents:
23916
diff
changeset
|
613 util.safehasattr(fp, 'tell')) |
20802
520df53ad26a
bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20801
diff
changeset
|
614 |
520df53ad26a
bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20801
diff
changeset
|
615 def _unpack(self, format): |
520df53ad26a
bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20801
diff
changeset
|
616 """unpack this struct format from the stream""" |
520df53ad26a
bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20801
diff
changeset
|
617 data = self._readexact(struct.calcsize(format)) |
520df53ad26a
bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20801
diff
changeset
|
618 return _unpack(format, data) |
520df53ad26a
bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20801
diff
changeset
|
619 |
520df53ad26a
bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20801
diff
changeset
|
620 def _readexact(self, size): |
520df53ad26a
bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20801
diff
changeset
|
621 """read exactly <size> bytes from the stream""" |
520df53ad26a
bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20801
diff
changeset
|
622 return changegroup.readexactly(self._fp, size) |
520df53ad26a
bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20801
diff
changeset
|
623 |
24070
de32e9881698
bundle2.unpackermixin: default value for seek() whence parameter
Eric Sumner <ericsumner@fb.com>
parents:
24048
diff
changeset
|
624 def seek(self, offset, whence=0): |
24026
3daef83a1873
bundle2.unpackermixin: control for underlying file descriptor
Eric Sumner <ericsumner@fb.com>
parents:
23916
diff
changeset
|
625 """move the underlying file pointer""" |
3daef83a1873
bundle2.unpackermixin: control for underlying file descriptor
Eric Sumner <ericsumner@fb.com>
parents:
23916
diff
changeset
|
626 if self._seekable: |
3daef83a1873
bundle2.unpackermixin: control for underlying file descriptor
Eric Sumner <ericsumner@fb.com>
parents:
23916
diff
changeset
|
627 return self._fp.seek(offset, whence) |
3daef83a1873
bundle2.unpackermixin: control for underlying file descriptor
Eric Sumner <ericsumner@fb.com>
parents:
23916
diff
changeset
|
628 else: |
3daef83a1873
bundle2.unpackermixin: control for underlying file descriptor
Eric Sumner <ericsumner@fb.com>
parents:
23916
diff
changeset
|
629 raise NotImplementedError(_('File pointer is not seekable')) |
3daef83a1873
bundle2.unpackermixin: control for underlying file descriptor
Eric Sumner <ericsumner@fb.com>
parents:
23916
diff
changeset
|
630 |
3daef83a1873
bundle2.unpackermixin: control for underlying file descriptor
Eric Sumner <ericsumner@fb.com>
parents:
23916
diff
changeset
|
631 def tell(self): |
3daef83a1873
bundle2.unpackermixin: control for underlying file descriptor
Eric Sumner <ericsumner@fb.com>
parents:
23916
diff
changeset
|
632 """return the file offset, or None if file is not seekable""" |
3daef83a1873
bundle2.unpackermixin: control for underlying file descriptor
Eric Sumner <ericsumner@fb.com>
parents:
23916
diff
changeset
|
633 if self._seekable: |
3daef83a1873
bundle2.unpackermixin: control for underlying file descriptor
Eric Sumner <ericsumner@fb.com>
parents:
23916
diff
changeset
|
634 try: |
3daef83a1873
bundle2.unpackermixin: control for underlying file descriptor
Eric Sumner <ericsumner@fb.com>
parents:
23916
diff
changeset
|
635 return self._fp.tell() |
25660
328739ea70c3
global: mass rewrite to use modern exception syntax
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25641
diff
changeset
|
636 except IOError as e: |
24026
3daef83a1873
bundle2.unpackermixin: control for underlying file descriptor
Eric Sumner <ericsumner@fb.com>
parents:
23916
diff
changeset
|
637 if e.errno == errno.ESPIPE: |
3daef83a1873
bundle2.unpackermixin: control for underlying file descriptor
Eric Sumner <ericsumner@fb.com>
parents:
23916
diff
changeset
|
638 self._seekable = False |
3daef83a1873
bundle2.unpackermixin: control for underlying file descriptor
Eric Sumner <ericsumner@fb.com>
parents:
23916
diff
changeset
|
639 else: |
3daef83a1873
bundle2.unpackermixin: control for underlying file descriptor
Eric Sumner <ericsumner@fb.com>
parents:
23916
diff
changeset
|
640 raise |
3daef83a1873
bundle2.unpackermixin: control for underlying file descriptor
Eric Sumner <ericsumner@fb.com>
parents:
23916
diff
changeset
|
641 return None |
3daef83a1873
bundle2.unpackermixin: control for underlying file descriptor
Eric Sumner <ericsumner@fb.com>
parents:
23916
diff
changeset
|
642 |
3daef83a1873
bundle2.unpackermixin: control for underlying file descriptor
Eric Sumner <ericsumner@fb.com>
parents:
23916
diff
changeset
|
643 def close(self): |
3daef83a1873
bundle2.unpackermixin: control for underlying file descriptor
Eric Sumner <ericsumner@fb.com>
parents:
23916
diff
changeset
|
644 """close underlying file""" |
3daef83a1873
bundle2.unpackermixin: control for underlying file descriptor
Eric Sumner <ericsumner@fb.com>
parents:
23916
diff
changeset
|
645 if util.safehasattr(self._fp, 'close'): |
3daef83a1873
bundle2.unpackermixin: control for underlying file descriptor
Eric Sumner <ericsumner@fb.com>
parents:
23916
diff
changeset
|
646 return self._fp.close() |
21013
a813caca89b3
bundle2: extract stream/unpack logic in an unpackermixin
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21005
diff
changeset
|
647 |
25640
39f0064a3079
bundle2.getunbundler: rename "header" to "magicstring"
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25518
diff
changeset
|
648 def getunbundler(ui, fp, magicstring=None): |
39f0064a3079
bundle2.getunbundler: rename "header" to "magicstring"
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25518
diff
changeset
|
649 """return a valid unbundler object for a given magicstring""" |
39f0064a3079
bundle2.getunbundler: rename "header" to "magicstring"
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25518
diff
changeset
|
650 if magicstring is None: |
39f0064a3079
bundle2.getunbundler: rename "header" to "magicstring"
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25518
diff
changeset
|
651 magicstring = changegroup.readexactly(fp, 4) |
39f0064a3079
bundle2.getunbundler: rename "header" to "magicstring"
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25518
diff
changeset
|
652 magic, version = magicstring[0:2], magicstring[2:4] |
24648
5cac3accdaa1
unbundle20: allow generic dispatch between unbundlers
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24642
diff
changeset
|
653 if magic != 'HG': |
26587
56b2bcea2529
error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26565
diff
changeset
|
654 raise error.Abort(_('not a Mercurial bundle')) |
24648
5cac3accdaa1
unbundle20: allow generic dispatch between unbundlers
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24642
diff
changeset
|
655 unbundlerclass = formatmap.get(version) |
5cac3accdaa1
unbundle20: allow generic dispatch between unbundlers
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24642
diff
changeset
|
656 if unbundlerclass is None: |
26587
56b2bcea2529
error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26565
diff
changeset
|
657 raise error.Abort(_('unknown bundle version %s') % version) |
24648
5cac3accdaa1
unbundle20: allow generic dispatch between unbundlers
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24642
diff
changeset
|
658 unbundler = unbundlerclass(ui, fp) |
25640
39f0064a3079
bundle2.getunbundler: rename "header" to "magicstring"
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25518
diff
changeset
|
659 indebug(ui, 'start processing of %s stream' % magicstring) |
24642
54e5c239c2d9
unbundle20: move header parsing into the 'getunbundler' function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24641
diff
changeset
|
660 return unbundler |
24641
60fecc5b14a4
unbundle20: retrieve unbundler instances through a factory function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24640
diff
changeset
|
661 |
21013
a813caca89b3
bundle2: extract stream/unpack logic in an unpackermixin
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21005
diff
changeset
|
662 class unbundle20(unpackermixin): |
a813caca89b3
bundle2: extract stream/unpack logic in an unpackermixin
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21005
diff
changeset
|
663 """interpret a bundle2 stream |
a813caca89b3
bundle2: extract stream/unpack logic in an unpackermixin
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21005
diff
changeset
|
664 |
21129
07bcbf326c8d
bundle2: use an official iterparts method to unbundle parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21066
diff
changeset
|
665 This class is fed with a binary stream and yields parts through its |
07bcbf326c8d
bundle2: use an official iterparts method to unbundle parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21066
diff
changeset
|
666 `iterparts` methods.""" |
21013
a813caca89b3
bundle2: extract stream/unpack logic in an unpackermixin
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21005
diff
changeset
|
667 |
26542
b87e4638dabf
bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26541
diff
changeset
|
668 _magicstring = 'HG20' |
b87e4638dabf
bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26541
diff
changeset
|
669 |
24642
54e5c239c2d9
unbundle20: move header parsing into the 'getunbundler' function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24641
diff
changeset
|
670 def __init__(self, ui, fp): |
21066
5ecfe76d0d96
bundle2: make header reading optional
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21062
diff
changeset
|
671 """If header is specified, we do not read it out of the stream.""" |
21013
a813caca89b3
bundle2: extract stream/unpack logic in an unpackermixin
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21005
diff
changeset
|
672 self.ui = ui |
26404
795f02a24b87
bundle2: allow compressed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26396
diff
changeset
|
673 self._decompressor = util.decompressors[None] |
26802
42f705f2c02d
bundle2: make unbundle.compressed return True when compressed
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26795
diff
changeset
|
674 self._compressed = None |
21013
a813caca89b3
bundle2: extract stream/unpack logic in an unpackermixin
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21005
diff
changeset
|
675 super(unbundle20, self).__init__(fp) |
a813caca89b3
bundle2: extract stream/unpack logic in an unpackermixin
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21005
diff
changeset
|
676 |
20802
520df53ad26a
bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20801
diff
changeset
|
677 @util.propertycache |
520df53ad26a
bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20801
diff
changeset
|
678 def params(self): |
21024
7731a2281cf0
spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents:
21020
diff
changeset
|
679 """dictionary of stream level parameters""" |
25320
697d8953b04d
bundle2: handle new line in 'indebug' function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25319
diff
changeset
|
680 indebug(self.ui, 'reading bundle2 stream parameters') |
20805
c5aaeca0cfbf
bundle2: support for unbundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20804
diff
changeset
|
681 params = {} |
c5aaeca0cfbf
bundle2: support for unbundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20804
diff
changeset
|
682 paramssize = self._unpack(_fstreamparamsize)[0] |
23011
006a81d07e57
bundle2: detect and disallow a negative chunk size
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23010
diff
changeset
|
683 if paramssize < 0: |
006a81d07e57
bundle2: detect and disallow a negative chunk size
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23010
diff
changeset
|
684 raise error.BundleValueError('negative bundle param size: %i' |
006a81d07e57
bundle2: detect and disallow a negative chunk size
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23010
diff
changeset
|
685 % paramssize) |
20805
c5aaeca0cfbf
bundle2: support for unbundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20804
diff
changeset
|
686 if paramssize: |
26541
d40029b4296e
bundle2: split parameter retrieval and processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26404
diff
changeset
|
687 params = self._readexact(paramssize) |
d40029b4296e
bundle2: split parameter retrieval and processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26404
diff
changeset
|
688 params = self._processallparams(params) |
20805
c5aaeca0cfbf
bundle2: support for unbundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20804
diff
changeset
|
689 return params |
20802
520df53ad26a
bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20801
diff
changeset
|
690 |
26541
d40029b4296e
bundle2: split parameter retrieval and processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26404
diff
changeset
|
691 def _processallparams(self, paramsblock): |
d40029b4296e
bundle2: split parameter retrieval and processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26404
diff
changeset
|
692 """""" |
d40029b4296e
bundle2: split parameter retrieval and processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26404
diff
changeset
|
693 params = {} |
d40029b4296e
bundle2: split parameter retrieval and processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26404
diff
changeset
|
694 for p in paramsblock.split(' '): |
d40029b4296e
bundle2: split parameter retrieval and processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26404
diff
changeset
|
695 p = p.split('=', 1) |
28883
032c4c2f802a
pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents:
28672
diff
changeset
|
696 p = [urlreq.unquote(i) for i in p] |
26541
d40029b4296e
bundle2: split parameter retrieval and processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26404
diff
changeset
|
697 if len(p) < 2: |
d40029b4296e
bundle2: split parameter retrieval and processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26404
diff
changeset
|
698 p.append(None) |
d40029b4296e
bundle2: split parameter retrieval and processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26404
diff
changeset
|
699 self._processparam(*p) |
d40029b4296e
bundle2: split parameter retrieval and processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26404
diff
changeset
|
700 params[p[0]] = p[1] |
d40029b4296e
bundle2: split parameter retrieval and processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26404
diff
changeset
|
701 return params |
d40029b4296e
bundle2: split parameter retrieval and processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26404
diff
changeset
|
702 |
d40029b4296e
bundle2: split parameter retrieval and processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26404
diff
changeset
|
703 |
20844
2631204d7305
bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20843
diff
changeset
|
704 def _processparam(self, name, value): |
2631204d7305
bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20843
diff
changeset
|
705 """process a parameter, applying its effect if needed |
2631204d7305
bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20843
diff
changeset
|
706 |
2631204d7305
bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20843
diff
changeset
|
707 Parameter starting with a lower case letter are advisory and will be |
2631204d7305
bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20843
diff
changeset
|
708 ignored when unknown. Those starting with an upper case letter are |
2631204d7305
bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20843
diff
changeset
|
709 mandatory and will this function will raise a KeyError when unknown. |
2631204d7305
bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20843
diff
changeset
|
710 |
2631204d7305
bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20843
diff
changeset
|
711 Note: no option are currently supported. Any input will be either |
2631204d7305
bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20843
diff
changeset
|
712 ignored or failing. |
2631204d7305
bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20843
diff
changeset
|
713 """ |
2631204d7305
bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20843
diff
changeset
|
714 if not name: |
2631204d7305
bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20843
diff
changeset
|
715 raise ValueError('empty parameter name') |
2631204d7305
bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20843
diff
changeset
|
716 if name[0] not in string.letters: |
2631204d7305
bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20843
diff
changeset
|
717 raise ValueError('non letter first character: %r' % name) |
26395
4e7b0bf9f0b1
unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26393
diff
changeset
|
718 try: |
4e7b0bf9f0b1
unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26393
diff
changeset
|
719 handler = b2streamparamsmap[name.lower()] |
4e7b0bf9f0b1
unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26393
diff
changeset
|
720 except KeyError: |
4e7b0bf9f0b1
unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26393
diff
changeset
|
721 if name[0].islower(): |
4e7b0bf9f0b1
unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26393
diff
changeset
|
722 indebug(self.ui, "ignoring unknown parameter %r" % name) |
4e7b0bf9f0b1
unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26393
diff
changeset
|
723 else: |
4e7b0bf9f0b1
unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26393
diff
changeset
|
724 raise error.BundleUnknownFeatureError(params=(name,)) |
20844
2631204d7305
bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20843
diff
changeset
|
725 else: |
26395
4e7b0bf9f0b1
unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26393
diff
changeset
|
726 handler(self, name, value) |
20844
2631204d7305
bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20843
diff
changeset
|
727 |
26542
b87e4638dabf
bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26541
diff
changeset
|
728 def _forwardchunks(self): |
b87e4638dabf
bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26541
diff
changeset
|
729 """utility to transfer a bundle2 as binary |
b87e4638dabf
bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26541
diff
changeset
|
730 |
b87e4638dabf
bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26541
diff
changeset
|
731 This is made necessary by the fact the 'getbundle' command over 'ssh' |
b87e4638dabf
bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26541
diff
changeset
|
732 have no way to know then the reply end, relying on the bundle to be |
b87e4638dabf
bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26541
diff
changeset
|
733 interpreted to know its end. This is terrible and we are sorry, but we |
b87e4638dabf
bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26541
diff
changeset
|
734 needed to move forward to get general delta enabled. |
b87e4638dabf
bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26541
diff
changeset
|
735 """ |
b87e4638dabf
bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26541
diff
changeset
|
736 yield self._magicstring |
b87e4638dabf
bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26541
diff
changeset
|
737 assert 'params' not in vars(self) |
b87e4638dabf
bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26541
diff
changeset
|
738 paramssize = self._unpack(_fstreamparamsize)[0] |
b87e4638dabf
bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26541
diff
changeset
|
739 if paramssize < 0: |
b87e4638dabf
bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26541
diff
changeset
|
740 raise error.BundleValueError('negative bundle param size: %i' |
b87e4638dabf
bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26541
diff
changeset
|
741 % paramssize) |
b87e4638dabf
bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26541
diff
changeset
|
742 yield _pack(_fstreamparamsize, paramssize) |
b87e4638dabf
bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26541
diff
changeset
|
743 if paramssize: |
b87e4638dabf
bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26541
diff
changeset
|
744 params = self._readexact(paramssize) |
b87e4638dabf
bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26541
diff
changeset
|
745 self._processallparams(params) |
b87e4638dabf
bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26541
diff
changeset
|
746 yield params |
b87e4638dabf
bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26541
diff
changeset
|
747 assert self._decompressor is util.decompressors[None] |
b87e4638dabf
bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26541
diff
changeset
|
748 # From there, payload might need to be decompressed |
b87e4638dabf
bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26541
diff
changeset
|
749 self._fp = self._decompressor(self._fp) |
b87e4638dabf
bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26541
diff
changeset
|
750 emptycount = 0 |
b87e4638dabf
bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26541
diff
changeset
|
751 while emptycount < 2: |
b87e4638dabf
bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26541
diff
changeset
|
752 # so we can brainlessly loop |
b87e4638dabf
bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26541
diff
changeset
|
753 assert _fpartheadersize == _fpayloadsize |
b87e4638dabf
bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26541
diff
changeset
|
754 size = self._unpack(_fpartheadersize)[0] |
b87e4638dabf
bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26541
diff
changeset
|
755 yield _pack(_fpartheadersize, size) |
b87e4638dabf
bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26541
diff
changeset
|
756 if size: |
b87e4638dabf
bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26541
diff
changeset
|
757 emptycount = 0 |
b87e4638dabf
bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26541
diff
changeset
|
758 else: |
b87e4638dabf
bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26541
diff
changeset
|
759 emptycount += 1 |
b87e4638dabf
bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26541
diff
changeset
|
760 continue |
b87e4638dabf
bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26541
diff
changeset
|
761 if size == flaginterrupt: |
b87e4638dabf
bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26541
diff
changeset
|
762 continue |
b87e4638dabf
bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26541
diff
changeset
|
763 elif size < 0: |
b87e4638dabf
bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26541
diff
changeset
|
764 raise error.BundleValueError('negative chunk size: %i') |
b87e4638dabf
bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26541
diff
changeset
|
765 yield self._readexact(size) |
b87e4638dabf
bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26541
diff
changeset
|
766 |
b87e4638dabf
bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26541
diff
changeset
|
767 |
21129
07bcbf326c8d
bundle2: use an official iterparts method to unbundle parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21066
diff
changeset
|
768 def iterparts(self): |
20802
520df53ad26a
bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20801
diff
changeset
|
769 """yield all parts contained in the stream""" |
520df53ad26a
bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20801
diff
changeset
|
770 # make sure param have been loaded |
520df53ad26a
bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20801
diff
changeset
|
771 self.params |
26404
795f02a24b87
bundle2: allow compressed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26396
diff
changeset
|
772 # From there, payload need to be decompressed |
795f02a24b87
bundle2: allow compressed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26396
diff
changeset
|
773 self._fp = self._decompressor(self._fp) |
25320
697d8953b04d
bundle2: handle new line in 'indebug' function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25319
diff
changeset
|
774 indebug(self.ui, 'start extraction of bundle2 parts') |
21014
a6246bba7b9e
bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21013
diff
changeset
|
775 headerblock = self._readpartheader() |
a6246bba7b9e
bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21013
diff
changeset
|
776 while headerblock is not None: |
a6246bba7b9e
bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21013
diff
changeset
|
777 part = unbundlepart(self.ui, headerblock, self._fp) |
20802
520df53ad26a
bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20801
diff
changeset
|
778 yield part |
24048
13d88b7eb3a0
bundle2: seek in part iterator
Eric Sumner <ericsumner@fb.com>
parents:
24047
diff
changeset
|
779 part.seek(0, 2) |
21014
a6246bba7b9e
bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21013
diff
changeset
|
780 headerblock = self._readpartheader() |
25320
697d8953b04d
bundle2: handle new line in 'indebug' function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25319
diff
changeset
|
781 indebug(self.ui, 'end of bundle2 stream') |
20802
520df53ad26a
bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20801
diff
changeset
|
782 |
21014
a6246bba7b9e
bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21013
diff
changeset
|
783 def _readpartheader(self): |
a6246bba7b9e
bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21013
diff
changeset
|
784 """reads a part header size and return the bytes blob |
20864
9a75d2559cff
bundle2: support unbundling empty part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20856
diff
changeset
|
785 |
21014
a6246bba7b9e
bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21013
diff
changeset
|
786 returns None if empty""" |
20864
9a75d2559cff
bundle2: support unbundling empty part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20856
diff
changeset
|
787 headersize = self._unpack(_fpartheadersize)[0] |
23011
006a81d07e57
bundle2: detect and disallow a negative chunk size
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23010
diff
changeset
|
788 if headersize < 0: |
006a81d07e57
bundle2: detect and disallow a negative chunk size
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23010
diff
changeset
|
789 raise error.BundleValueError('negative part header size: %i' |
006a81d07e57
bundle2: detect and disallow a negative chunk size
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23010
diff
changeset
|
790 % headersize) |
25320
697d8953b04d
bundle2: handle new line in 'indebug' function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25319
diff
changeset
|
791 indebug(self.ui, 'part header size: %i' % headersize) |
21014
a6246bba7b9e
bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21013
diff
changeset
|
792 if headersize: |
a6246bba7b9e
bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21013
diff
changeset
|
793 return self._readexact(headersize) |
a6246bba7b9e
bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21013
diff
changeset
|
794 return None |
20864
9a75d2559cff
bundle2: support unbundling empty part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20856
diff
changeset
|
795 |
24071
184a2f6f40da
bundle2.unbundle20: add compressed() method
Eric Sumner <ericsumner@fb.com>
parents:
24070
diff
changeset
|
796 def compressed(self): |
26802
42f705f2c02d
bundle2: make unbundle.compressed return True when compressed
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26795
diff
changeset
|
797 self.params # load params |
42f705f2c02d
bundle2: make unbundle.compressed return True when compressed
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26795
diff
changeset
|
798 return self._compressed |
20802
520df53ad26a
bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20801
diff
changeset
|
799 |
24686
e0e28e910fa3
bundle2: rename format, parts and config to final names
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24648
diff
changeset
|
800 formatmap = {'20': unbundle20} |
24648
5cac3accdaa1
unbundle20: allow generic dispatch between unbundlers
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24642
diff
changeset
|
801 |
26395
4e7b0bf9f0b1
unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26393
diff
changeset
|
802 b2streamparamsmap = {} |
4e7b0bf9f0b1
unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26393
diff
changeset
|
803 |
4e7b0bf9f0b1
unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26393
diff
changeset
|
804 def b2streamparamhandler(name): |
4e7b0bf9f0b1
unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26393
diff
changeset
|
805 """register a handler for a stream level parameter""" |
4e7b0bf9f0b1
unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26393
diff
changeset
|
806 def decorator(func): |
4e7b0bf9f0b1
unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26393
diff
changeset
|
807 assert name not in formatmap |
4e7b0bf9f0b1
unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26393
diff
changeset
|
808 b2streamparamsmap[name] = func |
4e7b0bf9f0b1
unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26393
diff
changeset
|
809 return func |
4e7b0bf9f0b1
unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26393
diff
changeset
|
810 return decorator |
4e7b0bf9f0b1
unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26393
diff
changeset
|
811 |
26404
795f02a24b87
bundle2: allow compressed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26396
diff
changeset
|
812 @b2streamparamhandler('compression') |
795f02a24b87
bundle2: allow compressed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26396
diff
changeset
|
813 def processcompression(unbundler, param, value): |
795f02a24b87
bundle2: allow compressed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26396
diff
changeset
|
814 """read compression parameter and install payload decompression""" |
795f02a24b87
bundle2: allow compressed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26396
diff
changeset
|
815 if value not in util.decompressors: |
795f02a24b87
bundle2: allow compressed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26396
diff
changeset
|
816 raise error.BundleUnknownFeatureError(params=(param,), |
795f02a24b87
bundle2: allow compressed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26396
diff
changeset
|
817 values=(value,)) |
795f02a24b87
bundle2: allow compressed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26396
diff
changeset
|
818 unbundler._decompressor = util.decompressors[value] |
26802
42f705f2c02d
bundle2: make unbundle.compressed return True when compressed
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26795
diff
changeset
|
819 if value is not None: |
42f705f2c02d
bundle2: make unbundle.compressed return True when compressed
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26795
diff
changeset
|
820 unbundler._compressed = True |
26404
795f02a24b87
bundle2: allow compressed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26396
diff
changeset
|
821 |
21005
3d38ebb586fe
bundle2: rename part to bundlepart
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21004
diff
changeset
|
822 class bundlepart(object): |
20856
8a6a86c9a5b5
bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20844
diff
changeset
|
823 """A bundle2 part contains application level payload |
8a6a86c9a5b5
bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20844
diff
changeset
|
824 |
8a6a86c9a5b5
bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20844
diff
changeset
|
825 The part `type` is used to route the part to the application level |
8a6a86c9a5b5
bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20844
diff
changeset
|
826 handler. |
21604
c399bf961cb9
bundle2: the ability to set ``data`` attribute of the part is now official
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21603
diff
changeset
|
827 |
c399bf961cb9
bundle2: the ability to set ``data`` attribute of the part is now official
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21603
diff
changeset
|
828 The part payload is contained in ``part.data``. It could be raw bytes or a |
21605
f9dabfceb259
bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21604
diff
changeset
|
829 generator of byte chunks. |
f9dabfceb259
bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21604
diff
changeset
|
830 |
f9dabfceb259
bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21604
diff
changeset
|
831 You can add parameters to the part using the ``addparam`` method. |
f9dabfceb259
bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21604
diff
changeset
|
832 Parameters can be either mandatory (default) or advisory. Remote side |
f9dabfceb259
bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21604
diff
changeset
|
833 should be able to safely ignore the advisory ones. |
f9dabfceb259
bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21604
diff
changeset
|
834 |
f9dabfceb259
bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21604
diff
changeset
|
835 Both data and parameters cannot be modified after the generation has begun. |
20856
8a6a86c9a5b5
bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20844
diff
changeset
|
836 """ |
8a6a86c9a5b5
bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20844
diff
changeset
|
837 |
20877
9e9e3a4e9261
bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20876
diff
changeset
|
838 def __init__(self, parttype, mandatoryparams=(), advisoryparams=(), |
23590
4440c7cc3728
bundle2.bundlepart: make mandatory part flag explicit in API
Eric Sumner <ericsumner@fb.com>
parents:
23586
diff
changeset
|
839 data='', mandatory=True): |
23868
405eacbecc52
bundle2: enforce parttype as alphanumerical
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23590
diff
changeset
|
840 validateparttype(parttype) |
20995
e995d104c87f
bundle2: add an integer id to part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20952
diff
changeset
|
841 self.id = None |
20856
8a6a86c9a5b5
bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20844
diff
changeset
|
842 self.type = parttype |
21604
c399bf961cb9
bundle2: the ability to set ``data`` attribute of the part is now official
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21603
diff
changeset
|
843 self._data = data |
21605
f9dabfceb259
bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21604
diff
changeset
|
844 self._mandatoryparams = list(mandatoryparams) |
f9dabfceb259
bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21604
diff
changeset
|
845 self._advisoryparams = list(advisoryparams) |
21607
054fa5176fa7
bundle2: forbid duplicate parameter keys
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21606
diff
changeset
|
846 # checking for duplicated entries |
054fa5176fa7
bundle2: forbid duplicate parameter keys
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21606
diff
changeset
|
847 self._seenparams = set() |
054fa5176fa7
bundle2: forbid duplicate parameter keys
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21606
diff
changeset
|
848 for pname, __ in self._mandatoryparams + self._advisoryparams: |
054fa5176fa7
bundle2: forbid duplicate parameter keys
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21606
diff
changeset
|
849 if pname in self._seenparams: |
054fa5176fa7
bundle2: forbid duplicate parameter keys
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21606
diff
changeset
|
850 raise RuntimeError('duplicated params: %s' % pname) |
054fa5176fa7
bundle2: forbid duplicate parameter keys
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21606
diff
changeset
|
851 self._seenparams.add(pname) |
21601
7ff01befc7ec
bundle2: track life cycle of parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21600
diff
changeset
|
852 # status of the part's generation: |
7ff01befc7ec
bundle2: track life cycle of parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21600
diff
changeset
|
853 # - None: not started, |
7ff01befc7ec
bundle2: track life cycle of parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21600
diff
changeset
|
854 # - False: currently generated, |
7ff01befc7ec
bundle2: track life cycle of parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21600
diff
changeset
|
855 # - True: generation done. |
7ff01befc7ec
bundle2: track life cycle of parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21600
diff
changeset
|
856 self._generated = None |
23590
4440c7cc3728
bundle2.bundlepart: make mandatory part flag explicit in API
Eric Sumner <ericsumner@fb.com>
parents:
23586
diff
changeset
|
857 self.mandatory = mandatory |
20856
8a6a86c9a5b5
bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20844
diff
changeset
|
858 |
24793
2ec894582ea2
bundle2: add a 'copy' method on parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24743
diff
changeset
|
859 def copy(self): |
2ec894582ea2
bundle2: add a 'copy' method on parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24743
diff
changeset
|
860 """return a copy of the part |
2ec894582ea2
bundle2: add a 'copy' method on parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24743
diff
changeset
|
861 |
2ec894582ea2
bundle2: add a 'copy' method on parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24743
diff
changeset
|
862 The new part have the very same content but no partid assigned yet. |
2ec894582ea2
bundle2: add a 'copy' method on parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24743
diff
changeset
|
863 Parts with generated data cannot be copied.""" |
2ec894582ea2
bundle2: add a 'copy' method on parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24743
diff
changeset
|
864 assert not util.safehasattr(self.data, 'next') |
2ec894582ea2
bundle2: add a 'copy' method on parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24743
diff
changeset
|
865 return self.__class__(self.type, self._mandatoryparams, |
2ec894582ea2
bundle2: add a 'copy' method on parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24743
diff
changeset
|
866 self._advisoryparams, self._data, self.mandatory) |
2ec894582ea2
bundle2: add a 'copy' method on parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24743
diff
changeset
|
867 |
21604
c399bf961cb9
bundle2: the ability to set ``data`` attribute of the part is now official
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21603
diff
changeset
|
868 # methods used to defines the part content |
27879
52a4ad62b006
cleanup: use modern @property/@foo.setter property specification
Augie Fackler <augie@google.com>
parents:
27751
diff
changeset
|
869 @property |
52a4ad62b006
cleanup: use modern @property/@foo.setter property specification
Augie Fackler <augie@google.com>
parents:
27751
diff
changeset
|
870 def data(self): |
52a4ad62b006
cleanup: use modern @property/@foo.setter property specification
Augie Fackler <augie@google.com>
parents:
27751
diff
changeset
|
871 return self._data |
52a4ad62b006
cleanup: use modern @property/@foo.setter property specification
Augie Fackler <augie@google.com>
parents:
27751
diff
changeset
|
872 |
52a4ad62b006
cleanup: use modern @property/@foo.setter property specification
Augie Fackler <augie@google.com>
parents:
27751
diff
changeset
|
873 @data.setter |
52a4ad62b006
cleanup: use modern @property/@foo.setter property specification
Augie Fackler <augie@google.com>
parents:
27751
diff
changeset
|
874 def data(self, data): |
21604
c399bf961cb9
bundle2: the ability to set ``data`` attribute of the part is now official
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21603
diff
changeset
|
875 if self._generated is not None: |
21618
7568f5c1c801
bundle2: move exception classes into the error module
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21617
diff
changeset
|
876 raise error.ReadOnlyPartError('part is being generated') |
21604
c399bf961cb9
bundle2: the ability to set ``data`` attribute of the part is now official
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21603
diff
changeset
|
877 self._data = data |
c399bf961cb9
bundle2: the ability to set ``data`` attribute of the part is now official
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21603
diff
changeset
|
878 |
21605
f9dabfceb259
bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21604
diff
changeset
|
879 @property |
f9dabfceb259
bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21604
diff
changeset
|
880 def mandatoryparams(self): |
f9dabfceb259
bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21604
diff
changeset
|
881 # make it an immutable tuple to force people through ``addparam`` |
f9dabfceb259
bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21604
diff
changeset
|
882 return tuple(self._mandatoryparams) |
f9dabfceb259
bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21604
diff
changeset
|
883 |
f9dabfceb259
bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21604
diff
changeset
|
884 @property |
f9dabfceb259
bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21604
diff
changeset
|
885 def advisoryparams(self): |
f9dabfceb259
bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21604
diff
changeset
|
886 # make it an immutable tuple to force people through ``addparam`` |
f9dabfceb259
bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21604
diff
changeset
|
887 return tuple(self._advisoryparams) |
f9dabfceb259
bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21604
diff
changeset
|
888 |
f9dabfceb259
bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21604
diff
changeset
|
889 def addparam(self, name, value='', mandatory=True): |
f9dabfceb259
bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21604
diff
changeset
|
890 if self._generated is not None: |
21618
7568f5c1c801
bundle2: move exception classes into the error module
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21617
diff
changeset
|
891 raise error.ReadOnlyPartError('part is being generated') |
21607
054fa5176fa7
bundle2: forbid duplicate parameter keys
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21606
diff
changeset
|
892 if name in self._seenparams: |
054fa5176fa7
bundle2: forbid duplicate parameter keys
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21606
diff
changeset
|
893 raise ValueError('duplicated params: %s' % name) |
054fa5176fa7
bundle2: forbid duplicate parameter keys
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21606
diff
changeset
|
894 self._seenparams.add(name) |
21605
f9dabfceb259
bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21604
diff
changeset
|
895 params = self._advisoryparams |
f9dabfceb259
bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21604
diff
changeset
|
896 if mandatory: |
f9dabfceb259
bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21604
diff
changeset
|
897 params = self._mandatoryparams |
f9dabfceb259
bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21604
diff
changeset
|
898 params.append((name, value)) |
f9dabfceb259
bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21604
diff
changeset
|
899 |
21601
7ff01befc7ec
bundle2: track life cycle of parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21600
diff
changeset
|
900 # methods used to generates the bundle2 stream |
25321
b44ee346211f
bundle2: add debug output for part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25320
diff
changeset
|
901 def getchunks(self, ui): |
21601
7ff01befc7ec
bundle2: track life cycle of parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21600
diff
changeset
|
902 if self._generated is not None: |
7ff01befc7ec
bundle2: track life cycle of parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21600
diff
changeset
|
903 raise RuntimeError('part can only be consumed once') |
7ff01befc7ec
bundle2: track life cycle of parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21600
diff
changeset
|
904 self._generated = False |
25323
21a25fb81d2c
bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25322
diff
changeset
|
905 |
21a25fb81d2c
bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25322
diff
changeset
|
906 if ui.debugflag: |
21a25fb81d2c
bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25322
diff
changeset
|
907 msg = ['bundle2-output-part: "%s"' % self.type] |
21a25fb81d2c
bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25322
diff
changeset
|
908 if not self.mandatory: |
21a25fb81d2c
bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25322
diff
changeset
|
909 msg.append(' (advisory)') |
21a25fb81d2c
bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25322
diff
changeset
|
910 nbmp = len(self.mandatoryparams) |
21a25fb81d2c
bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25322
diff
changeset
|
911 nbap = len(self.advisoryparams) |
21a25fb81d2c
bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25322
diff
changeset
|
912 if nbmp or nbap: |
21a25fb81d2c
bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25322
diff
changeset
|
913 msg.append(' (params:') |
21a25fb81d2c
bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25322
diff
changeset
|
914 if nbmp: |
21a25fb81d2c
bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25322
diff
changeset
|
915 msg.append(' %i mandatory' % nbmp) |
21a25fb81d2c
bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25322
diff
changeset
|
916 if nbap: |
21a25fb81d2c
bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25322
diff
changeset
|
917 msg.append(' %i advisory' % nbmp) |
21a25fb81d2c
bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25322
diff
changeset
|
918 msg.append(')') |
21a25fb81d2c
bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25322
diff
changeset
|
919 if not self.data: |
21a25fb81d2c
bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25322
diff
changeset
|
920 msg.append(' empty payload') |
21a25fb81d2c
bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25322
diff
changeset
|
921 elif util.safehasattr(self.data, 'next'): |
21a25fb81d2c
bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25322
diff
changeset
|
922 msg.append(' streamed payload') |
21a25fb81d2c
bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25322
diff
changeset
|
923 else: |
21a25fb81d2c
bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25322
diff
changeset
|
924 msg.append(' %i bytes payload' % len(self.data)) |
21a25fb81d2c
bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25322
diff
changeset
|
925 msg.append('\n') |
21a25fb81d2c
bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25322
diff
changeset
|
926 ui.debug(''.join(msg)) |
21a25fb81d2c
bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25322
diff
changeset
|
927 |
20877
9e9e3a4e9261
bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20876
diff
changeset
|
928 #### header |
23590
4440c7cc3728
bundle2.bundlepart: make mandatory part flag explicit in API
Eric Sumner <ericsumner@fb.com>
parents:
23586
diff
changeset
|
929 if self.mandatory: |
4440c7cc3728
bundle2.bundlepart: make mandatory part flag explicit in API
Eric Sumner <ericsumner@fb.com>
parents:
23586
diff
changeset
|
930 parttype = self.type.upper() |
4440c7cc3728
bundle2.bundlepart: make mandatory part flag explicit in API
Eric Sumner <ericsumner@fb.com>
parents:
23586
diff
changeset
|
931 else: |
4440c7cc3728
bundle2.bundlepart: make mandatory part flag explicit in API
Eric Sumner <ericsumner@fb.com>
parents:
23586
diff
changeset
|
932 parttype = self.type.lower() |
25321
b44ee346211f
bundle2: add debug output for part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25320
diff
changeset
|
933 outdebug(ui, 'part %s: "%s"' % (self.id, parttype)) |
20877
9e9e3a4e9261
bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20876
diff
changeset
|
934 ## parttype |
23590
4440c7cc3728
bundle2.bundlepart: make mandatory part flag explicit in API
Eric Sumner <ericsumner@fb.com>
parents:
23586
diff
changeset
|
935 header = [_pack(_fparttypesize, len(parttype)), |
4440c7cc3728
bundle2.bundlepart: make mandatory part flag explicit in API
Eric Sumner <ericsumner@fb.com>
parents:
23586
diff
changeset
|
936 parttype, _pack(_fpartid, self.id), |
20856
8a6a86c9a5b5
bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20844
diff
changeset
|
937 ] |
20877
9e9e3a4e9261
bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20876
diff
changeset
|
938 ## parameters |
9e9e3a4e9261
bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20876
diff
changeset
|
939 # count |
9e9e3a4e9261
bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20876
diff
changeset
|
940 manpar = self.mandatoryparams |
9e9e3a4e9261
bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20876
diff
changeset
|
941 advpar = self.advisoryparams |
9e9e3a4e9261
bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20876
diff
changeset
|
942 header.append(_pack(_fpartparamcount, len(manpar), len(advpar))) |
9e9e3a4e9261
bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20876
diff
changeset
|
943 # size |
9e9e3a4e9261
bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20876
diff
changeset
|
944 parsizes = [] |
9e9e3a4e9261
bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20876
diff
changeset
|
945 for key, value in manpar: |
9e9e3a4e9261
bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20876
diff
changeset
|
946 parsizes.append(len(key)) |
9e9e3a4e9261
bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20876
diff
changeset
|
947 parsizes.append(len(value)) |
9e9e3a4e9261
bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20876
diff
changeset
|
948 for key, value in advpar: |
9e9e3a4e9261
bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20876
diff
changeset
|
949 parsizes.append(len(key)) |
9e9e3a4e9261
bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20876
diff
changeset
|
950 parsizes.append(len(value)) |
9e9e3a4e9261
bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20876
diff
changeset
|
951 paramsizes = _pack(_makefpartparamsizes(len(parsizes) / 2), *parsizes) |
9e9e3a4e9261
bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20876
diff
changeset
|
952 header.append(paramsizes) |
9e9e3a4e9261
bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20876
diff
changeset
|
953 # key, value |
9e9e3a4e9261
bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20876
diff
changeset
|
954 for key, value in manpar: |
9e9e3a4e9261
bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20876
diff
changeset
|
955 header.append(key) |
9e9e3a4e9261
bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20876
diff
changeset
|
956 header.append(value) |
9e9e3a4e9261
bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20876
diff
changeset
|
957 for key, value in advpar: |
9e9e3a4e9261
bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20876
diff
changeset
|
958 header.append(key) |
9e9e3a4e9261
bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20876
diff
changeset
|
959 header.append(value) |
9e9e3a4e9261
bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20876
diff
changeset
|
960 ## finalize header |
20856
8a6a86c9a5b5
bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20844
diff
changeset
|
961 headerchunk = ''.join(header) |
25321
b44ee346211f
bundle2: add debug output for part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25320
diff
changeset
|
962 outdebug(ui, 'header chunk size: %i' % len(headerchunk)) |
20856
8a6a86c9a5b5
bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20844
diff
changeset
|
963 yield _pack(_fpartheadersize, len(headerchunk)) |
8a6a86c9a5b5
bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20844
diff
changeset
|
964 yield headerchunk |
20877
9e9e3a4e9261
bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20876
diff
changeset
|
965 ## payload |
23067
420a051616ce
bundle2: transmit exception during part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23066
diff
changeset
|
966 try: |
420a051616ce
bundle2: transmit exception during part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23066
diff
changeset
|
967 for chunk in self._payloadchunks(): |
25321
b44ee346211f
bundle2: add debug output for part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25320
diff
changeset
|
968 outdebug(ui, 'payload chunk size: %i' % len(chunk)) |
23067
420a051616ce
bundle2: transmit exception during part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23066
diff
changeset
|
969 yield _pack(_fpayloadsize, len(chunk)) |
420a051616ce
bundle2: transmit exception during part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23066
diff
changeset
|
970 yield chunk |
26144
4bc3707f3e67
bundle2: don't try to recover from a GeneratorExit (issue4785)
Augie Fackler <augie@google.com>
parents:
25919
diff
changeset
|
971 except GeneratorExit: |
4bc3707f3e67
bundle2: don't try to recover from a GeneratorExit (issue4785)
Augie Fackler <augie@google.com>
parents:
25919
diff
changeset
|
972 # GeneratorExit means that nobody is listening for our |
4bc3707f3e67
bundle2: don't try to recover from a GeneratorExit (issue4785)
Augie Fackler <augie@google.com>
parents:
25919
diff
changeset
|
973 # results anyway, so just bail quickly rather than trying |
4bc3707f3e67
bundle2: don't try to recover from a GeneratorExit (issue4785)
Augie Fackler <augie@google.com>
parents:
25919
diff
changeset
|
974 # to produce an error part. |
4bc3707f3e67
bundle2: don't try to recover from a GeneratorExit (issue4785)
Augie Fackler <augie@google.com>
parents:
25919
diff
changeset
|
975 ui.debug('bundle2-generatorexit\n') |
4bc3707f3e67
bundle2: don't try to recover from a GeneratorExit (issue4785)
Augie Fackler <augie@google.com>
parents:
25919
diff
changeset
|
976 raise |
25660
328739ea70c3
global: mass rewrite to use modern exception syntax
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25641
diff
changeset
|
977 except BaseException as exc: |
23067
420a051616ce
bundle2: transmit exception during part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23066
diff
changeset
|
978 # backup exception data for later |
25324
deed9c6b12d2
bundle2: add generic debug output regarding generated interruption
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25323
diff
changeset
|
979 ui.debug('bundle2-input-stream-interrupt: encoding exception %s' |
deed9c6b12d2
bundle2: add generic debug output regarding generated interruption
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25323
diff
changeset
|
980 % exc) |
23067
420a051616ce
bundle2: transmit exception during part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23066
diff
changeset
|
981 exc_info = sys.exc_info() |
420a051616ce
bundle2: transmit exception during part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23066
diff
changeset
|
982 msg = 'unexpected error: %s' % exc |
24686
e0e28e910fa3
bundle2: rename format, parts and config to final names
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24648
diff
changeset
|
983 interpart = bundlepart('error:abort', [('message', msg)], |
23590
4440c7cc3728
bundle2.bundlepart: make mandatory part flag explicit in API
Eric Sumner <ericsumner@fb.com>
parents:
23586
diff
changeset
|
984 mandatory=False) |
23067
420a051616ce
bundle2: transmit exception during part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23066
diff
changeset
|
985 interpart.id = 0 |
420a051616ce
bundle2: transmit exception during part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23066
diff
changeset
|
986 yield _pack(_fpayloadsize, -1) |
25321
b44ee346211f
bundle2: add debug output for part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25320
diff
changeset
|
987 for chunk in interpart.getchunks(ui=ui): |
23067
420a051616ce
bundle2: transmit exception during part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23066
diff
changeset
|
988 yield chunk |
25321
b44ee346211f
bundle2: add debug output for part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25320
diff
changeset
|
989 outdebug(ui, 'closing payload chunk') |
23067
420a051616ce
bundle2: transmit exception during part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23066
diff
changeset
|
990 # abort current part payload |
420a051616ce
bundle2: transmit exception during part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23066
diff
changeset
|
991 yield _pack(_fpayloadsize, 0) |
420a051616ce
bundle2: transmit exception during part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23066
diff
changeset
|
992 raise exc_info[0], exc_info[1], exc_info[2] |
21000
4cae06ae1562
bundle2: extract a _payloadchunks method for part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20998
diff
changeset
|
993 # end of payload |
25321
b44ee346211f
bundle2: add debug output for part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25320
diff
changeset
|
994 outdebug(ui, 'closing payload chunk') |
21000
4cae06ae1562
bundle2: extract a _payloadchunks method for part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20998
diff
changeset
|
995 yield _pack(_fpayloadsize, 0) |
21601
7ff01befc7ec
bundle2: track life cycle of parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21600
diff
changeset
|
996 self._generated = True |
21000
4cae06ae1562
bundle2: extract a _payloadchunks method for part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20998
diff
changeset
|
997 |
4cae06ae1562
bundle2: extract a _payloadchunks method for part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20998
diff
changeset
|
998 def _payloadchunks(self): |
4cae06ae1562
bundle2: extract a _payloadchunks method for part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20998
diff
changeset
|
999 """yield chunks of a the part payload |
4cae06ae1562
bundle2: extract a _payloadchunks method for part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20998
diff
changeset
|
1000 |
4cae06ae1562
bundle2: extract a _payloadchunks method for part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20998
diff
changeset
|
1001 Exists to handle the different methods to provide data to a part.""" |
20876
ddd56f3eb786
bundle2: support for bundling and unbundling payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20864
diff
changeset
|
1002 # we only support fixed size data now. |
ddd56f3eb786
bundle2: support for bundling and unbundling payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20864
diff
changeset
|
1003 # This will be improved in the future. |
21001
c93bb6a08fa1
bundle2: support chunk iterator as part data
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21000
diff
changeset
|
1004 if util.safehasattr(self.data, 'next'): |
c93bb6a08fa1
bundle2: support chunk iterator as part data
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21000
diff
changeset
|
1005 buff = util.chunkbuffer(self.data) |
c93bb6a08fa1
bundle2: support chunk iterator as part data
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21000
diff
changeset
|
1006 chunk = buff.read(preferedchunksize) |
c93bb6a08fa1
bundle2: support chunk iterator as part data
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21000
diff
changeset
|
1007 while chunk: |
c93bb6a08fa1
bundle2: support chunk iterator as part data
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21000
diff
changeset
|
1008 yield chunk |
c93bb6a08fa1
bundle2: support chunk iterator as part data
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21000
diff
changeset
|
1009 chunk = buff.read(preferedchunksize) |
c93bb6a08fa1
bundle2: support chunk iterator as part data
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21000
diff
changeset
|
1010 elif len(self.data): |
20876
ddd56f3eb786
bundle2: support for bundling and unbundling payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20864
diff
changeset
|
1011 yield self.data |
20802
520df53ad26a
bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20801
diff
changeset
|
1012 |
23066
ad144882318d
bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23029
diff
changeset
|
1013 |
ad144882318d
bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23029
diff
changeset
|
1014 flaginterrupt = -1 |
ad144882318d
bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23029
diff
changeset
|
1015 |
ad144882318d
bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23029
diff
changeset
|
1016 class interrupthandler(unpackermixin): |
ad144882318d
bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23029
diff
changeset
|
1017 """read one part and process it with restricted capability |
ad144882318d
bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23029
diff
changeset
|
1018 |
ad144882318d
bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23029
diff
changeset
|
1019 This allows to transmit exception raised on the producer size during part |
ad144882318d
bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23029
diff
changeset
|
1020 iteration while the consumer is reading a part. |
ad144882318d
bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23029
diff
changeset
|
1021 |
ad144882318d
bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23029
diff
changeset
|
1022 Part processed in this manner only have access to a ui object,""" |
ad144882318d
bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23029
diff
changeset
|
1023 |
ad144882318d
bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23029
diff
changeset
|
1024 def __init__(self, ui, fp): |
ad144882318d
bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23029
diff
changeset
|
1025 super(interrupthandler, self).__init__(fp) |
ad144882318d
bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23029
diff
changeset
|
1026 self.ui = ui |
ad144882318d
bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23029
diff
changeset
|
1027 |
ad144882318d
bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23029
diff
changeset
|
1028 def _readpartheader(self): |
ad144882318d
bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23029
diff
changeset
|
1029 """reads a part header size and return the bytes blob |
ad144882318d
bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23029
diff
changeset
|
1030 |
ad144882318d
bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23029
diff
changeset
|
1031 returns None if empty""" |
ad144882318d
bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23029
diff
changeset
|
1032 headersize = self._unpack(_fpartheadersize)[0] |
ad144882318d
bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23029
diff
changeset
|
1033 if headersize < 0: |
ad144882318d
bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23029
diff
changeset
|
1034 raise error.BundleValueError('negative part header size: %i' |
ad144882318d
bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23029
diff
changeset
|
1035 % headersize) |
25318
64dd8f25fcf4
bundle2: introduce a specific function for debug messages while unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25317
diff
changeset
|
1036 indebug(self.ui, 'part header size: %i\n' % headersize) |
23066
ad144882318d
bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23029
diff
changeset
|
1037 if headersize: |
ad144882318d
bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23029
diff
changeset
|
1038 return self._readexact(headersize) |
ad144882318d
bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23029
diff
changeset
|
1039 return None |
ad144882318d
bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23029
diff
changeset
|
1040 |
ad144882318d
bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23029
diff
changeset
|
1041 def __call__(self): |
25335
8f7137a85a0e
bundle2: add generic debug output regarding processed interruption
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25334
diff
changeset
|
1042 |
8f7137a85a0e
bundle2: add generic debug output regarding processed interruption
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25334
diff
changeset
|
1043 self.ui.debug('bundle2-input-stream-interrupt:' |
8f7137a85a0e
bundle2: add generic debug output regarding processed interruption
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25334
diff
changeset
|
1044 ' opening out of band context\n') |
25320
697d8953b04d
bundle2: handle new line in 'indebug' function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25319
diff
changeset
|
1045 indebug(self.ui, 'bundle2 stream interruption, looking for a part.') |
23066
ad144882318d
bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23029
diff
changeset
|
1046 headerblock = self._readpartheader() |
ad144882318d
bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23029
diff
changeset
|
1047 if headerblock is None: |
25320
697d8953b04d
bundle2: handle new line in 'indebug' function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25319
diff
changeset
|
1048 indebug(self.ui, 'no part found during interruption.') |
23066
ad144882318d
bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23029
diff
changeset
|
1049 return |
ad144882318d
bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23029
diff
changeset
|
1050 part = unbundlepart(self.ui, headerblock, self._fp) |
ad144882318d
bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23029
diff
changeset
|
1051 op = interruptoperation(self.ui) |
ad144882318d
bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23029
diff
changeset
|
1052 _processpart(op, part) |
25335
8f7137a85a0e
bundle2: add generic debug output regarding processed interruption
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25334
diff
changeset
|
1053 self.ui.debug('bundle2-input-stream-interrupt:' |
8f7137a85a0e
bundle2: add generic debug output regarding processed interruption
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25334
diff
changeset
|
1054 ' closing out of band context\n') |
23066
ad144882318d
bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23029
diff
changeset
|
1055 |
ad144882318d
bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23029
diff
changeset
|
1056 class interruptoperation(object): |
ad144882318d
bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23029
diff
changeset
|
1057 """A limited operation to be use by part handler during interruption |
ad144882318d
bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23029
diff
changeset
|
1058 |
ad144882318d
bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23029
diff
changeset
|
1059 It only have access to an ui object. |
ad144882318d
bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23029
diff
changeset
|
1060 """ |
ad144882318d
bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23029
diff
changeset
|
1061 |
ad144882318d
bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23029
diff
changeset
|
1062 def __init__(self, ui): |
ad144882318d
bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23029
diff
changeset
|
1063 self.ui = ui |
ad144882318d
bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23029
diff
changeset
|
1064 self.reply = None |
24878
e530cde6d115
bundle2: disable ouput capture unless we use http (issue4613 issue4615)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24851
diff
changeset
|
1065 self.captureoutput = False |
23066
ad144882318d
bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23029
diff
changeset
|
1066 |
ad144882318d
bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23029
diff
changeset
|
1067 @property |
ad144882318d
bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23029
diff
changeset
|
1068 def repo(self): |
ad144882318d
bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23029
diff
changeset
|
1069 raise RuntimeError('no repo access from stream interruption') |
ad144882318d
bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23029
diff
changeset
|
1070 |
ad144882318d
bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23029
diff
changeset
|
1071 def gettransaction(self): |
ad144882318d
bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23029
diff
changeset
|
1072 raise TransactionUnavailable('no repo access from stream interruption') |
ad144882318d
bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23029
diff
changeset
|
1073 |
21014
a6246bba7b9e
bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21013
diff
changeset
|
1074 class unbundlepart(unpackermixin): |
a6246bba7b9e
bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21013
diff
changeset
|
1075 """a bundle part read from a bundle""" |
a6246bba7b9e
bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21013
diff
changeset
|
1076 |
a6246bba7b9e
bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21013
diff
changeset
|
1077 def __init__(self, ui, header, fp): |
a6246bba7b9e
bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21013
diff
changeset
|
1078 super(unbundlepart, self).__init__(fp) |
a6246bba7b9e
bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21013
diff
changeset
|
1079 self.ui = ui |
a6246bba7b9e
bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21013
diff
changeset
|
1080 # unbundle state attr |
a6246bba7b9e
bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21013
diff
changeset
|
1081 self._headerdata = header |
21015
14dd49260246
bundle2: move the fromheader closure into the class itself
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21014
diff
changeset
|
1082 self._headeroffset = 0 |
21019
3dc09f831a2e
bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21016
diff
changeset
|
1083 self._initialized = False |
3dc09f831a2e
bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21016
diff
changeset
|
1084 self.consumed = False |
21014
a6246bba7b9e
bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21013
diff
changeset
|
1085 # part data |
a6246bba7b9e
bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21013
diff
changeset
|
1086 self.id = None |
a6246bba7b9e
bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21013
diff
changeset
|
1087 self.type = None |
a6246bba7b9e
bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21013
diff
changeset
|
1088 self.mandatoryparams = None |
a6246bba7b9e
bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21013
diff
changeset
|
1089 self.advisoryparams = None |
21610
d6056805f8f4
bundle2: introduce a ``params`` dictionary on unbundled parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21609
diff
changeset
|
1090 self.params = None |
21612
f221eb0531d9
bundle2: expose mandatory params in a mandatorykeys attribute
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21611
diff
changeset
|
1091 self.mandatorykeys = () |
21019
3dc09f831a2e
bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21016
diff
changeset
|
1092 self._payloadstream = None |
3dc09f831a2e
bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21016
diff
changeset
|
1093 self._readheader() |
23585
94b25d71dd0f
bundle2.unbundlepart: decouple mandatory from parttype
Eric Sumner <ericsumner@fb.com>
parents:
23439
diff
changeset
|
1094 self._mandatory = None |
24035
7eb26415bef6
bundle2.unbundlepart: keep an index of chunks and their locations
Eric Sumner <ericsumner@fb.com>
parents:
24034
diff
changeset
|
1095 self._chunkindex = [] #(payload, file) position tuples for chunk starts |
24036
c7601086338a
bundle2.unbundlepart: tell() implementation
Eric Sumner <ericsumner@fb.com>
parents:
24035
diff
changeset
|
1096 self._pos = 0 |
21014
a6246bba7b9e
bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21013
diff
changeset
|
1097 |
21015
14dd49260246
bundle2: move the fromheader closure into the class itself
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21014
diff
changeset
|
1098 def _fromheader(self, size): |
14dd49260246
bundle2: move the fromheader closure into the class itself
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21014
diff
changeset
|
1099 """return the next <size> byte from the header""" |
14dd49260246
bundle2: move the fromheader closure into the class itself
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21014
diff
changeset
|
1100 offset = self._headeroffset |
14dd49260246
bundle2: move the fromheader closure into the class itself
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21014
diff
changeset
|
1101 data = self._headerdata[offset:(offset + size)] |
21019
3dc09f831a2e
bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21016
diff
changeset
|
1102 self._headeroffset = offset + size |
21015
14dd49260246
bundle2: move the fromheader closure into the class itself
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21014
diff
changeset
|
1103 return data |
14dd49260246
bundle2: move the fromheader closure into the class itself
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21014
diff
changeset
|
1104 |
21016
b477afb1c81e
bundle2: move unpackheader closure into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21015
diff
changeset
|
1105 def _unpackheader(self, format): |
b477afb1c81e
bundle2: move unpackheader closure into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21015
diff
changeset
|
1106 """read given format from header |
b477afb1c81e
bundle2: move unpackheader closure into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21015
diff
changeset
|
1107 |
b477afb1c81e
bundle2: move unpackheader closure into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21015
diff
changeset
|
1108 This automatically compute the size of the format to read.""" |
b477afb1c81e
bundle2: move unpackheader closure into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21015
diff
changeset
|
1109 data = self._fromheader(struct.calcsize(format)) |
b477afb1c81e
bundle2: move unpackheader closure into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21015
diff
changeset
|
1110 return _unpack(format, data) |
b477afb1c81e
bundle2: move unpackheader closure into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21015
diff
changeset
|
1111 |
21608
3cb96ca90c17
bundle2: introduce an ``_initparams`` method
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21607
diff
changeset
|
1112 def _initparams(self, mandatoryparams, advisoryparams): |
3cb96ca90c17
bundle2: introduce an ``_initparams`` method
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21607
diff
changeset
|
1113 """internal function to setup all logic related parameters""" |
21609
63cc2594ef8a
bundle2: make sure unbundled part param are read-only
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21608
diff
changeset
|
1114 # make it read only to prevent people touching it by mistake. |
63cc2594ef8a
bundle2: make sure unbundled part param are read-only
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21608
diff
changeset
|
1115 self.mandatoryparams = tuple(mandatoryparams) |
63cc2594ef8a
bundle2: make sure unbundled part param are read-only
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21608
diff
changeset
|
1116 self.advisoryparams = tuple(advisoryparams) |
21610
d6056805f8f4
bundle2: introduce a ``params`` dictionary on unbundled parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21609
diff
changeset
|
1117 # user friendly UI |
d6056805f8f4
bundle2: introduce a ``params`` dictionary on unbundled parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21609
diff
changeset
|
1118 self.params = dict(self.mandatoryparams) |
d6056805f8f4
bundle2: introduce a ``params`` dictionary on unbundled parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21609
diff
changeset
|
1119 self.params.update(dict(self.advisoryparams)) |
21612
f221eb0531d9
bundle2: expose mandatory params in a mandatorykeys attribute
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21611
diff
changeset
|
1120 self.mandatorykeys = frozenset(p[0] for p in mandatoryparams) |
21608
3cb96ca90c17
bundle2: introduce an ``_initparams`` method
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21607
diff
changeset
|
1121 |
24035
7eb26415bef6
bundle2.unbundlepart: keep an index of chunks and their locations
Eric Sumner <ericsumner@fb.com>
parents:
24034
diff
changeset
|
1122 def _payloadchunks(self, chunknum=0): |
7eb26415bef6
bundle2.unbundlepart: keep an index of chunks and their locations
Eric Sumner <ericsumner@fb.com>
parents:
24034
diff
changeset
|
1123 '''seek to specified chunk and start yielding data''' |
7eb26415bef6
bundle2.unbundlepart: keep an index of chunks and their locations
Eric Sumner <ericsumner@fb.com>
parents:
24034
diff
changeset
|
1124 if len(self._chunkindex) == 0: |
7eb26415bef6
bundle2.unbundlepart: keep an index of chunks and their locations
Eric Sumner <ericsumner@fb.com>
parents:
24034
diff
changeset
|
1125 assert chunknum == 0, 'Must start with chunk 0' |
7eb26415bef6
bundle2.unbundlepart: keep an index of chunks and their locations
Eric Sumner <ericsumner@fb.com>
parents:
24034
diff
changeset
|
1126 self._chunkindex.append((0, super(unbundlepart, self).tell())) |
7eb26415bef6
bundle2.unbundlepart: keep an index of chunks and their locations
Eric Sumner <ericsumner@fb.com>
parents:
24034
diff
changeset
|
1127 else: |
7eb26415bef6
bundle2.unbundlepart: keep an index of chunks and their locations
Eric Sumner <ericsumner@fb.com>
parents:
24034
diff
changeset
|
1128 assert chunknum < len(self._chunkindex), \ |
7eb26415bef6
bundle2.unbundlepart: keep an index of chunks and their locations
Eric Sumner <ericsumner@fb.com>
parents:
24034
diff
changeset
|
1129 'Unknown chunk %d' % chunknum |
7eb26415bef6
bundle2.unbundlepart: keep an index of chunks and their locations
Eric Sumner <ericsumner@fb.com>
parents:
24034
diff
changeset
|
1130 super(unbundlepart, self).seek(self._chunkindex[chunknum][1]) |
7eb26415bef6
bundle2.unbundlepart: keep an index of chunks and their locations
Eric Sumner <ericsumner@fb.com>
parents:
24034
diff
changeset
|
1131 |
7eb26415bef6
bundle2.unbundlepart: keep an index of chunks and their locations
Eric Sumner <ericsumner@fb.com>
parents:
24034
diff
changeset
|
1132 pos = self._chunkindex[chunknum][0] |
24034
9881a1437799
bundle2.unbundlepart: raise payloadchunks from a closure to a method
Eric Sumner <ericsumner@fb.com>
parents:
24026
diff
changeset
|
1133 payloadsize = self._unpack(_fpayloadsize)[0] |
25320
697d8953b04d
bundle2: handle new line in 'indebug' function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25319
diff
changeset
|
1134 indebug(self.ui, 'payload chunk size: %i' % payloadsize) |
24034
9881a1437799
bundle2.unbundlepart: raise payloadchunks from a closure to a method
Eric Sumner <ericsumner@fb.com>
parents:
24026
diff
changeset
|
1135 while payloadsize: |
9881a1437799
bundle2.unbundlepart: raise payloadchunks from a closure to a method
Eric Sumner <ericsumner@fb.com>
parents:
24026
diff
changeset
|
1136 if payloadsize == flaginterrupt: |
9881a1437799
bundle2.unbundlepart: raise payloadchunks from a closure to a method
Eric Sumner <ericsumner@fb.com>
parents:
24026
diff
changeset
|
1137 # interruption detection, the handler will now read a |
9881a1437799
bundle2.unbundlepart: raise payloadchunks from a closure to a method
Eric Sumner <ericsumner@fb.com>
parents:
24026
diff
changeset
|
1138 # single part and process it. |
9881a1437799
bundle2.unbundlepart: raise payloadchunks from a closure to a method
Eric Sumner <ericsumner@fb.com>
parents:
24026
diff
changeset
|
1139 interrupthandler(self.ui, self._fp)() |
9881a1437799
bundle2.unbundlepart: raise payloadchunks from a closure to a method
Eric Sumner <ericsumner@fb.com>
parents:
24026
diff
changeset
|
1140 elif payloadsize < 0: |
9881a1437799
bundle2.unbundlepart: raise payloadchunks from a closure to a method
Eric Sumner <ericsumner@fb.com>
parents:
24026
diff
changeset
|
1141 msg = 'negative payload chunk size: %i' % payloadsize |
9881a1437799
bundle2.unbundlepart: raise payloadchunks from a closure to a method
Eric Sumner <ericsumner@fb.com>
parents:
24026
diff
changeset
|
1142 raise error.BundleValueError(msg) |
9881a1437799
bundle2.unbundlepart: raise payloadchunks from a closure to a method
Eric Sumner <ericsumner@fb.com>
parents:
24026
diff
changeset
|
1143 else: |
24035
7eb26415bef6
bundle2.unbundlepart: keep an index of chunks and their locations
Eric Sumner <ericsumner@fb.com>
parents:
24034
diff
changeset
|
1144 result = self._readexact(payloadsize) |
7eb26415bef6
bundle2.unbundlepart: keep an index of chunks and their locations
Eric Sumner <ericsumner@fb.com>
parents:
24034
diff
changeset
|
1145 chunknum += 1 |
7eb26415bef6
bundle2.unbundlepart: keep an index of chunks and their locations
Eric Sumner <ericsumner@fb.com>
parents:
24034
diff
changeset
|
1146 pos += payloadsize |
7eb26415bef6
bundle2.unbundlepart: keep an index of chunks and their locations
Eric Sumner <ericsumner@fb.com>
parents:
24034
diff
changeset
|
1147 if chunknum == len(self._chunkindex): |
7eb26415bef6
bundle2.unbundlepart: keep an index of chunks and their locations
Eric Sumner <ericsumner@fb.com>
parents:
24034
diff
changeset
|
1148 self._chunkindex.append((pos, |
7eb26415bef6
bundle2.unbundlepart: keep an index of chunks and their locations
Eric Sumner <ericsumner@fb.com>
parents:
24034
diff
changeset
|
1149 super(unbundlepart, self).tell())) |
7eb26415bef6
bundle2.unbundlepart: keep an index of chunks and their locations
Eric Sumner <ericsumner@fb.com>
parents:
24034
diff
changeset
|
1150 yield result |
24034
9881a1437799
bundle2.unbundlepart: raise payloadchunks from a closure to a method
Eric Sumner <ericsumner@fb.com>
parents:
24026
diff
changeset
|
1151 payloadsize = self._unpack(_fpayloadsize)[0] |
25320
697d8953b04d
bundle2: handle new line in 'indebug' function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25319
diff
changeset
|
1152 indebug(self.ui, 'payload chunk size: %i' % payloadsize) |
24034
9881a1437799
bundle2.unbundlepart: raise payloadchunks from a closure to a method
Eric Sumner <ericsumner@fb.com>
parents:
24026
diff
changeset
|
1153 |
24037
f0b498cfc5c8
bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents:
24036
diff
changeset
|
1154 def _findchunk(self, pos): |
f0b498cfc5c8
bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents:
24036
diff
changeset
|
1155 '''for a given payload position, return a chunk number and offset''' |
f0b498cfc5c8
bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents:
24036
diff
changeset
|
1156 for chunk, (ppos, fpos) in enumerate(self._chunkindex): |
f0b498cfc5c8
bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents:
24036
diff
changeset
|
1157 if ppos == pos: |
f0b498cfc5c8
bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents:
24036
diff
changeset
|
1158 return chunk, 0 |
f0b498cfc5c8
bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents:
24036
diff
changeset
|
1159 elif ppos > pos: |
f0b498cfc5c8
bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents:
24036
diff
changeset
|
1160 return chunk - 1, pos - self._chunkindex[chunk - 1][0] |
f0b498cfc5c8
bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents:
24036
diff
changeset
|
1161 raise ValueError('Unknown chunk') |
f0b498cfc5c8
bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents:
24036
diff
changeset
|
1162 |
21019
3dc09f831a2e
bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21016
diff
changeset
|
1163 def _readheader(self): |
21014
a6246bba7b9e
bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21013
diff
changeset
|
1164 """read the header and setup the object""" |
21016
b477afb1c81e
bundle2: move unpackheader closure into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21015
diff
changeset
|
1165 typesize = self._unpackheader(_fparttypesize)[0] |
21015
14dd49260246
bundle2: move the fromheader closure into the class itself
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21014
diff
changeset
|
1166 self.type = self._fromheader(typesize) |
25320
697d8953b04d
bundle2: handle new line in 'indebug' function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25319
diff
changeset
|
1167 indebug(self.ui, 'part type: "%s"' % self.type) |
21016
b477afb1c81e
bundle2: move unpackheader closure into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21015
diff
changeset
|
1168 self.id = self._unpackheader(_fpartid)[0] |
25320
697d8953b04d
bundle2: handle new line in 'indebug' function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25319
diff
changeset
|
1169 indebug(self.ui, 'part id: "%s"' % self.id) |
23585
94b25d71dd0f
bundle2.unbundlepart: decouple mandatory from parttype
Eric Sumner <ericsumner@fb.com>
parents:
23439
diff
changeset
|
1170 # extract mandatory bit from type |
94b25d71dd0f
bundle2.unbundlepart: decouple mandatory from parttype
Eric Sumner <ericsumner@fb.com>
parents:
23439
diff
changeset
|
1171 self.mandatory = (self.type != self.type.lower()) |
94b25d71dd0f
bundle2.unbundlepart: decouple mandatory from parttype
Eric Sumner <ericsumner@fb.com>
parents:
23439
diff
changeset
|
1172 self.type = self.type.lower() |
21014
a6246bba7b9e
bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21013
diff
changeset
|
1173 ## reading parameters |
a6246bba7b9e
bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21013
diff
changeset
|
1174 # param count |
21016
b477afb1c81e
bundle2: move unpackheader closure into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21015
diff
changeset
|
1175 mancount, advcount = self._unpackheader(_fpartparamcount) |
25320
697d8953b04d
bundle2: handle new line in 'indebug' function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25319
diff
changeset
|
1176 indebug(self.ui, 'part parameters: %i' % (mancount + advcount)) |
21014
a6246bba7b9e
bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21013
diff
changeset
|
1177 # param size |
21016
b477afb1c81e
bundle2: move unpackheader closure into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21015
diff
changeset
|
1178 fparamsizes = _makefpartparamsizes(mancount + advcount) |
b477afb1c81e
bundle2: move unpackheader closure into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21015
diff
changeset
|
1179 paramsizes = self._unpackheader(fparamsizes) |
21014
a6246bba7b9e
bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21013
diff
changeset
|
1180 # make it a list of couple again |
a6246bba7b9e
bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21013
diff
changeset
|
1181 paramsizes = zip(paramsizes[::2], paramsizes[1::2]) |
a6246bba7b9e
bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21013
diff
changeset
|
1182 # split mandatory from advisory |
a6246bba7b9e
bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21013
diff
changeset
|
1183 mansizes = paramsizes[:mancount] |
a6246bba7b9e
bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21013
diff
changeset
|
1184 advsizes = paramsizes[mancount:] |
23139
e53f6b72a0e4
spelling: fixes from proofreading of spell checker issues
Mads Kiilerich <madski@unity3d.com>
parents:
23067
diff
changeset
|
1185 # retrieve param value |
21014
a6246bba7b9e
bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21013
diff
changeset
|
1186 manparams = [] |
a6246bba7b9e
bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21013
diff
changeset
|
1187 for key, value in mansizes: |
21015
14dd49260246
bundle2: move the fromheader closure into the class itself
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21014
diff
changeset
|
1188 manparams.append((self._fromheader(key), self._fromheader(value))) |
21014
a6246bba7b9e
bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21013
diff
changeset
|
1189 advparams = [] |
a6246bba7b9e
bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21013
diff
changeset
|
1190 for key, value in advsizes: |
21015
14dd49260246
bundle2: move the fromheader closure into the class itself
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21014
diff
changeset
|
1191 advparams.append((self._fromheader(key), self._fromheader(value))) |
21608
3cb96ca90c17
bundle2: introduce an ``_initparams`` method
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21607
diff
changeset
|
1192 self._initparams(manparams, advparams) |
21014
a6246bba7b9e
bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21013
diff
changeset
|
1193 ## part payload |
24034
9881a1437799
bundle2.unbundlepart: raise payloadchunks from a closure to a method
Eric Sumner <ericsumner@fb.com>
parents:
24026
diff
changeset
|
1194 self._payloadstream = util.chunkbuffer(self._payloadchunks()) |
21019
3dc09f831a2e
bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21016
diff
changeset
|
1195 # we read the data, tell it |
3dc09f831a2e
bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21016
diff
changeset
|
1196 self._initialized = True |
3dc09f831a2e
bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21016
diff
changeset
|
1197 |
3dc09f831a2e
bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21016
diff
changeset
|
1198 def read(self, size=None): |
3dc09f831a2e
bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21016
diff
changeset
|
1199 """read payload data""" |
3dc09f831a2e
bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21016
diff
changeset
|
1200 if not self._initialized: |
3dc09f831a2e
bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21016
diff
changeset
|
1201 self._readheader() |
3dc09f831a2e
bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21016
diff
changeset
|
1202 if size is None: |
3dc09f831a2e
bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21016
diff
changeset
|
1203 data = self._payloadstream.read() |
3dc09f831a2e
bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21016
diff
changeset
|
1204 else: |
3dc09f831a2e
bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21016
diff
changeset
|
1205 data = self._payloadstream.read(size) |
25334
76cba1ecf55f
bundle2: add generic debug output regarding processed part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25333
diff
changeset
|
1206 self._pos += len(data) |
21019
3dc09f831a2e
bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21016
diff
changeset
|
1207 if size is None or len(data) < size: |
25334
76cba1ecf55f
bundle2: add generic debug output regarding processed part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25333
diff
changeset
|
1208 if not self.consumed and self._pos: |
76cba1ecf55f
bundle2: add generic debug output regarding processed part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25333
diff
changeset
|
1209 self.ui.debug('bundle2-input-part: total payload size %i\n' |
76cba1ecf55f
bundle2: add generic debug output regarding processed part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25333
diff
changeset
|
1210 % self._pos) |
21019
3dc09f831a2e
bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21016
diff
changeset
|
1211 self.consumed = True |
3dc09f831a2e
bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21016
diff
changeset
|
1212 return data |
3dc09f831a2e
bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21016
diff
changeset
|
1213 |
24036
c7601086338a
bundle2.unbundlepart: tell() implementation
Eric Sumner <ericsumner@fb.com>
parents:
24035
diff
changeset
|
1214 def tell(self): |
c7601086338a
bundle2.unbundlepart: tell() implementation
Eric Sumner <ericsumner@fb.com>
parents:
24035
diff
changeset
|
1215 return self._pos |
c7601086338a
bundle2.unbundlepart: tell() implementation
Eric Sumner <ericsumner@fb.com>
parents:
24035
diff
changeset
|
1216 |
24037
f0b498cfc5c8
bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents:
24036
diff
changeset
|
1217 def seek(self, offset, whence=0): |
f0b498cfc5c8
bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents:
24036
diff
changeset
|
1218 if whence == 0: |
f0b498cfc5c8
bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents:
24036
diff
changeset
|
1219 newpos = offset |
f0b498cfc5c8
bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents:
24036
diff
changeset
|
1220 elif whence == 1: |
f0b498cfc5c8
bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents:
24036
diff
changeset
|
1221 newpos = self._pos + offset |
f0b498cfc5c8
bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents:
24036
diff
changeset
|
1222 elif whence == 2: |
f0b498cfc5c8
bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents:
24036
diff
changeset
|
1223 if not self.consumed: |
f0b498cfc5c8
bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents:
24036
diff
changeset
|
1224 self.read() |
f0b498cfc5c8
bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents:
24036
diff
changeset
|
1225 newpos = self._chunkindex[-1][0] - offset |
f0b498cfc5c8
bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents:
24036
diff
changeset
|
1226 else: |
f0b498cfc5c8
bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents:
24036
diff
changeset
|
1227 raise ValueError('Unknown whence value: %r' % (whence,)) |
f0b498cfc5c8
bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents:
24036
diff
changeset
|
1228 |
f0b498cfc5c8
bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents:
24036
diff
changeset
|
1229 if newpos > self._chunkindex[-1][0] and not self.consumed: |
f0b498cfc5c8
bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents:
24036
diff
changeset
|
1230 self.read() |
f0b498cfc5c8
bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents:
24036
diff
changeset
|
1231 if not 0 <= newpos <= self._chunkindex[-1][0]: |
f0b498cfc5c8
bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents:
24036
diff
changeset
|
1232 raise ValueError('Offset out of range') |
f0b498cfc5c8
bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents:
24036
diff
changeset
|
1233 |
f0b498cfc5c8
bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents:
24036
diff
changeset
|
1234 if self._pos != newpos: |
f0b498cfc5c8
bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents:
24036
diff
changeset
|
1235 chunk, internaloffset = self._findchunk(newpos) |
f0b498cfc5c8
bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents:
24036
diff
changeset
|
1236 self._payloadstream = util.chunkbuffer(self._payloadchunks(chunk)) |
f0b498cfc5c8
bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents:
24036
diff
changeset
|
1237 adjust = self.read(internaloffset) |
f0b498cfc5c8
bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents:
24036
diff
changeset
|
1238 if len(adjust) != internaloffset: |
26587
56b2bcea2529
error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26565
diff
changeset
|
1239 raise error.Abort(_('Seek failed\n')) |
24037
f0b498cfc5c8
bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents:
24036
diff
changeset
|
1240 self._pos = newpos |
f0b498cfc5c8
bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents:
24036
diff
changeset
|
1241 |
25317
5a5b7046d00e
bundle2: add an informative comment to the capability dict
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25315
diff
changeset
|
1242 # These are only the static capabilities. |
5a5b7046d00e
bundle2: add an informative comment to the capability dict
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25315
diff
changeset
|
1243 # Check the 'getrepocaps' function for the rest. |
24686
e0e28e910fa3
bundle2: rename format, parts and config to final names
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24648
diff
changeset
|
1244 capabilities = {'HG20': (), |
25493
d8e7b0781ad7
bundle2: convey PushkeyFailed error over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25492
diff
changeset
|
1245 'error': ('abort', 'unsupportedcontent', 'pushraced', |
d8e7b0781ad7
bundle2: convey PushkeyFailed error over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25492
diff
changeset
|
1246 'pushkey'), |
24686
e0e28e910fa3
bundle2: rename format, parts and config to final names
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24648
diff
changeset
|
1247 'listkeys': (), |
e0e28e910fa3
bundle2: rename format, parts and config to final names
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24648
diff
changeset
|
1248 'pushkey': (), |
23029
149fc8a44184
bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents:
23011
diff
changeset
|
1249 'digests': tuple(sorted(util.DIGESTS.keys())), |
24686
e0e28e910fa3
bundle2: rename format, parts and config to final names
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24648
diff
changeset
|
1250 'remote-changegroup': ('http', 'https'), |
25401
d29201352af7
bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25336
diff
changeset
|
1251 'hgtagsfnodes': (), |
22341
2d16b39601b5
obsmarker: move bundle2caps from the localrepo class to the bundle2 module
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22340
diff
changeset
|
1252 } |
2d16b39601b5
obsmarker: move bundle2caps from the localrepo class to the bundle2 module
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22340
diff
changeset
|
1253 |
23439
743736fc7c41
bundle2-push: provide transaction to reply unbundler
Eric Sumner <ericsumner@fb.com>
parents:
23438
diff
changeset
|
1254 def getrepocaps(repo, allowpushback=False): |
22342
262c5cc126c1
bundle2: introduce a `getrepocaps` to retrieve the bundle2 caps of a repo
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22341
diff
changeset
|
1255 """return the bundle2 capabilities for a given repo |
262c5cc126c1
bundle2: introduce a `getrepocaps` to retrieve the bundle2 caps of a repo
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22341
diff
changeset
|
1256 |
22343
76677a2c1cfd
bundle2: advertise the obsmarker part in bundle2 capabilities
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22342
diff
changeset
|
1257 Exists to allow extensions (like evolution) to mutate the capabilities. |
22342
262c5cc126c1
bundle2: introduce a `getrepocaps` to retrieve the bundle2 caps of a repo
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22341
diff
changeset
|
1258 """ |
22343
76677a2c1cfd
bundle2: advertise the obsmarker part in bundle2 capabilities
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22342
diff
changeset
|
1259 caps = capabilities.copy() |
27953
88609cfa3745
changegroup: fix pulling to treemanifest repo from flat repo (issue5066)
Martin von Zweigbergk <martinvonz@google.com>
parents:
27879
diff
changeset
|
1260 caps['changegroup'] = tuple(sorted( |
88609cfa3745
changegroup: fix pulling to treemanifest repo from flat repo (issue5066)
Martin von Zweigbergk <martinvonz@google.com>
parents:
27879
diff
changeset
|
1261 changegroup.supportedincomingversions(repo))) |
22953
b1d694d3975e
obsolete: add exchange option
Durham Goode <durham@fb.com>
parents:
22661
diff
changeset
|
1262 if obsolete.isenabled(repo, obsolete.exchangeopt): |
22343
76677a2c1cfd
bundle2: advertise the obsmarker part in bundle2 capabilities
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22342
diff
changeset
|
1263 supportedformat = tuple('V%i' % v for v in obsolete.formats) |
24686
e0e28e910fa3
bundle2: rename format, parts and config to final names
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24648
diff
changeset
|
1264 caps['obsmarkers'] = supportedformat |
23439
743736fc7c41
bundle2-push: provide transaction to reply unbundler
Eric Sumner <ericsumner@fb.com>
parents:
23438
diff
changeset
|
1265 if allowpushback: |
24686
e0e28e910fa3
bundle2: rename format, parts and config to final names
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24648
diff
changeset
|
1266 caps['pushback'] = () |
22343
76677a2c1cfd
bundle2: advertise the obsmarker part in bundle2 capabilities
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22342
diff
changeset
|
1267 return caps |
22342
262c5cc126c1
bundle2: introduce a `getrepocaps` to retrieve the bundle2 caps of a repo
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22341
diff
changeset
|
1268 |
21644
17755dd8c509
bundle2: introduce a bundle2caps function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21628
diff
changeset
|
1269 def bundle2caps(remote): |
23139
e53f6b72a0e4
spelling: fixes from proofreading of spell checker issues
Mads Kiilerich <madski@unity3d.com>
parents:
23067
diff
changeset
|
1270 """return the bundle capabilities of a peer as dict""" |
24686
e0e28e910fa3
bundle2: rename format, parts and config to final names
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24648
diff
changeset
|
1271 raw = remote.capable('bundle2') |
21644
17755dd8c509
bundle2: introduce a bundle2caps function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21628
diff
changeset
|
1272 if not raw and raw != '': |
17755dd8c509
bundle2: introduce a bundle2caps function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21628
diff
changeset
|
1273 return {} |
28883
032c4c2f802a
pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents:
28672
diff
changeset
|
1274 capsblob = urlreq.unquote(remote.capable('bundle2')) |
21644
17755dd8c509
bundle2: introduce a bundle2caps function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21628
diff
changeset
|
1275 return decodecaps(capsblob) |
21014
a6246bba7b9e
bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21013
diff
changeset
|
1276 |
22344
9829b7948100
bundle2: add a `obsmarkersversion` function to extract supported version
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22343
diff
changeset
|
1277 def obsmarkersversion(caps): |
9829b7948100
bundle2: add a `obsmarkersversion` function to extract supported version
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22343
diff
changeset
|
1278 """extract the list of supported obsmarkers versions from a bundle2caps dict |
9829b7948100
bundle2: add a `obsmarkersversion` function to extract supported version
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22343
diff
changeset
|
1279 """ |
24686
e0e28e910fa3
bundle2: rename format, parts and config to final names
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24648
diff
changeset
|
1280 obscaps = caps.get('obsmarkers', ()) |
22344
9829b7948100
bundle2: add a `obsmarkersversion` function to extract supported version
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22343
diff
changeset
|
1281 return [int(c[1:]) for c in obscaps if c.startswith('V')] |
9829b7948100
bundle2: add a `obsmarkersversion` function to extract supported version
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22343
diff
changeset
|
1282 |
28666
ae53ecc47414
bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
1283 def writebundle(ui, cg, filename, bundletype, vfs=None, compression=None): |
ae53ecc47414
bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
1284 """Write a bundle file and return its filename. |
ae53ecc47414
bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
1285 |
ae53ecc47414
bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
1286 Existing files will not be overwritten. |
ae53ecc47414
bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
1287 If no filename is specified, a temporary file is created. |
ae53ecc47414
bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
1288 bz2 compression can be turned off. |
ae53ecc47414
bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
1289 The bundle file will be deleted in case of errors. |
ae53ecc47414
bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
1290 """ |
ae53ecc47414
bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
1291 |
ae53ecc47414
bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
1292 if bundletype == "HG20": |
ae53ecc47414
bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
1293 bundle = bundle20(ui) |
ae53ecc47414
bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
1294 bundle.setcompression(compression) |
ae53ecc47414
bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
1295 part = bundle.newpart('changegroup', data=cg.getchunks()) |
ae53ecc47414
bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
1296 part.addparam('version', cg.version) |
ae53ecc47414
bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
1297 chunkiter = bundle.getchunks() |
ae53ecc47414
bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
1298 else: |
ae53ecc47414
bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
1299 # compression argument is only for the bundle2 case |
ae53ecc47414
bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
1300 assert compression is None |
ae53ecc47414
bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
1301 if cg.version != '01': |
ae53ecc47414
bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
1302 raise error.Abort(_('old bundle types only supports v1 ' |
ae53ecc47414
bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
1303 'changegroups')) |
ae53ecc47414
bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
1304 header, comp = bundletypes[bundletype] |
ae53ecc47414
bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
1305 if comp not in util.compressors: |
ae53ecc47414
bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
1306 raise error.Abort(_('unknown stream compression type: %s') |
ae53ecc47414
bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
1307 % comp) |
ae53ecc47414
bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
1308 z = util.compressors[comp]() |
ae53ecc47414
bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
1309 subchunkiter = cg.getchunks() |
ae53ecc47414
bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
1310 def chunkiter(): |
ae53ecc47414
bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
1311 yield header |
ae53ecc47414
bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
1312 for chunk in subchunkiter: |
ae53ecc47414
bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
1313 yield z.compress(chunk) |
ae53ecc47414
bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
1314 yield z.flush() |
ae53ecc47414
bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
1315 chunkiter = chunkiter() |
ae53ecc47414
bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
1316 |
ae53ecc47414
bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
1317 # parse the changegroup data, otherwise we will block |
ae53ecc47414
bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
1318 # in case of sshrepo because we don't know the end of the stream |
ae53ecc47414
bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
1319 return changegroup.writechunks(ui, chunkiter, filename, vfs=vfs) |
ae53ecc47414
bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
1320 |
27734
5c0fd878779c
treemanifests: set bundle2 part parameter indicating treemanifest
Martin von Zweigbergk <martinvonz@google.com>
parents:
26829
diff
changeset
|
1321 @parthandler('changegroup', ('version', 'nbchanges', 'treemanifest')) |
20998
93a3c5b58635
bundle2: use reply part to return result of addchangegroup
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20997
diff
changeset
|
1322 def handlechangegroup(op, inpart): |
20950
c7ceae0faf69
bundle2: first crude version of bundling changeset with bundle2
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20949
diff
changeset
|
1323 """apply a changegroup part on the repo |
c7ceae0faf69
bundle2: first crude version of bundling changeset with bundle2
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20949
diff
changeset
|
1324 |
c7ceae0faf69
bundle2: first crude version of bundling changeset with bundle2
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20949
diff
changeset
|
1325 This is a very early implementation that will massive rework before being |
c7ceae0faf69
bundle2: first crude version of bundling changeset with bundle2
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20949
diff
changeset
|
1326 inflicted to any end-user. |
c7ceae0faf69
bundle2: first crude version of bundling changeset with bundle2
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20949
diff
changeset
|
1327 """ |
20952
b24ee5076b94
bundle2: make it possible have a global transaction for the unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20950
diff
changeset
|
1328 # Make sure we trigger a transaction creation |
b24ee5076b94
bundle2: make it possible have a global transaction for the unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20950
diff
changeset
|
1329 # |
b24ee5076b94
bundle2: make it possible have a global transaction for the unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20950
diff
changeset
|
1330 # The addchangegroup function will get a transaction object by itself, but |
b24ee5076b94
bundle2: make it possible have a global transaction for the unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20950
diff
changeset
|
1331 # we need to make sure we trigger the creation of a transaction object used |
b24ee5076b94
bundle2: make it possible have a global transaction for the unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20950
diff
changeset
|
1332 # for the whole processing scope. |
b24ee5076b94
bundle2: make it possible have a global transaction for the unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20950
diff
changeset
|
1333 op.gettransaction() |
23170
02e8f9b60052
bundle2: support a "version" argument in `changegroup` part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23169
diff
changeset
|
1334 unpackerversion = inpart.params.get('version', '01') |
02e8f9b60052
bundle2: support a "version" argument in `changegroup` part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23169
diff
changeset
|
1335 # We should raise an appropriate exception here |
27751
a40e2f7fe49d
changegroup: hide packermap behind methods
Martin von Zweigbergk <martinvonz@google.com>
parents:
27734
diff
changeset
|
1336 cg = changegroup.getunbundler(unpackerversion, inpart, None) |
23001
4df9b5e62f70
bundle2: add a comment about addchangegroup source and url
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22953
diff
changeset
|
1337 # the source and url passed here are overwritten by the one contained in |
4df9b5e62f70
bundle2: add a comment about addchangegroup source and url
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22953
diff
changeset
|
1338 # the transaction.hookargs argument. So 'bundle2' is a placeholder |
25518
ca656f3dffd7
bundle2: provide number of changesets information to 'addchangegroup'
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25507
diff
changeset
|
1339 nbchangesets = None |
ca656f3dffd7
bundle2: provide number of changesets information to 'addchangegroup'
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25507
diff
changeset
|
1340 if 'nbchanges' in inpart.params: |
ca656f3dffd7
bundle2: provide number of changesets information to 'addchangegroup'
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25507
diff
changeset
|
1341 nbchangesets = int(inpart.params.get('nbchanges')) |
27734
5c0fd878779c
treemanifests: set bundle2 part parameter indicating treemanifest
Martin von Zweigbergk <martinvonz@google.com>
parents:
26829
diff
changeset
|
1342 if ('treemanifest' in inpart.params and |
5c0fd878779c
treemanifests: set bundle2 part parameter indicating treemanifest
Martin von Zweigbergk <martinvonz@google.com>
parents:
26829
diff
changeset
|
1343 'treemanifest' not in op.repo.requirements): |
5c0fd878779c
treemanifests: set bundle2 part parameter indicating treemanifest
Martin von Zweigbergk <martinvonz@google.com>
parents:
26829
diff
changeset
|
1344 if len(op.repo.changelog) != 0: |
5c0fd878779c
treemanifests: set bundle2 part parameter indicating treemanifest
Martin von Zweigbergk <martinvonz@google.com>
parents:
26829
diff
changeset
|
1345 raise error.Abort(_( |
5c0fd878779c
treemanifests: set bundle2 part parameter indicating treemanifest
Martin von Zweigbergk <martinvonz@google.com>
parents:
26829
diff
changeset
|
1346 "bundle contains tree manifests, but local repo is " |
5c0fd878779c
treemanifests: set bundle2 part parameter indicating treemanifest
Martin von Zweigbergk <martinvonz@google.com>
parents:
26829
diff
changeset
|
1347 "non-empty and does not use tree manifests")) |
5c0fd878779c
treemanifests: set bundle2 part parameter indicating treemanifest
Martin von Zweigbergk <martinvonz@google.com>
parents:
26829
diff
changeset
|
1348 op.repo.requirements.add('treemanifest') |
5c0fd878779c
treemanifests: set bundle2 part parameter indicating treemanifest
Martin von Zweigbergk <martinvonz@google.com>
parents:
26829
diff
changeset
|
1349 op.repo._applyopenerreqs() |
5c0fd878779c
treemanifests: set bundle2 part parameter indicating treemanifest
Martin von Zweigbergk <martinvonz@google.com>
parents:
26829
diff
changeset
|
1350 op.repo._writerequirements() |
26698
c94cdeeb586a
bundle2: use cg?unpacker.apply() instead of changegroup.addchangegroup()
Augie Fackler <augie@google.com>
parents:
26685
diff
changeset
|
1351 ret = cg.apply(op.repo, 'bundle2', 'bundle2', expectedtotal=nbchangesets) |
20950
c7ceae0faf69
bundle2: first crude version of bundling changeset with bundle2
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20949
diff
changeset
|
1352 op.records.add('changegroup', {'return': ret}) |
20998
93a3c5b58635
bundle2: use reply part to return result of addchangegroup
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20997
diff
changeset
|
1353 if op.reply is not None: |
23139
e53f6b72a0e4
spelling: fixes from proofreading of spell checker issues
Mads Kiilerich <madski@unity3d.com>
parents:
23067
diff
changeset
|
1354 # This is definitely not the final form of this |
20998
93a3c5b58635
bundle2: use reply part to return result of addchangegroup
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20997
diff
changeset
|
1355 # return. But one need to start somewhere. |
24686
e0e28e910fa3
bundle2: rename format, parts and config to final names
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24648
diff
changeset
|
1356 part = op.reply.newpart('reply:changegroup', mandatory=False) |
21606
e55888447958
bundle2: update part creators to ``addparam`` when relevant
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21605
diff
changeset
|
1357 part.addparam('in-reply-to', str(inpart.id), mandatory=False) |
e55888447958
bundle2: update part creators to ``addparam`` when relevant
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21605
diff
changeset
|
1358 part.addparam('return', '%i' % ret, mandatory=False) |
21019
3dc09f831a2e
bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21016
diff
changeset
|
1359 assert not inpart.read() |
20950
c7ceae0faf69
bundle2: first crude version of bundling changeset with bundle2
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20949
diff
changeset
|
1360 |
23029
149fc8a44184
bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents:
23011
diff
changeset
|
1361 _remotechangegroupparams = tuple(['url', 'size', 'digests'] + |
149fc8a44184
bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents:
23011
diff
changeset
|
1362 ['digest:%s' % k for k in util.DIGESTS.keys()]) |
24686
e0e28e910fa3
bundle2: rename format, parts and config to final names
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24648
diff
changeset
|
1363 @parthandler('remote-changegroup', _remotechangegroupparams) |
23029
149fc8a44184
bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents:
23011
diff
changeset
|
1364 def handleremotechangegroup(op, inpart): |
149fc8a44184
bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents:
23011
diff
changeset
|
1365 """apply a bundle10 on the repo, given an url and validation information |
149fc8a44184
bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents:
23011
diff
changeset
|
1366 |
149fc8a44184
bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents:
23011
diff
changeset
|
1367 All the information about the remote bundle to import are given as |
149fc8a44184
bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents:
23011
diff
changeset
|
1368 parameters. The parameters include: |
149fc8a44184
bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents:
23011
diff
changeset
|
1369 - url: the url to the bundle10. |
149fc8a44184
bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents:
23011
diff
changeset
|
1370 - size: the bundle10 file size. It is used to validate what was |
149fc8a44184
bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents:
23011
diff
changeset
|
1371 retrieved by the client matches the server knowledge about the bundle. |
149fc8a44184
bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents:
23011
diff
changeset
|
1372 - digests: a space separated list of the digest types provided as |
149fc8a44184
bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents:
23011
diff
changeset
|
1373 parameters. |
149fc8a44184
bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents:
23011
diff
changeset
|
1374 - digest:<digest-type>: the hexadecimal representation of the digest with |
149fc8a44184
bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents:
23011
diff
changeset
|
1375 that name. Like the size, it is used to validate what was retrieved by |
149fc8a44184
bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents:
23011
diff
changeset
|
1376 the client matches what the server knows about the bundle. |
149fc8a44184
bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents:
23011
diff
changeset
|
1377 |
149fc8a44184
bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents:
23011
diff
changeset
|
1378 When multiple digest types are given, all of them are checked. |
149fc8a44184
bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents:
23011
diff
changeset
|
1379 """ |
149fc8a44184
bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents:
23011
diff
changeset
|
1380 try: |
149fc8a44184
bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents:
23011
diff
changeset
|
1381 raw_url = inpart.params['url'] |
149fc8a44184
bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents:
23011
diff
changeset
|
1382 except KeyError: |
26587
56b2bcea2529
error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26565
diff
changeset
|
1383 raise error.Abort(_('remote-changegroup: missing "%s" param') % 'url') |
23029
149fc8a44184
bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents:
23011
diff
changeset
|
1384 parsed_url = util.url(raw_url) |
24686
e0e28e910fa3
bundle2: rename format, parts and config to final names
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24648
diff
changeset
|
1385 if parsed_url.scheme not in capabilities['remote-changegroup']: |
26587
56b2bcea2529
error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26565
diff
changeset
|
1386 raise error.Abort(_('remote-changegroup does not support %s urls') % |
23029
149fc8a44184
bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents:
23011
diff
changeset
|
1387 parsed_url.scheme) |
149fc8a44184
bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents:
23011
diff
changeset
|
1388 |
149fc8a44184
bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents:
23011
diff
changeset
|
1389 try: |
149fc8a44184
bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents:
23011
diff
changeset
|
1390 size = int(inpart.params['size']) |
149fc8a44184
bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents:
23011
diff
changeset
|
1391 except ValueError: |
26587
56b2bcea2529
error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26565
diff
changeset
|
1392 raise error.Abort(_('remote-changegroup: invalid value for param "%s"') |
23029
149fc8a44184
bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents:
23011
diff
changeset
|
1393 % 'size') |
149fc8a44184
bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents:
23011
diff
changeset
|
1394 except KeyError: |
26587
56b2bcea2529
error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26565
diff
changeset
|
1395 raise error.Abort(_('remote-changegroup: missing "%s" param') % 'size') |
23029
149fc8a44184
bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents:
23011
diff
changeset
|
1396 |
149fc8a44184
bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents:
23011
diff
changeset
|
1397 digests = {} |
149fc8a44184
bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents:
23011
diff
changeset
|
1398 for typ in inpart.params.get('digests', '').split(): |
149fc8a44184
bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents:
23011
diff
changeset
|
1399 param = 'digest:%s' % typ |
149fc8a44184
bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents:
23011
diff
changeset
|
1400 try: |
149fc8a44184
bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents:
23011
diff
changeset
|
1401 value = inpart.params[param] |
149fc8a44184
bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents:
23011
diff
changeset
|
1402 except KeyError: |
26587
56b2bcea2529
error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26565
diff
changeset
|
1403 raise error.Abort(_('remote-changegroup: missing "%s" param') % |
23029
149fc8a44184
bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents:
23011
diff
changeset
|
1404 param) |
149fc8a44184
bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents:
23011
diff
changeset
|
1405 digests[typ] = value |
149fc8a44184
bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents:
23011
diff
changeset
|
1406 |
149fc8a44184
bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents:
23011
diff
changeset
|
1407 real_part = util.digestchecker(url.open(op.ui, raw_url), size, digests) |
149fc8a44184
bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents:
23011
diff
changeset
|
1408 |
149fc8a44184
bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents:
23011
diff
changeset
|
1409 # Make sure we trigger a transaction creation |
149fc8a44184
bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents:
23011
diff
changeset
|
1410 # |
149fc8a44184
bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents:
23011
diff
changeset
|
1411 # The addchangegroup function will get a transaction object by itself, but |
149fc8a44184
bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents:
23011
diff
changeset
|
1412 # we need to make sure we trigger the creation of a transaction object used |
149fc8a44184
bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents:
23011
diff
changeset
|
1413 # for the whole processing scope. |
149fc8a44184
bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents:
23011
diff
changeset
|
1414 op.gettransaction() |
25919
8221fefaea08
bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25660
diff
changeset
|
1415 from . import exchange |
23029
149fc8a44184
bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents:
23011
diff
changeset
|
1416 cg = exchange.readbundle(op.repo.ui, real_part, raw_url) |
149fc8a44184
bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents:
23011
diff
changeset
|
1417 if not isinstance(cg, changegroup.cg1unpacker): |
26587
56b2bcea2529
error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26565
diff
changeset
|
1418 raise error.Abort(_('%s: not a bundle version 1.0') % |
23029
149fc8a44184
bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents:
23011
diff
changeset
|
1419 util.hidepassword(raw_url)) |
26698
c94cdeeb586a
bundle2: use cg?unpacker.apply() instead of changegroup.addchangegroup()
Augie Fackler <augie@google.com>
parents:
26685
diff
changeset
|
1420 ret = cg.apply(op.repo, 'bundle2', 'bundle2') |
23029
149fc8a44184
bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents:
23011
diff
changeset
|
1421 op.records.add('changegroup', {'return': ret}) |
149fc8a44184
bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents:
23011
diff
changeset
|
1422 if op.reply is not None: |
23139
e53f6b72a0e4
spelling: fixes from proofreading of spell checker issues
Mads Kiilerich <madski@unity3d.com>
parents:
23067
diff
changeset
|
1423 # This is definitely not the final form of this |
23029
149fc8a44184
bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents:
23011
diff
changeset
|
1424 # return. But one need to start somewhere. |
24686
e0e28e910fa3
bundle2: rename format, parts and config to final names
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24648
diff
changeset
|
1425 part = op.reply.newpart('reply:changegroup') |
23029
149fc8a44184
bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents:
23011
diff
changeset
|
1426 part.addparam('in-reply-to', str(inpart.id), mandatory=False) |
149fc8a44184
bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents:
23011
diff
changeset
|
1427 part.addparam('return', '%i' % ret, mandatory=False) |
149fc8a44184
bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents:
23011
diff
changeset
|
1428 try: |
149fc8a44184
bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents:
23011
diff
changeset
|
1429 real_part.validate() |
26587
56b2bcea2529
error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26565
diff
changeset
|
1430 except error.Abort as e: |
56b2bcea2529
error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26565
diff
changeset
|
1431 raise error.Abort(_('bundle at %s is corrupted:\n%s') % |
23029
149fc8a44184
bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents:
23011
diff
changeset
|
1432 (util.hidepassword(raw_url), str(e))) |
149fc8a44184
bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents:
23011
diff
changeset
|
1433 assert not inpart.read() |
149fc8a44184
bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents:
23011
diff
changeset
|
1434 |
24686
e0e28e910fa3
bundle2: rename format, parts and config to final names
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24648
diff
changeset
|
1435 @parthandler('reply:changegroup', ('return', 'in-reply-to')) |
22548
8a1ae133770a
bundle2: rename functions that have the same name
Mike Hommey <mh@glandium.org>
parents:
22390
diff
changeset
|
1436 def handlereplychangegroup(op, inpart): |
21611
71b7b3f79a3c
bundle2: use the new ``part.params`` dictionary
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21610
diff
changeset
|
1437 ret = int(inpart.params['return']) |
71b7b3f79a3c
bundle2: use the new ``part.params`` dictionary
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21610
diff
changeset
|
1438 replyto = int(inpart.params['in-reply-to']) |
71b7b3f79a3c
bundle2: use the new ``part.params`` dictionary
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21610
diff
changeset
|
1439 op.records.add('changegroup', {'return': ret}, replyto) |
20950
c7ceae0faf69
bundle2: first crude version of bundling changeset with bundle2
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20949
diff
changeset
|
1440 |
24686
e0e28e910fa3
bundle2: rename format, parts and config to final names
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24648
diff
changeset
|
1441 @parthandler('check:heads') |
22548
8a1ae133770a
bundle2: rename functions that have the same name
Mike Hommey <mh@glandium.org>
parents:
22390
diff
changeset
|
1442 def handlecheckheads(op, inpart): |
21060
0bea9db7543b
bundle2: add a "check:heads" handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21024
diff
changeset
|
1443 """check that head of the repo did not change |
0bea9db7543b
bundle2: add a "check:heads" handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21024
diff
changeset
|
1444 |
0bea9db7543b
bundle2: add a "check:heads" handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21024
diff
changeset
|
1445 This is used to detect a push race when using unbundle. |
0bea9db7543b
bundle2: add a "check:heads" handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21024
diff
changeset
|
1446 This replaces the "heads" argument of unbundle.""" |
0bea9db7543b
bundle2: add a "check:heads" handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21024
diff
changeset
|
1447 h = inpart.read(20) |
0bea9db7543b
bundle2: add a "check:heads" handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21024
diff
changeset
|
1448 heads = [] |
0bea9db7543b
bundle2: add a "check:heads" handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21024
diff
changeset
|
1449 while len(h) == 20: |
0bea9db7543b
bundle2: add a "check:heads" handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21024
diff
changeset
|
1450 heads.append(h) |
0bea9db7543b
bundle2: add a "check:heads" handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21024
diff
changeset
|
1451 h = inpart.read(20) |
0bea9db7543b
bundle2: add a "check:heads" handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21024
diff
changeset
|
1452 assert not h |
26565
ee1bcb9aa6e4
bundle2: add op.gettransaction() to handlers that need the lock
Durham Goode <durham@fb.com>
parents:
26542
diff
changeset
|
1453 # Trigger a transaction so that we are guaranteed to have the lock now. |
ee1bcb9aa6e4
bundle2: add op.gettransaction() to handlers that need the lock
Durham Goode <durham@fb.com>
parents:
26542
diff
changeset
|
1454 if op.ui.configbool('experimental', 'bundle2lazylocking'): |
ee1bcb9aa6e4
bundle2: add op.gettransaction() to handlers that need the lock
Durham Goode <durham@fb.com>
parents:
26542
diff
changeset
|
1455 op.gettransaction() |
21060
0bea9db7543b
bundle2: add a "check:heads" handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21024
diff
changeset
|
1456 if heads != op.repo.heads(): |
21185
5b3717e1a3ea
bundle2: add an error message to push race error
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21184
diff
changeset
|
1457 raise error.PushRaced('repository changed while pushing - ' |
5b3717e1a3ea
bundle2: add an error message to push race error
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21184
diff
changeset
|
1458 'please try again') |
21130
1ff06386217f
bundle2: introduce `replycaps` part for on-demand reply
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21129
diff
changeset
|
1459 |
24686
e0e28e910fa3
bundle2: rename format, parts and config to final names
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24648
diff
changeset
|
1460 @parthandler('output') |
21131
b7435117d951
bundle2: capture remote stdout while unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21130
diff
changeset
|
1461 def handleoutput(op, inpart): |
b7435117d951
bundle2: capture remote stdout while unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21130
diff
changeset
|
1462 """forward output captured on the server to the client""" |
b7435117d951
bundle2: capture remote stdout while unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21130
diff
changeset
|
1463 for line in inpart.read().splitlines(): |
24846
e79dd1c9753e
bundle2: issue remote output as "status" (issue4612)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24795
diff
changeset
|
1464 op.ui.status(('remote: %s\n' % line)) |
21131
b7435117d951
bundle2: capture remote stdout while unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21130
diff
changeset
|
1465 |
24686
e0e28e910fa3
bundle2: rename format, parts and config to final names
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24648
diff
changeset
|
1466 @parthandler('replycaps') |
21130
1ff06386217f
bundle2: introduce `replycaps` part for on-demand reply
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21129
diff
changeset
|
1467 def handlereplycaps(op, inpart): |
1ff06386217f
bundle2: introduce `replycaps` part for on-demand reply
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21129
diff
changeset
|
1468 """Notify that a reply bundle should be created |
1ff06386217f
bundle2: introduce `replycaps` part for on-demand reply
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21129
diff
changeset
|
1469 |
21138
f469879d27ec
bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21137
diff
changeset
|
1470 The payload contains the capabilities information for the reply""" |
f469879d27ec
bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21137
diff
changeset
|
1471 caps = decodecaps(inpart.read()) |
21130
1ff06386217f
bundle2: introduce `replycaps` part for on-demand reply
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21129
diff
changeset
|
1472 if op.reply is None: |
21135
98fbf3adfd83
bundle2: add capabilities support in `replycaps` part
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21134
diff
changeset
|
1473 op.reply = bundle20(op.ui, caps) |
21131
b7435117d951
bundle2: capture remote stdout while unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21130
diff
changeset
|
1474 |
26829
58f1645f72c3
bundle2: attribute remote failures to remote (issue4788)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26802
diff
changeset
|
1475 class AbortFromPart(error.Abort): |
58f1645f72c3
bundle2: attribute remote failures to remote (issue4788)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26802
diff
changeset
|
1476 """Sub-class of Abort that denotes an error from a bundle2 part.""" |
58f1645f72c3
bundle2: attribute remote failures to remote (issue4788)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26802
diff
changeset
|
1477 |
24686
e0e28e910fa3
bundle2: rename format, parts and config to final names
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24648
diff
changeset
|
1478 @parthandler('error:abort', ('message', 'hint')) |
24741
bb67e52362d6
bundle2: fix names for error part handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24733
diff
changeset
|
1479 def handleerrorabort(op, inpart): |
21177
952af771bc17
bundle2: gracefully handle abort during unbundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21176
diff
changeset
|
1480 """Used to transmit abort error over the wire""" |
26829
58f1645f72c3
bundle2: attribute remote failures to remote (issue4788)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26802
diff
changeset
|
1481 raise AbortFromPart(inpart.params['message'], |
58f1645f72c3
bundle2: attribute remote failures to remote (issue4788)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26802
diff
changeset
|
1482 hint=inpart.params.get('hint')) |
21183
4345274adc4b
bundle2: gracefully handle UnknownPartError during unbundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21179
diff
changeset
|
1483 |
25493
d8e7b0781ad7
bundle2: convey PushkeyFailed error over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25492
diff
changeset
|
1484 @parthandler('error:pushkey', ('namespace', 'key', 'new', 'old', 'ret', |
d8e7b0781ad7
bundle2: convey PushkeyFailed error over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25492
diff
changeset
|
1485 'in-reply-to')) |
d8e7b0781ad7
bundle2: convey PushkeyFailed error over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25492
diff
changeset
|
1486 def handleerrorpushkey(op, inpart): |
d8e7b0781ad7
bundle2: convey PushkeyFailed error over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25492
diff
changeset
|
1487 """Used to transmit failure of a mandatory pushkey over the wire""" |
d8e7b0781ad7
bundle2: convey PushkeyFailed error over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25492
diff
changeset
|
1488 kwargs = {} |
d8e7b0781ad7
bundle2: convey PushkeyFailed error over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25492
diff
changeset
|
1489 for name in ('namespace', 'key', 'new', 'old', 'ret'): |
d8e7b0781ad7
bundle2: convey PushkeyFailed error over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25492
diff
changeset
|
1490 value = inpart.params.get(name) |
d8e7b0781ad7
bundle2: convey PushkeyFailed error over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25492
diff
changeset
|
1491 if value is not None: |
d8e7b0781ad7
bundle2: convey PushkeyFailed error over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25492
diff
changeset
|
1492 kwargs[name] = value |
d8e7b0781ad7
bundle2: convey PushkeyFailed error over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25492
diff
changeset
|
1493 raise error.PushkeyFailed(inpart.params['in-reply-to'], **kwargs) |
d8e7b0781ad7
bundle2: convey PushkeyFailed error over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25492
diff
changeset
|
1494 |
24686
e0e28e910fa3
bundle2: rename format, parts and config to final names
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24648
diff
changeset
|
1495 @parthandler('error:unsupportedcontent', ('parttype', 'params')) |
24741
bb67e52362d6
bundle2: fix names for error part handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24733
diff
changeset
|
1496 def handleerrorunsupportedcontent(op, inpart): |
21619
292331e906d7
bundle2: rename b2x:error:unknownpart to b2x:error:unsupportedcontent
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21618
diff
changeset
|
1497 """Used to transmit unknown content error over the wire""" |
21622
457492741007
bundle2: support transmission of params error over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21620
diff
changeset
|
1498 kwargs = {} |
21627
3e8bcc90f07c
bundle2: support None parttype in BundleValueError
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21626
diff
changeset
|
1499 parttype = inpart.params.get('parttype') |
3e8bcc90f07c
bundle2: support None parttype in BundleValueError
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21626
diff
changeset
|
1500 if parttype is not None: |
3e8bcc90f07c
bundle2: support None parttype in BundleValueError
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21626
diff
changeset
|
1501 kwargs['parttype'] = parttype |
21622
457492741007
bundle2: support transmission of params error over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21620
diff
changeset
|
1502 params = inpart.params.get('params') |
457492741007
bundle2: support transmission of params error over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21620
diff
changeset
|
1503 if params is not None: |
457492741007
bundle2: support transmission of params error over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21620
diff
changeset
|
1504 kwargs['params'] = params.split('\0') |
457492741007
bundle2: support transmission of params error over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21620
diff
changeset
|
1505 |
26393
cff70549a959
bundle2: rename error exception class for unsupported feature
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26269
diff
changeset
|
1506 raise error.BundleUnknownFeatureError(**kwargs) |
21186
9f3652e851f8
bundle2: gracefully handle PushRaced error during unbundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21185
diff
changeset
|
1507 |
24686
e0e28e910fa3
bundle2: rename format, parts and config to final names
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24648
diff
changeset
|
1508 @parthandler('error:pushraced', ('message',)) |
24741
bb67e52362d6
bundle2: fix names for error part handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24733
diff
changeset
|
1509 def handleerrorpushraced(op, inpart): |
21186
9f3652e851f8
bundle2: gracefully handle PushRaced error during unbundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21185
diff
changeset
|
1510 """Used to transmit push race error over the wire""" |
21611
71b7b3f79a3c
bundle2: use the new ``part.params`` dictionary
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21610
diff
changeset
|
1511 raise error.ResponseError(_('push failed:'), inpart.params['message']) |
21655
35095f332846
bundle: introduce a listkey handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21644
diff
changeset
|
1512 |
24686
e0e28e910fa3
bundle2: rename format, parts and config to final names
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24648
diff
changeset
|
1513 @parthandler('listkeys', ('namespace',)) |
21655
35095f332846
bundle: introduce a listkey handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21644
diff
changeset
|
1514 def handlelistkeys(op, inpart): |
35095f332846
bundle: introduce a listkey handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21644
diff
changeset
|
1515 """retrieve pushkey namespace content stored in a bundle2""" |
35095f332846
bundle: introduce a listkey handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21644
diff
changeset
|
1516 namespace = inpart.params['namespace'] |
35095f332846
bundle: introduce a listkey handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21644
diff
changeset
|
1517 r = pushkey.decodekeys(inpart.read()) |
35095f332846
bundle: introduce a listkey handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21644
diff
changeset
|
1518 op.records.add('listkeys', (namespace, r)) |
21660
e87d2a12d41b
bundle2: add ``pushkey`` support
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21655
diff
changeset
|
1519 |
24686
e0e28e910fa3
bundle2: rename format, parts and config to final names
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24648
diff
changeset
|
1520 @parthandler('pushkey', ('namespace', 'key', 'old', 'new')) |
21660
e87d2a12d41b
bundle2: add ``pushkey`` support
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21655
diff
changeset
|
1521 def handlepushkey(op, inpart): |
e87d2a12d41b
bundle2: add ``pushkey`` support
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21655
diff
changeset
|
1522 """process a pushkey request""" |
e87d2a12d41b
bundle2: add ``pushkey`` support
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21655
diff
changeset
|
1523 dec = pushkey.decode |
e87d2a12d41b
bundle2: add ``pushkey`` support
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21655
diff
changeset
|
1524 namespace = dec(inpart.params['namespace']) |
e87d2a12d41b
bundle2: add ``pushkey`` support
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21655
diff
changeset
|
1525 key = dec(inpart.params['key']) |
e87d2a12d41b
bundle2: add ``pushkey`` support
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21655
diff
changeset
|
1526 old = dec(inpart.params['old']) |
e87d2a12d41b
bundle2: add ``pushkey`` support
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21655
diff
changeset
|
1527 new = dec(inpart.params['new']) |
26565
ee1bcb9aa6e4
bundle2: add op.gettransaction() to handlers that need the lock
Durham Goode <durham@fb.com>
parents:
26542
diff
changeset
|
1528 # Grab the transaction to ensure that we have the lock before performing the |
ee1bcb9aa6e4
bundle2: add op.gettransaction() to handlers that need the lock
Durham Goode <durham@fb.com>
parents:
26542
diff
changeset
|
1529 # pushkey. |
ee1bcb9aa6e4
bundle2: add op.gettransaction() to handlers that need the lock
Durham Goode <durham@fb.com>
parents:
26542
diff
changeset
|
1530 if op.ui.configbool('experimental', 'bundle2lazylocking'): |
ee1bcb9aa6e4
bundle2: add op.gettransaction() to handlers that need the lock
Durham Goode <durham@fb.com>
parents:
26542
diff
changeset
|
1531 op.gettransaction() |
21660
e87d2a12d41b
bundle2: add ``pushkey`` support
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21655
diff
changeset
|
1532 ret = op.repo.pushkey(namespace, key, old, new) |
e87d2a12d41b
bundle2: add ``pushkey`` support
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21655
diff
changeset
|
1533 record = {'namespace': namespace, |
e87d2a12d41b
bundle2: add ``pushkey`` support
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21655
diff
changeset
|
1534 'key': key, |
e87d2a12d41b
bundle2: add ``pushkey`` support
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21655
diff
changeset
|
1535 'old': old, |
e87d2a12d41b
bundle2: add ``pushkey`` support
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21655
diff
changeset
|
1536 'new': new} |
e87d2a12d41b
bundle2: add ``pushkey`` support
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21655
diff
changeset
|
1537 op.records.add('pushkey', record) |
e87d2a12d41b
bundle2: add ``pushkey`` support
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21655
diff
changeset
|
1538 if op.reply is not None: |
24686
e0e28e910fa3
bundle2: rename format, parts and config to final names
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24648
diff
changeset
|
1539 rpart = op.reply.newpart('reply:pushkey') |
21660
e87d2a12d41b
bundle2: add ``pushkey`` support
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21655
diff
changeset
|
1540 rpart.addparam('in-reply-to', str(inpart.id), mandatory=False) |
e87d2a12d41b
bundle2: add ``pushkey`` support
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21655
diff
changeset
|
1541 rpart.addparam('return', '%i' % ret, mandatory=False) |
25481
6de96cb31176
bundle2: abort when a mandatory pushkey part fails
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25401
diff
changeset
|
1542 if inpart.mandatory and not ret: |
25484
a5192774e925
bundle2: introduce a PushkeyFail error to abort unbundle on pushkey error
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25481
diff
changeset
|
1543 kwargs = {} |
a5192774e925
bundle2: introduce a PushkeyFail error to abort unbundle on pushkey error
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25481
diff
changeset
|
1544 for key in ('namespace', 'key', 'new', 'old', 'ret'): |
a5192774e925
bundle2: introduce a PushkeyFail error to abort unbundle on pushkey error
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25481
diff
changeset
|
1545 if key in inpart.params: |
a5192774e925
bundle2: introduce a PushkeyFail error to abort unbundle on pushkey error
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25481
diff
changeset
|
1546 kwargs[key] = inpart.params[key] |
a5192774e925
bundle2: introduce a PushkeyFail error to abort unbundle on pushkey error
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25481
diff
changeset
|
1547 raise error.PushkeyFailed(partid=str(inpart.id), **kwargs) |
21660
e87d2a12d41b
bundle2: add ``pushkey`` support
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21655
diff
changeset
|
1548 |
24686
e0e28e910fa3
bundle2: rename format, parts and config to final names
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24648
diff
changeset
|
1549 @parthandler('reply:pushkey', ('return', 'in-reply-to')) |
21660
e87d2a12d41b
bundle2: add ``pushkey`` support
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21655
diff
changeset
|
1550 def handlepushkeyreply(op, inpart): |
e87d2a12d41b
bundle2: add ``pushkey`` support
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21655
diff
changeset
|
1551 """retrieve the result of a pushkey request""" |
e87d2a12d41b
bundle2: add ``pushkey`` support
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21655
diff
changeset
|
1552 ret = int(inpart.params['return']) |
e87d2a12d41b
bundle2: add ``pushkey`` support
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21655
diff
changeset
|
1553 partid = int(inpart.params['in-reply-to']) |
e87d2a12d41b
bundle2: add ``pushkey`` support
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21655
diff
changeset
|
1554 op.records.add('pushkey', {'return': ret}, partid) |
22336
60786c8a2f70
bundle2: add an obsmarkers part handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21900
diff
changeset
|
1555 |
24686
e0e28e910fa3
bundle2: rename format, parts and config to final names
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24648
diff
changeset
|
1556 @parthandler('obsmarkers') |
22336
60786c8a2f70
bundle2: add an obsmarkers part handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21900
diff
changeset
|
1557 def handleobsmarker(op, inpart): |
60786c8a2f70
bundle2: add an obsmarkers part handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21900
diff
changeset
|
1558 """add a stream of obsmarkers to the repo""" |
60786c8a2f70
bundle2: add an obsmarkers part handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21900
diff
changeset
|
1559 tr = op.gettransaction() |
24733
c00e4338fa4b
obsolete: experimental flag to get debug about obsmarkers exchange
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24686
diff
changeset
|
1560 markerdata = inpart.read() |
c00e4338fa4b
obsolete: experimental flag to get debug about obsmarkers exchange
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24686
diff
changeset
|
1561 if op.ui.config('experimental', 'obsmarkers-exchange-debug', False): |
c00e4338fa4b
obsolete: experimental flag to get debug about obsmarkers exchange
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24686
diff
changeset
|
1562 op.ui.write(('obsmarker-exchange: %i bytes received\n') |
c00e4338fa4b
obsolete: experimental flag to get debug about obsmarkers exchange
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24686
diff
changeset
|
1563 % len(markerdata)) |
26685
ea390d889d3a
bundle2: gracefully skip 'obsmarkers' part if evolution is disabled
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26587
diff
changeset
|
1564 # The mergemarkers call will crash if marker creation is not enabled. |
ea390d889d3a
bundle2: gracefully skip 'obsmarkers' part if evolution is disabled
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26587
diff
changeset
|
1565 # we want to avoid this if the part is advisory. |
ea390d889d3a
bundle2: gracefully skip 'obsmarkers' part if evolution is disabled
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26587
diff
changeset
|
1566 if not inpart.mandatory and op.repo.obsstore.readonly: |
ea390d889d3a
bundle2: gracefully skip 'obsmarkers' part if evolution is disabled
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26587
diff
changeset
|
1567 op.repo.ui.debug('ignoring obsolescence markers, feature not enabled') |
ea390d889d3a
bundle2: gracefully skip 'obsmarkers' part if evolution is disabled
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26587
diff
changeset
|
1568 return |
24733
c00e4338fa4b
obsolete: experimental flag to get debug about obsmarkers exchange
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24686
diff
changeset
|
1569 new = op.repo.obsstore.mergemarkers(tr, markerdata) |
22337
c380fe290290
obsmarker: write a message with the number of markers added through bundle2
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22336
diff
changeset
|
1570 if new: |
c380fe290290
obsmarker: write a message with the number of markers added through bundle2
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22336
diff
changeset
|
1571 op.repo.ui.status(_('%i new obsolescence markers\n') % new) |
22338
8fcd56095d3b
obssmarker: add a bundle2 record with the number of markers added
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22337
diff
changeset
|
1572 op.records.add('obsmarkers', {'new': new}) |
22340
394a17de6a2d
obsmarker: produce a reply part for markers received through bundle2
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22338
diff
changeset
|
1573 if op.reply is not None: |
24686
e0e28e910fa3
bundle2: rename format, parts and config to final names
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24648
diff
changeset
|
1574 rpart = op.reply.newpart('reply:obsmarkers') |
22340
394a17de6a2d
obsmarker: produce a reply part for markers received through bundle2
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22338
diff
changeset
|
1575 rpart.addparam('in-reply-to', str(inpart.id), mandatory=False) |
394a17de6a2d
obsmarker: produce a reply part for markers received through bundle2
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22338
diff
changeset
|
1576 rpart.addparam('new', '%i' % new, mandatory=False) |
394a17de6a2d
obsmarker: produce a reply part for markers received through bundle2
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22338
diff
changeset
|
1577 |
394a17de6a2d
obsmarker: produce a reply part for markers received through bundle2
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22338
diff
changeset
|
1578 |
24686
e0e28e910fa3
bundle2: rename format, parts and config to final names
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24648
diff
changeset
|
1579 @parthandler('reply:obsmarkers', ('new', 'in-reply-to')) |
25506
86472038dfd2
bundle2: rename duplicate handlepushkeyreply to handleobsmarkerreply
Martin von Zweigbergk <martinvonz@google.com>
parents:
25493
diff
changeset
|
1580 def handleobsmarkerreply(op, inpart): |
22340
394a17de6a2d
obsmarker: produce a reply part for markers received through bundle2
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22338
diff
changeset
|
1581 """retrieve the result of a pushkey request""" |
394a17de6a2d
obsmarker: produce a reply part for markers received through bundle2
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22338
diff
changeset
|
1582 ret = int(inpart.params['new']) |
394a17de6a2d
obsmarker: produce a reply part for markers received through bundle2
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22338
diff
changeset
|
1583 partid = int(inpart.params['in-reply-to']) |
394a17de6a2d
obsmarker: produce a reply part for markers received through bundle2
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22338
diff
changeset
|
1584 op.records.add('obsmarkers', {'new': ret}, partid) |
25401
d29201352af7
bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25336
diff
changeset
|
1585 |
d29201352af7
bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25336
diff
changeset
|
1586 @parthandler('hgtagsfnodes') |
d29201352af7
bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25336
diff
changeset
|
1587 def handlehgtagsfnodes(op, inpart): |
d29201352af7
bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25336
diff
changeset
|
1588 """Applies .hgtags fnodes cache entries to the local repo. |
d29201352af7
bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25336
diff
changeset
|
1589 |
d29201352af7
bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25336
diff
changeset
|
1590 Payload is pairs of 20 byte changeset nodes and filenodes. |
d29201352af7
bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25336
diff
changeset
|
1591 """ |
26565
ee1bcb9aa6e4
bundle2: add op.gettransaction() to handlers that need the lock
Durham Goode <durham@fb.com>
parents:
26542
diff
changeset
|
1592 # Grab the transaction so we ensure that we have the lock at this point. |
ee1bcb9aa6e4
bundle2: add op.gettransaction() to handlers that need the lock
Durham Goode <durham@fb.com>
parents:
26542
diff
changeset
|
1593 if op.ui.configbool('experimental', 'bundle2lazylocking'): |
ee1bcb9aa6e4
bundle2: add op.gettransaction() to handlers that need the lock
Durham Goode <durham@fb.com>
parents:
26542
diff
changeset
|
1594 op.gettransaction() |
25401
d29201352af7
bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25336
diff
changeset
|
1595 cache = tags.hgtagsfnodescache(op.repo.unfiltered()) |
d29201352af7
bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25336
diff
changeset
|
1596 |
d29201352af7
bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25336
diff
changeset
|
1597 count = 0 |
d29201352af7
bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25336
diff
changeset
|
1598 while True: |
d29201352af7
bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25336
diff
changeset
|
1599 node = inpart.read(20) |
d29201352af7
bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25336
diff
changeset
|
1600 fnode = inpart.read(20) |
d29201352af7
bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25336
diff
changeset
|
1601 if len(node) < 20 or len(fnode) < 20: |
25641
c0bdfe87b245
bundle2: reword debug message for invalid .hgtags data
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25640
diff
changeset
|
1602 op.ui.debug('ignoring incomplete received .hgtags fnodes data\n') |
25401
d29201352af7
bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25336
diff
changeset
|
1603 break |
d29201352af7
bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25336
diff
changeset
|
1604 cache.setfnode(node, fnode) |
d29201352af7
bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25336
diff
changeset
|
1605 count += 1 |
d29201352af7
bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25336
diff
changeset
|
1606 |
d29201352af7
bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25336
diff
changeset
|
1607 cache.write() |
d29201352af7
bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25336
diff
changeset
|
1608 op.ui.debug('applied %i hgtags fnodes cache entries\n' % count) |