mercurial/bundle2.py
author Joerg Sonnenberger <joerg@bec.de>
Mon, 29 Mar 2021 01:52:06 +0200
changeset 47055 d55b71393907
parent 46907 ffd3e823a7e5
child 47096 1680c94741f8
permissions -rw-r--r--
node: replace nullid and friends with nodeconstants class The introduction of 256bit hashes require changes to nullid and other constant magic values. Start pushing them down from repository and revlog where sensible. Differential Revision: https://phab.mercurial-scm.org/D9465
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
20801
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
     1
# bundle2.py - generic container format to transmit arbitrary data.
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
     2
#
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
     3
# Copyright 2013 Facebook, Inc.
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
     4
#
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
     5
# This software may be used and distributed according to the terms of the
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
     6
# GNU General Public License version 2 or any later version.
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
     7
"""Handling of the new bundle2 format
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
     8
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
     9
The goal of bundle2 is to act as an atomically packet to transmit a set of
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    10
payloads in an application agnostic way. It consist in a sequence of "parts"
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    11
that will be handed to and processed by the application layer.
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    12
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    13
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    14
General format architecture
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    15
===========================
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    16
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    17
The format is architectured as follow
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    18
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    19
 - magic string
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    20
 - stream level parameters
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    21
 - payload parts (any number)
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    22
 - end of stream marker.
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    23
20856
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
    24
the Binary format
20801
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    25
============================
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    26
21024
7731a2281cf0 spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents: 21020
diff changeset
    27
All numbers are unsigned and big-endian.
20801
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    28
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    29
stream level parameters
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    30
------------------------
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    31
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    32
Binary format is as follow
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    33
23009
90f86ad3d4ff bundle2: change header size and make them signed (new format)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23008
diff changeset
    34
:params size: int32
20801
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    35
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    36
  The total number of Bytes used by the parameters
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    37
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    38
:params value: arbitrary number of Bytes
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    39
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    40
  A blob of `params size` containing the serialized version of all stream level
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    41
  parameters.
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    42
21024
7731a2281cf0 spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents: 21020
diff changeset
    43
  The blob contains a space separated list of parameters. Parameters with value
20811
9785c3f8f598 bundle2: urlquote stream parameter name and value
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20810
diff changeset
    44
  are stored in the form `<name>=<value>`. Both name and value are urlquoted.
20804
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
    45
20813
8c74b3ce5b70 bundle2: refuse empty parameter name
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20812
diff changeset
    46
  Empty name are obviously forbidden.
8c74b3ce5b70 bundle2: refuse empty parameter name
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20812
diff changeset
    47
20844
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
    48
  Name MUST start with a letter. If this first letter is lower case, the
21024
7731a2281cf0 spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents: 21020
diff changeset
    49
  parameter is advisory and can be safely ignored. However when the first
20844
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
    50
  letter is capital, the parameter is mandatory and the bundling process MUST
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
    51
  stop if he is not able to proceed it.
20814
8532f5e1b9df bundle2: force the first char of parameter to be an letter.
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20813
diff changeset
    52
20808
4c9130c7a29f bundle2: clarify stream parameter design in the documentation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20805
diff changeset
    53
  Stream parameters use a simple textual format for two main reasons:
20804
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
    54
21024
7731a2281cf0 spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents: 21020
diff changeset
    55
  - Stream level parameters should remain simple and we want to discourage any
20808
4c9130c7a29f bundle2: clarify stream parameter design in the documentation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20805
diff changeset
    56
    crazy usage.
21024
7731a2281cf0 spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents: 21020
diff changeset
    57
  - Textual data allow easy human inspection of a bundle2 header in case of
20808
4c9130c7a29f bundle2: clarify stream parameter design in the documentation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20805
diff changeset
    58
    troubles.
4c9130c7a29f bundle2: clarify stream parameter design in the documentation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20805
diff changeset
    59
4c9130c7a29f bundle2: clarify stream parameter design in the documentation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20805
diff changeset
    60
  Any Applicative level options MUST go into a bundle2 part instead.
20801
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    61
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    62
Payload part
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    63
------------------------
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    64
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    65
Binary format is as follow
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    66
23009
90f86ad3d4ff bundle2: change header size and make them signed (new format)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23008
diff changeset
    67
:header size: int32
20801
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    68
25507
5bee483705e8 bundle2: clarify in docstring that header size is for a single header
Martin von Zweigbergk <martinvonz@google.com>
parents: 25506
diff changeset
    69
  The total number of Bytes used by the part header. When the header is empty
20801
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    70
  (size = 0) this is interpreted as the end of stream marker.
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    71
20856
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
    72
:header:
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
    73
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
    74
    The header defines how to interpret the part. It contains two piece of
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
    75
    data: the part type, and the part parameters.
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
    76
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
    77
    The part type is used to route an application level handler, that can
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
    78
    interpret payload.
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
    79
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
    80
    Part parameters are passed to the application level handler.  They are
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
    81
    meant to convey information that will help the application level object to
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
    82
    interpret the part payload.
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
    83
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
    84
    The binary format of the header is has follow
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
    85
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
    86
    :typesize: (one byte)
20877
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
    87
23916
a3f7c781786b bundle2: fix parttype enforcement
Matt Mackall <mpm@selenic.com>
parents: 23868
diff changeset
    88
    :parttype: alphanumerical part name (restricted to [a-zA-Z0-9_:-]*)
20877
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
    89
20995
e995d104c87f bundle2: add an integer id to part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20952
diff changeset
    90
    :partid: A 32bits integer (unique in the bundle) that can be used to refer
e995d104c87f bundle2: add an integer id to part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20952
diff changeset
    91
             to this part.
e995d104c87f bundle2: add an integer id to part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20952
diff changeset
    92
20877
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
    93
    :parameters:
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
    94
21024
7731a2281cf0 spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents: 21020
diff changeset
    95
        Part's parameter may have arbitrary content, the binary structure is::
20877
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
    96
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
    97
            <mandatory-count><advisory-count><param-sizes><param-data>
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
    98
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
    99
        :mandatory-count: 1 byte, number of mandatory parameters
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
   100
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
   101
        :advisory-count:  1 byte, number of advisory parameters
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
   102
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
   103
        :param-sizes:
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
   104
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
   105
            N couple of bytes, where N is the total number of parameters. Each
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
   106
            couple contains (<size-of-key>, <size-of-value) for one parameter.
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
   107
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
   108
        :param-data:
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
   109
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
   110
            A blob of bytes from which each parameter key and value can be
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
   111
            retrieved using the list of size couples stored in the previous
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
   112
            field.
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
   113
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
   114
            Mandatory parameters comes first, then the advisory ones.
20856
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
   115
21607
054fa5176fa7 bundle2: forbid duplicate parameter keys
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21606
diff changeset
   116
            Each parameter's key MUST be unique within the part.
054fa5176fa7 bundle2: forbid duplicate parameter keys
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21606
diff changeset
   117
20856
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
   118
:payload:
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
   119
20876
ddd56f3eb786 bundle2: support for bundling and unbundling payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20864
diff changeset
   120
    payload is a series of `<chunksize><chunkdata>`.
ddd56f3eb786 bundle2: support for bundling and unbundling payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20864
diff changeset
   121
23009
90f86ad3d4ff bundle2: change header size and make them signed (new format)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23008
diff changeset
   122
    `chunksize` is an int32, `chunkdata` are plain bytes (as much as
20876
ddd56f3eb786 bundle2: support for bundling and unbundling payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20864
diff changeset
   123
    `chunksize` says)` The payload part is concluded by a zero size chunk.
ddd56f3eb786 bundle2: support for bundling and unbundling payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20864
diff changeset
   124
ddd56f3eb786 bundle2: support for bundling and unbundling payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20864
diff changeset
   125
    The current implementation always produces either zero or one chunk.
21024
7731a2281cf0 spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents: 21020
diff changeset
   126
    This is an implementation limitation that will ultimately be lifted.
20891
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20890
diff changeset
   127
23009
90f86ad3d4ff bundle2: change header size and make them signed (new format)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23008
diff changeset
   128
    `chunksize` can be negative to trigger special case processing. No such
90f86ad3d4ff bundle2: change header size and make them signed (new format)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23008
diff changeset
   129
    processing is in place yet.
90f86ad3d4ff bundle2: change header size and make them signed (new format)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23008
diff changeset
   130
20891
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20890
diff changeset
   131
Bundle processing
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20890
diff changeset
   132
============================
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20890
diff changeset
   133
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20890
diff changeset
   134
Each part is processed in order using a "part handler". Handler are registered
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20890
diff changeset
   135
for a certain part type.
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20890
diff changeset
   136
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20890
diff changeset
   137
The matching of a part to its handler is case insensitive. The case of the
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20890
diff changeset
   138
part type is used to know if a part is mandatory or advisory. If the Part type
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20890
diff changeset
   139
contains any uppercase char it is considered mandatory. When no handler is
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20890
diff changeset
   140
known for a Mandatory part, the process is aborted and an exception is raised.
20892
6fe95448596d bundle2: read the whole bundle from stream on abort
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20891
diff changeset
   141
If the part is advisory and no handler is known, the part is ignored. When the
6fe95448596d bundle2: read the whole bundle from stream on abort
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20891
diff changeset
   142
process is aborted, the full bundle is still read from the stream to keep the
6fe95448596d bundle2: read the whole bundle from stream on abort
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20891
diff changeset
   143
channel usable. But none of the part read from an abort are processed. In the
6fe95448596d bundle2: read the whole bundle from stream on abort
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20891
diff changeset
   144
future, dropping the stream may become an option for channel we do not care to
6fe95448596d bundle2: read the whole bundle from stream on abort
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20891
diff changeset
   145
preserve.
20801
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
   146
"""
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
   147
33698
da7c285ec6da bundle2: use modern Python division
Augie Fackler <augie@google.com>
parents: 33693
diff changeset
   148
from __future__ import absolute_import, division
25919
8221fefaea08 bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
   149
36970
79b73be4dda5 rev-branch-cache: add a function to generate a part
Boris Feld <boris.feld@octobus.net>
parents: 36969
diff changeset
   150
import collections
24026
3daef83a1873 bundle2.unpackermixin: control for underlying file descriptor
Eric Sumner <ericsumner@fb.com>
parents: 23916
diff changeset
   151
import errno
35069
241d9caca11e bundle2: use os.SEEK_* constants
Gregory Szorc <gregory.szorc@gmail.com>
parents: 34820
diff changeset
   152
import os
25919
8221fefaea08 bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
   153
import re
20814
8532f5e1b9df bundle2: force the first char of parameter to be an letter.
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20813
diff changeset
   154
import string
25919
8221fefaea08 bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
   155
import struct
8221fefaea08 bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
   156
import sys
20804
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
   157
25919
8221fefaea08 bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
   158
from .i18n import _
46114
59fa3890d40a node: import symbols explicitly
Joerg Sonnenberger <joerg@bec.de>
parents: 45957
diff changeset
   159
from .node import (
59fa3890d40a node: import symbols explicitly
Joerg Sonnenberger <joerg@bec.de>
parents: 45957
diff changeset
   160
    hex,
59fa3890d40a node: import symbols explicitly
Joerg Sonnenberger <joerg@bec.de>
parents: 45957
diff changeset
   161
    short,
59fa3890d40a node: import symbols explicitly
Joerg Sonnenberger <joerg@bec.de>
parents: 45957
diff changeset
   162
)
25919
8221fefaea08 bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
   163
from . import (
35267
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35141
diff changeset
   164
    bookmarks,
25919
8221fefaea08 bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
   165
    changegroup,
36969
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part
Boris Feld <boris.feld@octobus.net>
parents: 36458
diff changeset
   166
    encoding,
25919
8221fefaea08 bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
   167
    error,
8221fefaea08 bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
   168
    obsolete,
33043
e8c8d81eb864 bundle: add config option to include phases
Martin von Zweigbergk <martinvonz@google.com>
parents: 33042
diff changeset
   169
    phases,
25919
8221fefaea08 bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
   170
    pushkey,
30031
0f6d6fdd3c2a pycompat: provide 'ispy3' constant
Yuya Nishihara <yuya@tcha.org>
parents: 29941
diff changeset
   171
    pycompat,
45392
77b8588dd84e requirements: introduce new requirements related module
Pulkit Goyal <7895pulkit@gmail.com>
parents: 45391
diff changeset
   172
    requirements,
45106
a03c177a4679 scmutil: add writereporequirements() and route requires writing through it
Pulkit Goyal <7895pulkit@gmail.com>
parents: 44406
diff changeset
   173
    scmutil,
35758
b996ddf5963d bundle2: add a 'stream' part handler for stream cloning
Boris Feld <boris.feld@octobus.net>
parents: 35655
diff changeset
   174
    streamclone,
25919
8221fefaea08 bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
   175
    tags,
8221fefaea08 bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
   176
    url,
8221fefaea08 bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
   177
    util,
8221fefaea08 bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
   178
)
46907
ffd3e823a7e5 urlutil: extract `url` related code from `util` into the new module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46793
diff changeset
   179
from .utils import (
ffd3e823a7e5 urlutil: extract `url` related code from `util` into the new module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46793
diff changeset
   180
    stringutil,
ffd3e823a7e5 urlutil: extract `url` related code from `util` into the new module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46793
diff changeset
   181
    urlutil,
ffd3e823a7e5 urlutil: extract `url` related code from `util` into the new module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46793
diff changeset
   182
)
20802
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
   183
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28672
diff changeset
   184
urlerr = util.urlerr
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28672
diff changeset
   185
urlreq = util.urlreq
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28672
diff changeset
   186
20804
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
   187
_pack = struct.pack
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
   188
_unpack = struct.unpack
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
   189
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   190
_fstreamparamsize = b'>i'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   191
_fpartheadersize = b'>i'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   192
_fparttypesize = b'>B'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   193
_fpartid = b'>I'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   194
_fpayloadsize = b'>i'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   195
_fpartparamcount = b'>BB'
20877
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
   196
35793
4fb2bb61597c bundle2: increase payload part chunk size to 32kb
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35792
diff changeset
   197
preferedchunksize = 32768
21001
c93bb6a08fa1 bundle2: support chunk iterator as part data
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21000
diff changeset
   198
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   199
_parttypeforbidden = re.compile(b'[^a-zA-Z0-9_:-]')
23868
405eacbecc52 bundle2: enforce parttype as alphanumerical
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23590
diff changeset
   200
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
   201
25313
8f2c362bcb58 bundle2: introduce a specific function for bundling debug message
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25181
diff changeset
   202
def outdebug(ui, message):
8f2c362bcb58 bundle2: introduce a specific function for bundling debug message
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25181
diff changeset
   203
    """debug regarding output stream (bundling)"""
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   204
    if ui.configbool(b'devel', b'bundle2.debug'):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   205
        ui.debug(b'bundle2-output: %s\n' % message)
25313
8f2c362bcb58 bundle2: introduce a specific function for bundling debug message
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25181
diff changeset
   206
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
   207
25318
64dd8f25fcf4 bundle2: introduce a specific function for debug messages while unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25317
diff changeset
   208
def indebug(ui, message):
64dd8f25fcf4 bundle2: introduce a specific function for debug messages while unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25317
diff changeset
   209
    """debug on input stream (unbundling)"""
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   210
    if ui.configbool(b'devel', b'bundle2.debug'):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   211
        ui.debug(b'bundle2-input: %s\n' % message)
25318
64dd8f25fcf4 bundle2: introduce a specific function for debug messages while unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25317
diff changeset
   212
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
   213
23868
405eacbecc52 bundle2: enforce parttype as alphanumerical
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23590
diff changeset
   214
def validateparttype(parttype):
405eacbecc52 bundle2: enforce parttype as alphanumerical
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23590
diff changeset
   215
    """raise ValueError if a parttype contains invalid character"""
23916
a3f7c781786b bundle2: fix parttype enforcement
Matt Mackall <mpm@selenic.com>
parents: 23868
diff changeset
   216
    if _parttypeforbidden.search(parttype):
23868
405eacbecc52 bundle2: enforce parttype as alphanumerical
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23590
diff changeset
   217
        raise ValueError(parttype)
405eacbecc52 bundle2: enforce parttype as alphanumerical
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23590
diff changeset
   218
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
   219
20877
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
   220
def _makefpartparamsizes(nbparams):
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
   221
    """return a struct format to read part parameter sizes
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
   222
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
   223
    The number parameters is variable so we need to build that format
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
   224
    dynamically.
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
   225
    """
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   226
    return b'>' + (b'BB' * nbparams)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
   227
20804
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
   228
20890
ec7fc110faee bundle2: introduce a `parthandler` decorator
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20889
diff changeset
   229
parthandlermapping = {}
20889
deed5edb72de bundle2: first version of a bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20887
diff changeset
   230
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
   231
21623
5b26d82e4e2a bundle2: make it possible to declare params handled by a part handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21622
diff changeset
   232
def parthandler(parttype, params=()):
20890
ec7fc110faee bundle2: introduce a `parthandler` decorator
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20889
diff changeset
   233
    """decorator that register a function as a bundle2 part handler
ec7fc110faee bundle2: introduce a `parthandler` decorator
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20889
diff changeset
   234
ec7fc110faee bundle2: introduce a `parthandler` decorator
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20889
diff changeset
   235
    eg::
ec7fc110faee bundle2: introduce a `parthandler` decorator
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20889
diff changeset
   236
21624
d61066d787c8 bundle2: declare supported parameters for all handlers
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21623
diff changeset
   237
        @parthandler('myparttype', ('mandatory', 'param', 'handled'))
20890
ec7fc110faee bundle2: introduce a `parthandler` decorator
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20889
diff changeset
   238
        def myparttypehandler(...):
ec7fc110faee bundle2: introduce a `parthandler` decorator
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20889
diff changeset
   239
            '''process a part of type "my part".'''
ec7fc110faee bundle2: introduce a `parthandler` decorator
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20889
diff changeset
   240
            ...
ec7fc110faee bundle2: introduce a `parthandler` decorator
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20889
diff changeset
   241
    """
23868
405eacbecc52 bundle2: enforce parttype as alphanumerical
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23590
diff changeset
   242
    validateparttype(parttype)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
   243
20890
ec7fc110faee bundle2: introduce a `parthandler` decorator
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20889
diff changeset
   244
    def _decorator(func):
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
   245
        lparttype = parttype.lower()  # enforce lower case matching.
20891
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20890
diff changeset
   246
        assert lparttype not in parthandlermapping
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20890
diff changeset
   247
        parthandlermapping[lparttype] = func
21623
5b26d82e4e2a bundle2: make it possible to declare params handled by a part handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21622
diff changeset
   248
        func.params = frozenset(params)
20890
ec7fc110faee bundle2: introduce a `parthandler` decorator
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20889
diff changeset
   249
        return func
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
   250
20890
ec7fc110faee bundle2: introduce a `parthandler` decorator
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20889
diff changeset
   251
    return _decorator
20889
deed5edb72de bundle2: first version of a bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20887
diff changeset
   252
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
   253
20949
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
   254
class unbundlerecords(object):
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
   255
    """keep record of what happens during and unbundle
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
   256
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
   257
    New records are added using `records.add('cat', obj)`. Where 'cat' is a
21024
7731a2281cf0 spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents: 21020
diff changeset
   258
    category of record and obj is an arbitrary object.
20949
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
   259
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
   260
    `records['cat']` will return all entries of this category 'cat'.
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
   261
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
   262
    Iterating on the object itself will yield `('category', obj)` tuples
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
   263
    for all entries.
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
   264
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
   265
    All iterations happens in chronological order.
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
   266
    """
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
   267
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
   268
    def __init__(self):
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
   269
        self._categories = {}
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
   270
        self._sequences = []
20996
ed3c5e18a047 bundle2: add reply awareness to unbundlerecords
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20995
diff changeset
   271
        self._replies = {}
20949
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
   272
20996
ed3c5e18a047 bundle2: add reply awareness to unbundlerecords
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20995
diff changeset
   273
    def add(self, category, entry, inreplyto=None):
20949
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
   274
        """add a new record of a given category.
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
   275
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
   276
        The entry can then be retrieved in the list returned by
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
   277
        self['category']."""
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
   278
        self._categories.setdefault(category, []).append(entry)
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
   279
        self._sequences.append((category, entry))
20996
ed3c5e18a047 bundle2: add reply awareness to unbundlerecords
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20995
diff changeset
   280
        if inreplyto is not None:
ed3c5e18a047 bundle2: add reply awareness to unbundlerecords
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20995
diff changeset
   281
            self.getreplies(inreplyto).add(category, entry)
ed3c5e18a047 bundle2: add reply awareness to unbundlerecords
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20995
diff changeset
   282
ed3c5e18a047 bundle2: add reply awareness to unbundlerecords
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20995
diff changeset
   283
    def getreplies(self, partid):
23139
e53f6b72a0e4 spelling: fixes from proofreading of spell checker issues
Mads Kiilerich <madski@unity3d.com>
parents: 23067
diff changeset
   284
        """get the records that are replies to a specific part"""
20996
ed3c5e18a047 bundle2: add reply awareness to unbundlerecords
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20995
diff changeset
   285
        return self._replies.setdefault(partid, unbundlerecords())
20949
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
   286
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
   287
    def __getitem__(self, cat):
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
   288
        return tuple(self._categories.get(cat, ()))
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
   289
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
   290
    def __iter__(self):
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
   291
        return iter(self._sequences)
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
   292
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
   293
    def __len__(self):
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
   294
        return len(self._sequences)
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
   295
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
   296
    def __nonzero__(self):
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
   297
        return bool(self._sequences)
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
   298
31483
413b44003462 py3: add __bool__ to every class defining __nonzero__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30985
diff changeset
   299
    __bool__ = __nonzero__
413b44003462 py3: add __bool__ to every class defining __nonzero__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30985
diff changeset
   300
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
   301
20948
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
   302
class bundleoperation(object):
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
   303
    """an object that represents a single bundling process
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
   304
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
   305
    Its purpose is to carry unbundle-related objects and states.
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
   306
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
   307
    A new object should be created at the beginning of each bundle processing.
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
   308
    The object is to be returned by the processing function.
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
   309
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
   310
    The object has very little content now it will ultimately contain:
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
   311
    * an access to the repo the bundle is applied to,
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
   312
    * a ui object,
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
   313
    * a way to retrieve a transaction to add changes to the repo,
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
   314
    * a way to record the result of processing each part,
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
   315
    * a way to construct a bundle response when applicable.
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
   316
    """
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
   317
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   318
    def __init__(self, repo, transactiongetter, captureoutput=True, source=b''):
20948
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
   319
        self.repo = repo
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
   320
        self.ui = repo.ui
20949
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
   321
        self.records = unbundlerecords()
20997
d7df4b7378ae bundle2: produce a bundle2 reply
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20996
diff changeset
   322
        self.reply = None
24878
e530cde6d115 bundle2: disable ouput capture unless we use http (issue4613 issue4615)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24851
diff changeset
   323
        self.captureoutput = captureoutput
33692
f3407d56a6e8 bundle2: add the capability to store hookargs on bundle operation object
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33541
diff changeset
   324
        self.hookargs = {}
33693
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33692
diff changeset
   325
        self._gettransaction = transactiongetter
35274
1f30cbac34b6 bundle2: add a 'modes' dictionary to the bundle operation
Boris Feld <boris.feld@octobus.net>
parents: 35270
diff changeset
   326
        # carries value that can modify part behavior
1f30cbac34b6 bundle2: add a 'modes' dictionary to the bundle operation
Boris Feld <boris.feld@octobus.net>
parents: 35270
diff changeset
   327
        self.modes = {}
37237
7e906d8a825f bundle2: add 'source' atrribute to bundleoperation class
Pulkit Goyal <7895pulkit@gmail.com>
parents: 37169
diff changeset
   328
        self.source = source
33693
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33692
diff changeset
   329
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33692
diff changeset
   330
    def gettransaction(self):
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33692
diff changeset
   331
        transaction = self._gettransaction()
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33692
diff changeset
   332
33831
b3a41f4d837b bundle2: relax the condition to update transaction.hookargs
Yuya Nishihara <yuya@tcha.org>
parents: 33830
diff changeset
   333
        if self.hookargs:
33693
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33692
diff changeset
   334
            # the ones added to the transaction supercede those added
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33692
diff changeset
   335
            # to the operation.
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33692
diff changeset
   336
            self.hookargs.update(transaction.hookargs)
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33692
diff changeset
   337
            transaction.hookargs = self.hookargs
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33692
diff changeset
   338
33831
b3a41f4d837b bundle2: relax the condition to update transaction.hookargs
Yuya Nishihara <yuya@tcha.org>
parents: 33830
diff changeset
   339
        # mark the hookargs as flushed.  further attempts to add to
b3a41f4d837b bundle2: relax the condition to update transaction.hookargs
Yuya Nishihara <yuya@tcha.org>
parents: 33830
diff changeset
   340
        # hookargs will result in an abort.
b3a41f4d837b bundle2: relax the condition to update transaction.hookargs
Yuya Nishihara <yuya@tcha.org>
parents: 33830
diff changeset
   341
        self.hookargs = None
33693
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33692
diff changeset
   342
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33692
diff changeset
   343
        return transaction
33692
f3407d56a6e8 bundle2: add the capability to store hookargs on bundle operation object
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33541
diff changeset
   344
f3407d56a6e8 bundle2: add the capability to store hookargs on bundle operation object
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33541
diff changeset
   345
    def addhookargs(self, hookargs):
33693
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33692
diff changeset
   346
        if self.hookargs is None:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
   347
            raise error.ProgrammingError(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   348
                b'attempted to add hookargs to '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   349
                b'operation after transaction started'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
   350
            )
33692
f3407d56a6e8 bundle2: add the capability to store hookargs on bundle operation object
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33541
diff changeset
   351
        self.hookargs.update(hookargs)
20948
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
   352
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
   353
20952
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20950
diff changeset
   354
class TransactionUnavailable(RuntimeError):
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20950
diff changeset
   355
    pass
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20950
diff changeset
   356
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
   357
20952
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20950
diff changeset
   358
def _notransaction():
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20950
diff changeset
   359
    """default method to get a transaction while processing a bundle
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20950
diff changeset
   360
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20950
diff changeset
   361
    Raise an exception to highlight the fact that no transaction was expected
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20950
diff changeset
   362
    to be created"""
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20950
diff changeset
   363
    raise TransactionUnavailable()
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20950
diff changeset
   364
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
   365
37240
684a6a261f30 bundle2: make source a mandatory argument for bundle2.applybundle() (API)
Pulkit Goyal <7895pulkit@gmail.com>
parents: 37239
diff changeset
   366
def applybundle(repo, unbundler, tr, source, url=None, **kwargs):
26790
28a6c2d72097 bundle2: introduce an "applybundle" function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26698
diff changeset
   367
    # transform me into unbundler.apply() as soon as the freeze is lifted
33055
18c2489ac96d bundle: make applybundle() delegate v1 bundles to applybundle1()
Martin von Zweigbergk <martinvonz@google.com>
parents: 33052
diff changeset
   368
    if isinstance(unbundler, unbundle20):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   369
        tr.hookargs[b'bundle2'] = b'1'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   370
        if source is not None and b'source' not in tr.hookargs:
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   371
            tr.hookargs[b'source'] = source
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   372
        if url is not None and b'url' not in tr.hookargs:
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   373
            tr.hookargs[b'url'] = url
37238
9041c91561fc bunlde2: add 'source' as an optional argument to processbundle()
Pulkit Goyal <7895pulkit@gmail.com>
parents: 37237
diff changeset
   374
        return processbundle(repo, unbundler, lambda: tr, source=source)
33055
18c2489ac96d bundle: make applybundle() delegate v1 bundles to applybundle1()
Martin von Zweigbergk <martinvonz@google.com>
parents: 33052
diff changeset
   375
    else:
33056
1d2b68952950 bundle: inline applybundle1()
Martin von Zweigbergk <martinvonz@google.com>
parents: 33055
diff changeset
   376
        # the transactiongetter won't be used, but we might as well set it
37239
f7d3915d5b3a bundleoperation: pass the source argument from all the users
Pulkit Goyal <7895pulkit@gmail.com>
parents: 37238
diff changeset
   377
        op = bundleoperation(repo, lambda: tr, source=source)
33056
1d2b68952950 bundle: inline applybundle1()
Martin von Zweigbergk <martinvonz@google.com>
parents: 33055
diff changeset
   378
        _processchangegroup(op, unbundler, tr, source, url, **kwargs)
1d2b68952950 bundle: inline applybundle1()
Martin von Zweigbergk <martinvonz@google.com>
parents: 33055
diff changeset
   379
        return op
26790
28a6c2d72097 bundle2: introduce an "applybundle" function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26698
diff changeset
   380
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
   381
34162
e9e0e1143fc5 bundle2: move part iterator a separate class
Durham Goode <durham@fb.com>
parents: 34116
diff changeset
   382
class partiterator(object):
34164
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
   383
    def __init__(self, repo, op, unbundler):
34163
550343626bb2 bundle2: move part counter to partiterator
Durham Goode <durham@fb.com>
parents: 34162
diff changeset
   384
        self.repo = repo
34164
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
   385
        self.op = op
34162
e9e0e1143fc5 bundle2: move part iterator a separate class
Durham Goode <durham@fb.com>
parents: 34116
diff changeset
   386
        self.unbundler = unbundler
34163
550343626bb2 bundle2: move part counter to partiterator
Durham Goode <durham@fb.com>
parents: 34162
diff changeset
   387
        self.iterator = None
550343626bb2 bundle2: move part counter to partiterator
Durham Goode <durham@fb.com>
parents: 34162
diff changeset
   388
        self.count = 0
34265
e71890f27767 bundle2: move processpart stream maintenance into part iterator
Durham Goode <durham@fb.com>
parents: 34256
diff changeset
   389
        self.current = None
34162
e9e0e1143fc5 bundle2: move part iterator a separate class
Durham Goode <durham@fb.com>
parents: 34116
diff changeset
   390
e9e0e1143fc5 bundle2: move part iterator a separate class
Durham Goode <durham@fb.com>
parents: 34116
diff changeset
   391
    def __enter__(self):
34163
550343626bb2 bundle2: move part counter to partiterator
Durham Goode <durham@fb.com>
parents: 34162
diff changeset
   392
        def func():
42935
181ee2118a96 bundle2: fix an off-by-one in debug message of number of parts
Martin von Zweigbergk <martinvonz@google.com>
parents: 42900
diff changeset
   393
            itr = enumerate(self.unbundler.iterparts(), 1)
34163
550343626bb2 bundle2: move part counter to partiterator
Durham Goode <durham@fb.com>
parents: 34162
diff changeset
   394
            for count, p in itr:
550343626bb2 bundle2: move part counter to partiterator
Durham Goode <durham@fb.com>
parents: 34162
diff changeset
   395
                self.count = count
34265
e71890f27767 bundle2: move processpart stream maintenance into part iterator
Durham Goode <durham@fb.com>
parents: 34256
diff changeset
   396
                self.current = p
34163
550343626bb2 bundle2: move part counter to partiterator
Durham Goode <durham@fb.com>
parents: 34162
diff changeset
   397
                yield p
35135
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35134
diff changeset
   398
                p.consume()
34265
e71890f27767 bundle2: move processpart stream maintenance into part iterator
Durham Goode <durham@fb.com>
parents: 34256
diff changeset
   399
                self.current = None
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
   400
34163
550343626bb2 bundle2: move part counter to partiterator
Durham Goode <durham@fb.com>
parents: 34162
diff changeset
   401
        self.iterator = func()
550343626bb2 bundle2: move part counter to partiterator
Durham Goode <durham@fb.com>
parents: 34162
diff changeset
   402
        return self.iterator
34162
e9e0e1143fc5 bundle2: move part iterator a separate class
Durham Goode <durham@fb.com>
parents: 34116
diff changeset
   403
34164
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
   404
    def __exit__(self, type, exc, tb):
34163
550343626bb2 bundle2: move part counter to partiterator
Durham Goode <durham@fb.com>
parents: 34162
diff changeset
   405
        if not self.iterator:
550343626bb2 bundle2: move part counter to partiterator
Durham Goode <durham@fb.com>
parents: 34162
diff changeset
   406
            return
550343626bb2 bundle2: move part counter to partiterator
Durham Goode <durham@fb.com>
parents: 34162
diff changeset
   407
34637
5f79f5f8487a bundle2: immediate exit for ctrl+c (issue5692)
Durham Goode <durham@fb.com>
parents: 34330
diff changeset
   408
        # Only gracefully abort in a normal exception situation. User aborts
5f79f5f8487a bundle2: immediate exit for ctrl+c (issue5692)
Durham Goode <durham@fb.com>
parents: 34330
diff changeset
   409
        # like Ctrl+C throw a KeyboardInterrupt which is not a base Exception,
5f79f5f8487a bundle2: immediate exit for ctrl+c (issue5692)
Durham Goode <durham@fb.com>
parents: 34330
diff changeset
   410
        # and should not gracefully cleanup.
5f79f5f8487a bundle2: immediate exit for ctrl+c (issue5692)
Durham Goode <durham@fb.com>
parents: 34330
diff changeset
   411
        if isinstance(exc, Exception):
34164
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
   412
            # Any exceptions seeking to the end of the bundle at this point are
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
   413
            # almost certainly related to the underlying stream being bad.
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
   414
            # And, chances are that the exception we're handling is related to
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
   415
            # getting in that bad state. So, we swallow the seeking error and
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
   416
            # re-raise the original error.
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
   417
            seekerror = False
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
   418
            try:
34637
5f79f5f8487a bundle2: immediate exit for ctrl+c (issue5692)
Durham Goode <durham@fb.com>
parents: 34330
diff changeset
   419
                if self.current:
5f79f5f8487a bundle2: immediate exit for ctrl+c (issue5692)
Durham Goode <durham@fb.com>
parents: 34330
diff changeset
   420
                    # consume the part content to not corrupt the stream.
35135
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35134
diff changeset
   421
                    self.current.consume()
34637
5f79f5f8487a bundle2: immediate exit for ctrl+c (issue5692)
Durham Goode <durham@fb.com>
parents: 34330
diff changeset
   422
34164
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
   423
                for part in self.iterator:
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
   424
                    # consume the bundle content
35135
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35134
diff changeset
   425
                    part.consume()
34164
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
   426
            except Exception:
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
   427
                seekerror = True
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
   428
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
   429
            # Small hack to let caller code distinguish exceptions from bundle2
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
   430
            # processing from processing the old format. This is mostly needed
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
   431
            # to handle different return codes to unbundle according to the type
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
   432
            # of bundle. We should probably clean up or drop this return code
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
   433
            # craziness in a future version.
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
   434
            exc.duringunbundle2 = True
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
   435
            salvaged = []
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
   436
            replycaps = None
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
   437
            if self.op.reply is not None:
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
   438
                salvaged = self.op.reply.salvageoutput()
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
   439
                replycaps = self.op.reply.capabilities
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
   440
            exc._replycaps = replycaps
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
   441
            exc._bundle2salvagedoutput = salvaged
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
   442
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
   443
            # Re-raising from a variable loses the original stack. So only use
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
   444
            # that form if we need to.
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
   445
            if seekerror:
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
   446
                raise exc
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
   447
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
   448
        self.repo.ui.debug(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   449
            b'bundle2-input-bundle: %i parts total\n' % self.count
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
   450
        )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
   451
34162
e9e0e1143fc5 bundle2: move part iterator a separate class
Durham Goode <durham@fb.com>
parents: 34116
diff changeset
   452
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   453
def processbundle(repo, unbundler, transactiongetter=None, op=None, source=b''):
20889
deed5edb72de bundle2: first version of a bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20887
diff changeset
   454
    """This function process a bundle, apply effect to/from a repo
deed5edb72de bundle2: first version of a bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20887
diff changeset
   455
20947
c33d7bf53812 bundle2: feed a unbundle20 to the `processbundle` function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20892
diff changeset
   456
    It iterates over each part then searches for and uses the proper handling
c33d7bf53812 bundle2: feed a unbundle20 to the `processbundle` function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20892
diff changeset
   457
    code to process the part. Parts are processed in order.
20889
deed5edb72de bundle2: first version of a bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20887
diff changeset
   458
20891
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20890
diff changeset
   459
    Unknown Mandatory part will abort the process.
24851
df0ce98c882f bundle2: also save output when error happens during part processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24849
diff changeset
   460
df0ce98c882f bundle2: also save output when error happens during part processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24849
diff changeset
   461
    It is temporarily possible to provide a prebuilt bundleoperation to the
df0ce98c882f bundle2: also save output when error happens during part processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24849
diff changeset
   462
    function. This is used to ensure output is properly propagated in case of
df0ce98c882f bundle2: also save output when error happens during part processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24849
diff changeset
   463
    an error during the unbundling. This output capturing part will likely be
df0ce98c882f bundle2: also save output when error happens during part processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24849
diff changeset
   464
    reworked and this ability will probably go away in the process.
20889
deed5edb72de bundle2: first version of a bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20887
diff changeset
   465
    """
24851
df0ce98c882f bundle2: also save output when error happens during part processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24849
diff changeset
   466
    if op is None:
df0ce98c882f bundle2: also save output when error happens during part processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24849
diff changeset
   467
        if transactiongetter is None:
df0ce98c882f bundle2: also save output when error happens during part processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24849
diff changeset
   468
            transactiongetter = _notransaction
37239
f7d3915d5b3a bundleoperation: pass the source argument from all the users
Pulkit Goyal <7895pulkit@gmail.com>
parents: 37238
diff changeset
   469
        op = bundleoperation(repo, transactiongetter, source=source)
20889
deed5edb72de bundle2: first version of a bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20887
diff changeset
   470
    # todo:
deed5edb72de bundle2: first version of a bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20887
diff changeset
   471
    # - replace this is a init function soon.
deed5edb72de bundle2: first version of a bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20887
diff changeset
   472
    # - exception catching
deed5edb72de bundle2: first version of a bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20887
diff changeset
   473
    unbundler.params
25331
53ca47ef645d bundle2: add generic debug output regarding processed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25324
diff changeset
   474
    if repo.ui.debugflag:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   475
        msg = [b'bundle2-input-bundle:']
25331
53ca47ef645d bundle2: add generic debug output regarding processed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25324
diff changeset
   476
        if unbundler.params:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   477
            msg.append(b' %i params' % len(unbundler.params))
33791
119e1c6be1ce bundle2: fix transaction availability detection
Boris Feld <boris.feld@octobus.net>
parents: 33737
diff changeset
   478
        if op._gettransaction is None or op._gettransaction is _notransaction:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   479
            msg.append(b' no-transaction')
25331
53ca47ef645d bundle2: add generic debug output regarding processed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25324
diff changeset
   480
        else:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   481
            msg.append(b' with-transaction')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   482
        msg.append(b'\n')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   483
        repo.ui.debug(b''.join(msg))
34162
e9e0e1143fc5 bundle2: move part iterator a separate class
Durham Goode <durham@fb.com>
parents: 34116
diff changeset
   484
34268
f010097c885c bundle2: move part processing to a separate function
Durham Goode <durham@fb.com>
parents: 34267
diff changeset
   485
    processparts(repo, op, unbundler)
f010097c885c bundle2: move part processing to a separate function
Durham Goode <durham@fb.com>
parents: 34267
diff changeset
   486
f010097c885c bundle2: move part processing to a separate function
Durham Goode <durham@fb.com>
parents: 34267
diff changeset
   487
    return op
f010097c885c bundle2: move part processing to a separate function
Durham Goode <durham@fb.com>
parents: 34267
diff changeset
   488
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
   489
34268
f010097c885c bundle2: move part processing to a separate function
Durham Goode <durham@fb.com>
parents: 34267
diff changeset
   490
def processparts(repo, op, unbundler):
34164
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
   491
    with partiterator(repo, op, unbundler) as parts:
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
   492
        for part in parts:
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
   493
            _processpart(op, part)
25332
0cdd1c811705 bundle2: add generic debug output at the end of bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25331
diff changeset
   494
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
   495
33050
f0efd2bffe1e bundle: extract _processchangegroup() method
Martin von Zweigbergk <martinvonz@google.com>
parents: 33049
diff changeset
   496
def _processchangegroup(op, cg, tr, source, url, **kwargs):
33461
bb72031f0ea8 changegroup: stop returning and recording added nodes in 'cg.apply'
Boris Feld <boris.feld@octobus.net>
parents: 33460
diff changeset
   497
    ret = cg.apply(op.repo, tr, source, url, **kwargs)
45957
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45682
diff changeset
   498
    op.records.add(
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45682
diff changeset
   499
        b'changegroup',
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45682
diff changeset
   500
        {
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45682
diff changeset
   501
            b'return': ret,
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45682
diff changeset
   502
        },
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45682
diff changeset
   503
    )
33050
f0efd2bffe1e bundle: extract _processchangegroup() method
Martin von Zweigbergk <martinvonz@google.com>
parents: 33049
diff changeset
   504
    return ret
f0efd2bffe1e bundle: extract _processchangegroup() method
Martin von Zweigbergk <martinvonz@google.com>
parents: 33049
diff changeset
   505
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
   506
34266
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34265
diff changeset
   507
def _gethandler(op, part):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   508
    status = b'unknown'  # used by debug output
34266
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34265
diff changeset
   509
    try:
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34265
diff changeset
   510
        handler = parthandlermapping.get(part.type)
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34265
diff changeset
   511
        if handler is None:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   512
            status = b'unsupported-type'
34266
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34265
diff changeset
   513
            raise error.BundleUnknownFeatureError(parttype=part.type)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   514
        indebug(op.ui, b'found a handler for part %s' % part.type)
34266
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34265
diff changeset
   515
        unknownparams = part.mandatorykeys - handler.params
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34265
diff changeset
   516
        if unknownparams:
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34265
diff changeset
   517
            unknownparams = list(unknownparams)
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34265
diff changeset
   518
            unknownparams.sort()
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   519
            status = b'unsupported-params (%s)' % b', '.join(unknownparams)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
   520
            raise error.BundleUnknownFeatureError(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
   521
                parttype=part.type, params=unknownparams
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
   522
            )
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   523
        status = b'supported'
34266
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34265
diff changeset
   524
    except error.BundleUnknownFeatureError as exc:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
   525
        if part.mandatory:  # mandatory parts
34266
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34265
diff changeset
   526
            raise
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   527
        indebug(op.ui, b'ignoring unsupported advisory part %s' % exc)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
   528
        return  # skip to part processing
34266
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34265
diff changeset
   529
    finally:
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34265
diff changeset
   530
        if op.ui.debugflag:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   531
            msg = [b'bundle2-input-part: "%s"' % part.type]
34266
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34265
diff changeset
   532
            if not part.mandatory:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   533
                msg.append(b' (advisory)')
34266
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34265
diff changeset
   534
            nbmp = len(part.mandatorykeys)
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34265
diff changeset
   535
            nbap = len(part.params) - nbmp
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34265
diff changeset
   536
            if nbmp or nbap:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   537
                msg.append(b' (params:')
34266
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34265
diff changeset
   538
                if nbmp:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   539
                    msg.append(b' %i mandatory' % nbmp)
34266
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34265
diff changeset
   540
                if nbap:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   541
                    msg.append(b' %i advisory' % nbmp)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   542
                msg.append(b')')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   543
            msg.append(b' %s\n' % status)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   544
            op.ui.debug(b''.join(msg))
34266
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34265
diff changeset
   545
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34265
diff changeset
   546
    return handler
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34265
diff changeset
   547
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
   548
23008
d3137827016a bundle2: extract processing of part into its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23001
diff changeset
   549
def _processpart(op, part):
d3137827016a bundle2: extract processing of part into its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23001
diff changeset
   550
    """process a single part from a bundle
d3137827016a bundle2: extract processing of part into its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23001
diff changeset
   551
d3137827016a bundle2: extract processing of part into its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23001
diff changeset
   552
    The part is guaranteed to have been fully consumed when the function exits
d3137827016a bundle2: extract processing of part into its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23001
diff changeset
   553
    (even if an exception is raised)."""
34267
cc7b37c90616 bundle2: remove unnecessary try finally
Durham Goode <durham@fb.com>
parents: 34266
diff changeset
   554
    handler = _gethandler(op, part)
cc7b37c90616 bundle2: remove unnecessary try finally
Durham Goode <durham@fb.com>
parents: 34266
diff changeset
   555
    if handler is None:
cc7b37c90616 bundle2: remove unnecessary try finally
Durham Goode <durham@fb.com>
parents: 34266
diff changeset
   556
        return
23008
d3137827016a bundle2: extract processing of part into its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23001
diff changeset
   557
34267
cc7b37c90616 bundle2: remove unnecessary try finally
Durham Goode <durham@fb.com>
parents: 34266
diff changeset
   558
    # handler is called outside the above try block so that we don't
cc7b37c90616 bundle2: remove unnecessary try finally
Durham Goode <durham@fb.com>
parents: 34266
diff changeset
   559
    # risk catching KeyErrors from anything other than the
cc7b37c90616 bundle2: remove unnecessary try finally
Durham Goode <durham@fb.com>
parents: 34266
diff changeset
   560
    # parthandlermapping lookup (any KeyError raised by handler()
cc7b37c90616 bundle2: remove unnecessary try finally
Durham Goode <durham@fb.com>
parents: 34266
diff changeset
   561
    # itself represents a defect of a different variety).
cc7b37c90616 bundle2: remove unnecessary try finally
Durham Goode <durham@fb.com>
parents: 34266
diff changeset
   562
    output = None
cc7b37c90616 bundle2: remove unnecessary try finally
Durham Goode <durham@fb.com>
parents: 34266
diff changeset
   563
    if op.captureoutput and op.reply is not None:
cc7b37c90616 bundle2: remove unnecessary try finally
Durham Goode <durham@fb.com>
parents: 34266
diff changeset
   564
        op.ui.pushbuffer(error=True, subproc=True)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   565
        output = b''
34267
cc7b37c90616 bundle2: remove unnecessary try finally
Durham Goode <durham@fb.com>
parents: 34266
diff changeset
   566
    try:
cc7b37c90616 bundle2: remove unnecessary try finally
Durham Goode <durham@fb.com>
parents: 34266
diff changeset
   567
        handler(op, part)
23008
d3137827016a bundle2: extract processing of part into its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23001
diff changeset
   568
    finally:
34267
cc7b37c90616 bundle2: remove unnecessary try finally
Durham Goode <durham@fb.com>
parents: 34266
diff changeset
   569
        if output is not None:
cc7b37c90616 bundle2: remove unnecessary try finally
Durham Goode <durham@fb.com>
parents: 34266
diff changeset
   570
            output = op.ui.popbuffer()
cc7b37c90616 bundle2: remove unnecessary try finally
Durham Goode <durham@fb.com>
parents: 34266
diff changeset
   571
        if output:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   572
            outpart = op.reply.newpart(b'output', data=output, mandatory=False)
34267
cc7b37c90616 bundle2: remove unnecessary try finally
Durham Goode <durham@fb.com>
parents: 34266
diff changeset
   573
            outpart.addparam(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   574
                b'in-reply-to', pycompat.bytestr(part.id), mandatory=False
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
   575
            )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
   576
23008
d3137827016a bundle2: extract processing of part into its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23001
diff changeset
   577
21138
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
   578
def decodecaps(blob):
23139
e53f6b72a0e4 spelling: fixes from proofreading of spell checker issues
Mads Kiilerich <madski@unity3d.com>
parents: 23067
diff changeset
   579
    """decode a bundle2 caps bytes blob into a dictionary
21138
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
   580
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
   581
    The blob is a list of capabilities (one per line)
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
   582
    Capabilities may have values using a line of the form::
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
   583
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
   584
        capability=value1,value2,value3
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
   585
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
   586
    The values are always a list."""
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
   587
    caps = {}
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
   588
    for line in blob.splitlines():
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
   589
        if not line:
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
   590
            continue
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   591
        if b'=' not in line:
21138
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
   592
            key, vals = line, ()
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
   593
        else:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   594
            key, vals = line.split(b'=', 1)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   595
            vals = vals.split(b',')
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28672
diff changeset
   596
        key = urlreq.unquote(key)
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28672
diff changeset
   597
        vals = [urlreq.unquote(v) for v in vals]
21138
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
   598
        caps[key] = vals
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
   599
    return caps
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
   600
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
   601
21139
2b8c82f7f11d bundle2: capabilities encoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21138
diff changeset
   602
def encodecaps(caps):
2b8c82f7f11d bundle2: capabilities encoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21138
diff changeset
   603
    """encode a bundle2 caps dictionary into a bytes blob"""
2b8c82f7f11d bundle2: capabilities encoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21138
diff changeset
   604
    chunks = []
2b8c82f7f11d bundle2: capabilities encoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21138
diff changeset
   605
    for ca in sorted(caps):
2b8c82f7f11d bundle2: capabilities encoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21138
diff changeset
   606
        vals = caps[ca]
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28672
diff changeset
   607
        ca = urlreq.quote(ca)
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28672
diff changeset
   608
        vals = [urlreq.quote(v) for v in vals]
21139
2b8c82f7f11d bundle2: capabilities encoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21138
diff changeset
   609
        if vals:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   610
            ca = b"%s=%s" % (ca, b','.join(vals))
21139
2b8c82f7f11d bundle2: capabilities encoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21138
diff changeset
   611
        chunks.append(ca)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   612
    return b'\n'.join(chunks)
21139
2b8c82f7f11d bundle2: capabilities encoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21138
diff changeset
   613
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
   614
28666
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
   615
bundletypes = {
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   616
    b"": (b"", b'UN'),  # only when using unbundle on ssh and old http servers
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
   617
    # since the unification ssh accepts a header but there
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
   618
    # is no capability signaling it.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   619
    b"HG20": (),  # special-cased below
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   620
    b"HG10UN": (b"HG10UN", b'UN'),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   621
    b"HG10BZ": (b"HG10", b'BZ'),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   622
    b"HG10GZ": (b"HG10GZ", b'GZ'),
28666
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
   623
}
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
   624
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
   625
# hgweb uses this list to communicate its preferred type
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   626
bundlepriority = [b'HG10GZ', b'HG10BZ', b'HG10UN']
28666
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
   627
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
   628
20801
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
   629
class bundle20(object):
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
   630
    """represent an outgoing bundle2 container
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
   631
21599
57cd844d7a5b bundle2: have ``newpart`` automatically add the part to the bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21598
diff changeset
   632
    Use the `addparam` method to add stream level parameter. and `newpart` to
20856
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
   633
    populate it. Then call `getchunks` to retrieve all the binary chunks of
21024
7731a2281cf0 spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents: 21020
diff changeset
   634
    data that compose the bundle2 container."""
20801
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
   635
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   636
    _magicstring = b'HG20'
24640
685639f9430d bundle20: move magic string into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24071
diff changeset
   637
21134
2f8c4fa237f5 bundle2: adds a capabilities attribute on bundler20
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21133
diff changeset
   638
    def __init__(self, ui, capabilities=()):
20842
938718d72624 bundle2: print debug information during bundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20814
diff changeset
   639
        self.ui = ui
20801
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
   640
        self._params = []
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
   641
        self._parts = []
21136
b6fd496e5c72 bundle2: support for capabilities with values
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21135
diff changeset
   642
        self.capabilities = dict(capabilities)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   643
        self._compengine = util.compengines.forbundletype(b'UN')
30757
511a4bf52754 bundle2: allow compression options to be passed to compressor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30448
diff changeset
   644
        self._compopts = None
35787
a84dbc87dae9 exchange: send bundle2 stream clones uncompressed
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35783
diff changeset
   645
        # If compression is being handled by a consumer of the raw
a84dbc87dae9 exchange: send bundle2 stream clones uncompressed
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35783
diff changeset
   646
        # data (e.g. the wire protocol), unsetting this flag tells
a84dbc87dae9 exchange: send bundle2 stream clones uncompressed
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35783
diff changeset
   647
        # consumers that the bundle is best left uncompressed.
a84dbc87dae9 exchange: send bundle2 stream clones uncompressed
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35783
diff changeset
   648
        self.prefercompressed = True
26404
795f02a24b87 bundle2: allow compressed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26396
diff changeset
   649
30757
511a4bf52754 bundle2: allow compression options to be passed to compressor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30448
diff changeset
   650
    def setcompression(self, alg, compopts=None):
26404
795f02a24b87 bundle2: allow compressed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26396
diff changeset
   651
        """setup core part compression to <alg>"""
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   652
        if alg in (None, b'UN'):
26404
795f02a24b87 bundle2: allow compressed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26396
diff changeset
   653
            return
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   654
        assert not any(n.lower() == b'compression' for n, v in self._params)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   655
        self.addparam(b'Compression', alg)
30361
f81002f736d7 bundle2: use new compression engine API for compression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30177
diff changeset
   656
        self._compengine = util.compengines.forbundletype(alg)
30757
511a4bf52754 bundle2: allow compression options to be passed to compressor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30448
diff changeset
   657
        self._compopts = compopts
20801
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
   658
21900
b8bd97085ec9 bundle2: add a ``bundle20.nbparts`` property
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21660
diff changeset
   659
    @property
b8bd97085ec9 bundle2: add a ``bundle20.nbparts`` property
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21660
diff changeset
   660
    def nbparts(self):
b8bd97085ec9 bundle2: add a ``bundle20.nbparts`` property
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21660
diff changeset
   661
        """total number of parts added to the bundler"""
b8bd97085ec9 bundle2: add a ``bundle20.nbparts`` property
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21660
diff changeset
   662
        return len(self._parts)
b8bd97085ec9 bundle2: add a ``bundle20.nbparts`` property
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21660
diff changeset
   663
21597
1daad9dcdba2 bundle2: small doc update on the bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21186
diff changeset
   664
    # methods used to defines the bundle2 content
20804
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
   665
    def addparam(self, name, value=None):
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
   666
        """add a stream level parameter"""
20813
8c74b3ce5b70 bundle2: refuse empty parameter name
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20812
diff changeset
   667
        if not name:
38610
d474b3b44d4f bundle2: use ProgrammingError to report bad use of addparam()
Yuya Nishihara <yuya@tcha.org>
parents: 37837
diff changeset
   668
            raise error.ProgrammingError(b'empty parameter name')
43536
ea25b4673231 bundle2: more pytype suppressions around bytestring
Augie Fackler <augie@google.com>
parents: 43131
diff changeset
   669
        if name[0:1] not in pycompat.bytestr(
ea25b4673231 bundle2: more pytype suppressions around bytestring
Augie Fackler <augie@google.com>
parents: 43131
diff changeset
   670
            string.ascii_letters  # pytype: disable=wrong-arg-types
ea25b4673231 bundle2: more pytype suppressions around bytestring
Augie Fackler <augie@google.com>
parents: 43131
diff changeset
   671
        ):
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
   672
            raise error.ProgrammingError(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
   673
                b'non letter first character: %s' % name
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
   674
            )
20804
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
   675
        self._params.append((name, value))
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
   676
20856
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
   677
    def addpart(self, part):
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
   678
        """add a new part to the bundle2 container
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
   679
21024
7731a2281cf0 spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents: 21020
diff changeset
   680
        Parts contains the actual applicative payload."""
20995
e995d104c87f bundle2: add an integer id to part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20952
diff changeset
   681
        assert part.id is None
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
   682
        part.id = len(self._parts)  # very cheap counter
20856
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
   683
        self._parts.append(part)
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
   684
21598
1b0dbb91de5b bundle2: add a ``newpart`` method to ``bundle20``
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21597
diff changeset
   685
    def newpart(self, typeid, *args, **kwargs):
21602
cc33ae50bab3 bundle2: warn about error during initialization in ``newpart`` docstring
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21601
diff changeset
   686
        """create a new part and add it to the containers
cc33ae50bab3 bundle2: warn about error during initialization in ``newpart`` docstring
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21601
diff changeset
   687
cc33ae50bab3 bundle2: warn about error during initialization in ``newpart`` docstring
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21601
diff changeset
   688
        As the part is directly added to the containers. For now, this means
cc33ae50bab3 bundle2: warn about error during initialization in ``newpart`` docstring
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21601
diff changeset
   689
        that any failure to properly initialize the part after calling
cc33ae50bab3 bundle2: warn about error during initialization in ``newpart`` docstring
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21601
diff changeset
   690
        ``newpart`` should result in a failure of the whole bundling process.
cc33ae50bab3 bundle2: warn about error during initialization in ``newpart`` docstring
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21601
diff changeset
   691
cc33ae50bab3 bundle2: warn about error during initialization in ``newpart`` docstring
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21601
diff changeset
   692
        You can still fall back to manually create and add if you need better
cc33ae50bab3 bundle2: warn about error during initialization in ``newpart`` docstring
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21601
diff changeset
   693
        control."""
21598
1b0dbb91de5b bundle2: add a ``newpart`` method to ``bundle20``
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21597
diff changeset
   694
        part = bundlepart(typeid, *args, **kwargs)
21599
57cd844d7a5b bundle2: have ``newpart`` automatically add the part to the bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21598
diff changeset
   695
        self.addpart(part)
21598
1b0dbb91de5b bundle2: add a ``newpart`` method to ``bundle20``
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21597
diff changeset
   696
        return part
1b0dbb91de5b bundle2: add a ``newpart`` method to ``bundle20``
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21597
diff changeset
   697
21597
1daad9dcdba2 bundle2: small doc update on the bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21186
diff changeset
   698
    # methods used to generate the bundle2 stream
20801
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
   699
    def getchunks(self):
25322
1f7c0403f8be bundle2: add generic debug output regarding generated bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25321
diff changeset
   700
        if self.ui.debugflag:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   701
            msg = [b'bundle2-output-bundle: "%s",' % self._magicstring]
25322
1f7c0403f8be bundle2: add generic debug output regarding generated bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25321
diff changeset
   702
            if self._params:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   703
                msg.append(b' (%i params)' % len(self._params))
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   704
            msg.append(b' %i parts total\n' % len(self._parts))
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   705
            self.ui.debug(b''.join(msg))
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   706
        outdebug(self.ui, b'start emission of %s stream' % self._magicstring)
24640
685639f9430d bundle20: move magic string into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24071
diff changeset
   707
        yield self._magicstring
20804
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
   708
        param = self._paramchunk()
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   709
        outdebug(self.ui, b'bundle parameter: %s' % param)
20804
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
   710
        yield _pack(_fstreamparamsize, len(param))
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
   711
        if param:
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
   712
            yield param
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
   713
        for chunk in self._compengine.compressstream(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
   714
            self._getcorechunk(), self._compopts
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
   715
        ):
30367
5925bda42dbd bundle2: use compressstream compression engine API
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30363
diff changeset
   716
            yield chunk
20802
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
   717
20804
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
   718
    def _paramchunk(self):
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
   719
        """return a encoded version of all stream parameters"""
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
   720
        blocks = []
20809
b93bb639451a bundle2: support for bundling parameter value
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20808
diff changeset
   721
        for par, value in self._params:
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28672
diff changeset
   722
            par = urlreq.quote(par)
20809
b93bb639451a bundle2: support for bundling parameter value
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20808
diff changeset
   723
            if value is not None:
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28672
diff changeset
   724
                value = urlreq.quote(value)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   725
                par = b'%s=%s' % (par, value)
20809
b93bb639451a bundle2: support for bundling parameter value
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20808
diff changeset
   726
            blocks.append(par)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   727
        return b' '.join(blocks)
20804
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
   728
26396
d90c30801cdf bundle20: extract core payload generation in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26395
diff changeset
   729
    def _getcorechunk(self):
d90c30801cdf bundle20: extract core payload generation in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26395
diff changeset
   730
        """yield chunk for the core part of the bundle
d90c30801cdf bundle20: extract core payload generation in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26395
diff changeset
   731
d90c30801cdf bundle20: extract core payload generation in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26395
diff changeset
   732
        (all but headers and parameters)"""
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   733
        outdebug(self.ui, b'start of parts')
26396
d90c30801cdf bundle20: extract core payload generation in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26395
diff changeset
   734
        for part in self._parts:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   735
            outdebug(self.ui, b'bundle part: "%s"' % part.type)
26396
d90c30801cdf bundle20: extract core payload generation in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26395
diff changeset
   736
            for chunk in part.getchunks(ui=self.ui):
d90c30801cdf bundle20: extract core payload generation in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26395
diff changeset
   737
                yield chunk
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   738
        outdebug(self.ui, b'end of bundle')
26396
d90c30801cdf bundle20: extract core payload generation in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26395
diff changeset
   739
        yield _pack(_fpartheadersize, 0)
d90c30801cdf bundle20: extract core payload generation in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26395
diff changeset
   740
24794
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24793
diff changeset
   741
    def salvageoutput(self):
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24793
diff changeset
   742
        """return a list with a copy of all output parts in the bundle
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24793
diff changeset
   743
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24793
diff changeset
   744
        This is meant to be used during error handling to make sure we preserve
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24793
diff changeset
   745
        server output"""
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24793
diff changeset
   746
        salvaged = []
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24793
diff changeset
   747
        for part in self._parts:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   748
            if part.type.startswith(b'output'):
24794
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24793
diff changeset
   749
                salvaged.append(part.copy())
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24793
diff changeset
   750
        return salvaged
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24793
diff changeset
   751
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24793
diff changeset
   752
21013
a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21005
diff changeset
   753
class unpackermixin(object):
a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21005
diff changeset
   754
    """A mixin to extract bytes and struct data from a stream"""
20802
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
   755
21013
a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21005
diff changeset
   756
    def __init__(self, fp):
20802
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
   757
        self._fp = fp
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
   758
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
   759
    def _unpack(self, format):
31862
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31861
diff changeset
   760
        """unpack this struct format from the stream
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31861
diff changeset
   761
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31861
diff changeset
   762
        This method is meant for internal usage by the bundle2 protocol only.
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31861
diff changeset
   763
        They directly manipulate the low level stream including bundle2 level
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31861
diff changeset
   764
        instruction.
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31861
diff changeset
   765
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31861
diff changeset
   766
        Do not use it to implement higher-level logic or methods."""
20802
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
   767
        data = self._readexact(struct.calcsize(format))
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
   768
        return _unpack(format, data)
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
   769
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
   770
    def _readexact(self, size):
31862
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31861
diff changeset
   771
        """read exactly <size> bytes from the stream
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31861
diff changeset
   772
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31861
diff changeset
   773
        This method is meant for internal usage by the bundle2 protocol only.
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31861
diff changeset
   774
        They directly manipulate the low level stream including bundle2 level
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31861
diff changeset
   775
        instruction.
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31861
diff changeset
   776
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31861
diff changeset
   777
        Do not use it to implement higher-level logic or methods."""
20802
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
   778
        return changegroup.readexactly(self._fp, size)
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
   779
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
   780
25640
39f0064a3079 bundle2.getunbundler: rename "header" to "magicstring"
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25518
diff changeset
   781
def getunbundler(ui, fp, magicstring=None):
39f0064a3079 bundle2.getunbundler: rename "header" to "magicstring"
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25518
diff changeset
   782
    """return a valid unbundler object for a given magicstring"""
39f0064a3079 bundle2.getunbundler: rename "header" to "magicstring"
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25518
diff changeset
   783
    if magicstring is None:
39f0064a3079 bundle2.getunbundler: rename "header" to "magicstring"
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25518
diff changeset
   784
        magicstring = changegroup.readexactly(fp, 4)
39f0064a3079 bundle2.getunbundler: rename "header" to "magicstring"
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25518
diff changeset
   785
    magic, version = magicstring[0:2], magicstring[2:4]
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   786
    if magic != b'HG':
33134
0aae80d14fed bundle2: add some debugging information to the not-a-bundle error
Siddharth Agarwal <sid0@fb.com>
parents: 33056
diff changeset
   787
        ui.debug(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   788
            b"error: invalid magic: %r (version %r), should be 'HG'\n"
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
   789
            % (magic, version)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
   790
        )
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   791
        raise error.Abort(_(b'not a Mercurial bundle'))
24648
5cac3accdaa1 unbundle20: allow generic dispatch between unbundlers
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24642
diff changeset
   792
    unbundlerclass = formatmap.get(version)
5cac3accdaa1 unbundle20: allow generic dispatch between unbundlers
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24642
diff changeset
   793
    if unbundlerclass is None:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   794
        raise error.Abort(_(b'unknown bundle version %s') % version)
24648
5cac3accdaa1 unbundle20: allow generic dispatch between unbundlers
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24642
diff changeset
   795
    unbundler = unbundlerclass(ui, fp)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   796
    indebug(ui, b'start processing of %s stream' % magicstring)
24642
54e5c239c2d9 unbundle20: move header parsing into the 'getunbundler' function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24641
diff changeset
   797
    return unbundler
24641
60fecc5b14a4 unbundle20: retrieve unbundler instances through a factory function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24640
diff changeset
   798
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
   799
21013
a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21005
diff changeset
   800
class unbundle20(unpackermixin):
a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21005
diff changeset
   801
    """interpret a bundle2 stream
a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21005
diff changeset
   802
21129
07bcbf326c8d bundle2: use an official iterparts method to unbundle parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21066
diff changeset
   803
    This class is fed with a binary stream and yields parts through its
07bcbf326c8d bundle2: use an official iterparts method to unbundle parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21066
diff changeset
   804
    `iterparts` methods."""
21013
a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21005
diff changeset
   805
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   806
    _magicstring = b'HG20'
26542
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
   807
24642
54e5c239c2d9 unbundle20: move header parsing into the 'getunbundler' function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24641
diff changeset
   808
    def __init__(self, ui, fp):
21066
5ecfe76d0d96 bundle2: make header reading optional
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21062
diff changeset
   809
        """If header is specified, we do not read it out of the stream."""
21013
a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21005
diff changeset
   810
        self.ui = ui
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   811
        self._compengine = util.compengines.forbundletype(b'UN')
26802
42f705f2c02d bundle2: make unbundle.compressed return True when compressed
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26795
diff changeset
   812
        self._compressed = None
21013
a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21005
diff changeset
   813
        super(unbundle20, self).__init__(fp)
a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21005
diff changeset
   814
20802
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
   815
    @util.propertycache
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
   816
    def params(self):
21024
7731a2281cf0 spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents: 21020
diff changeset
   817
        """dictionary of stream level parameters"""
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   818
        indebug(self.ui, b'reading bundle2 stream parameters')
20805
c5aaeca0cfbf bundle2: support for unbundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20804
diff changeset
   819
        params = {}
c5aaeca0cfbf bundle2: support for unbundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20804
diff changeset
   820
        paramssize = self._unpack(_fstreamparamsize)[0]
23011
006a81d07e57 bundle2: detect and disallow a negative chunk size
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23010
diff changeset
   821
        if paramssize < 0:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
   822
            raise error.BundleValueError(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   823
                b'negative bundle param size: %i' % paramssize
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
   824
            )
20805
c5aaeca0cfbf bundle2: support for unbundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20804
diff changeset
   825
        if paramssize:
26541
d40029b4296e bundle2: split parameter retrieval and processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26404
diff changeset
   826
            params = self._readexact(paramssize)
d40029b4296e bundle2: split parameter retrieval and processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26404
diff changeset
   827
            params = self._processallparams(params)
20805
c5aaeca0cfbf bundle2: support for unbundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20804
diff changeset
   828
        return params
20802
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
   829
26541
d40029b4296e bundle2: split parameter retrieval and processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26404
diff changeset
   830
    def _processallparams(self, paramsblock):
d40029b4296e bundle2: split parameter retrieval and processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26404
diff changeset
   831
        """"""
29591
6215b5537ba5 bundle2: use a sorted dict for holding parameters
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29294
diff changeset
   832
        params = util.sortdict()
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   833
        for p in paramsblock.split(b' '):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   834
            p = p.split(b'=', 1)
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28672
diff changeset
   835
            p = [urlreq.unquote(i) for i in p]
26541
d40029b4296e bundle2: split parameter retrieval and processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26404
diff changeset
   836
            if len(p) < 2:
d40029b4296e bundle2: split parameter retrieval and processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26404
diff changeset
   837
                p.append(None)
d40029b4296e bundle2: split parameter retrieval and processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26404
diff changeset
   838
            self._processparam(*p)
d40029b4296e bundle2: split parameter retrieval and processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26404
diff changeset
   839
            params[p[0]] = p[1]
d40029b4296e bundle2: split parameter retrieval and processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26404
diff changeset
   840
        return params
d40029b4296e bundle2: split parameter retrieval and processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26404
diff changeset
   841
20844
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
   842
    def _processparam(self, name, value):
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
   843
        """process a parameter, applying its effect if needed
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
   844
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
   845
        Parameter starting with a lower case letter are advisory and will be
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
   846
        ignored when unknown.  Those starting with an upper case letter are
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
   847
        mandatory and will this function will raise a KeyError when unknown.
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
   848
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
   849
        Note: no option are currently supported. Any input will be either
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
   850
              ignored or failing.
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
   851
        """
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
   852
        if not name:
43554
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43543
diff changeset
   853
            raise ValueError('empty parameter name')
43536
ea25b4673231 bundle2: more pytype suppressions around bytestring
Augie Fackler <augie@google.com>
parents: 43131
diff changeset
   854
        if name[0:1] not in pycompat.bytestr(
ea25b4673231 bundle2: more pytype suppressions around bytestring
Augie Fackler <augie@google.com>
parents: 43131
diff changeset
   855
            string.ascii_letters  # pytype: disable=wrong-arg-types
ea25b4673231 bundle2: more pytype suppressions around bytestring
Augie Fackler <augie@google.com>
parents: 43131
diff changeset
   856
        ):
43554
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43543
diff changeset
   857
            raise ValueError('non letter first character: %s' % name)
26395
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26393
diff changeset
   858
        try:
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26393
diff changeset
   859
            handler = b2streamparamsmap[name.lower()]
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26393
diff changeset
   860
        except KeyError:
34284
a48b3db0baba bundle2: stop using %r to quote part names
Augie Fackler <augie@google.com>
parents: 34268
diff changeset
   861
            if name[0:1].islower():
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   862
                indebug(self.ui, b"ignoring unknown parameter %s" % name)
26395
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26393
diff changeset
   863
            else:
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26393
diff changeset
   864
                raise error.BundleUnknownFeatureError(params=(name,))
20844
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
   865
        else:
26395
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26393
diff changeset
   866
            handler(self, name, value)
20844
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
   867
26542
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
   868
    def _forwardchunks(self):
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
   869
        """utility to transfer a bundle2 as binary
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
   870
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
   871
        This is made necessary by the fact the 'getbundle' command over 'ssh'
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
   872
        have no way to know then the reply end, relying on the bundle to be
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
   873
        interpreted to know its end. This is terrible and we are sorry, but we
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
   874
        needed to move forward to get general delta enabled.
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
   875
        """
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
   876
        yield self._magicstring
43421
be384a2052aa py3: don't use bytes with vars() or __dict__
Martin von Zweigbergk <martinvonz@google.com>
parents: 43131
diff changeset
   877
        assert 'params' not in vars(self)
26542
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
   878
        paramssize = self._unpack(_fstreamparamsize)[0]
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
   879
        if paramssize < 0:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
   880
            raise error.BundleValueError(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   881
                b'negative bundle param size: %i' % paramssize
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
   882
            )
26542
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
   883
        if paramssize:
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
   884
            params = self._readexact(paramssize)
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
   885
            self._processallparams(params)
42143
29569f2db929 bundle2: handle compression in _forwardchunks
Joerg Sonnenberger <joerg@bec.de>
parents: 42057
diff changeset
   886
            # The payload itself is decompressed below, so drop
29569f2db929 bundle2: handle compression in _forwardchunks
Joerg Sonnenberger <joerg@bec.de>
parents: 42057
diff changeset
   887
            # the compression parameter passed down to compensate.
29569f2db929 bundle2: handle compression in _forwardchunks
Joerg Sonnenberger <joerg@bec.de>
parents: 42057
diff changeset
   888
            outparams = []
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   889
            for p in params.split(b' '):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   890
                k, v = p.split(b'=', 1)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   891
                if k.lower() != b'compression':
42143
29569f2db929 bundle2: handle compression in _forwardchunks
Joerg Sonnenberger <joerg@bec.de>
parents: 42057
diff changeset
   892
                    outparams.append(p)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   893
            outparams = b' '.join(outparams)
42143
29569f2db929 bundle2: handle compression in _forwardchunks
Joerg Sonnenberger <joerg@bec.de>
parents: 42057
diff changeset
   894
            yield _pack(_fstreamparamsize, len(outparams))
29569f2db929 bundle2: handle compression in _forwardchunks
Joerg Sonnenberger <joerg@bec.de>
parents: 42057
diff changeset
   895
            yield outparams
29569f2db929 bundle2: handle compression in _forwardchunks
Joerg Sonnenberger <joerg@bec.de>
parents: 42057
diff changeset
   896
        else:
29569f2db929 bundle2: handle compression in _forwardchunks
Joerg Sonnenberger <joerg@bec.de>
parents: 42057
diff changeset
   897
            yield _pack(_fstreamparamsize, paramssize)
26542
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
   898
        # From there, payload might need to be decompressed
30363
d045b4091197 bundle2: use compression engines API to obtain decompressor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30361
diff changeset
   899
        self._fp = self._compengine.decompressorreader(self._fp)
26542
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
   900
        emptycount = 0
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
   901
        while emptycount < 2:
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
   902
            # so we can brainlessly loop
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
   903
            assert _fpartheadersize == _fpayloadsize
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
   904
            size = self._unpack(_fpartheadersize)[0]
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
   905
            yield _pack(_fpartheadersize, size)
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
   906
            if size:
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
   907
                emptycount = 0
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
   908
            else:
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
   909
                emptycount += 1
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
   910
                continue
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
   911
            if size == flaginterrupt:
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
   912
                continue
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
   913
            elif size < 0:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   914
                raise error.BundleValueError(b'negative chunk size: %i')
26542
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
   915
            yield self._readexact(size)
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
   916
35137
da91e7309daf bundle2: don't use seekable bundle2 parts by default (issue5691)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35136
diff changeset
   917
    def iterparts(self, seekable=False):
20802
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
   918
        """yield all parts contained in the stream"""
35137
da91e7309daf bundle2: don't use seekable bundle2 parts by default (issue5691)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35136
diff changeset
   919
        cls = seekableunbundlepart if seekable else unbundlepart
20802
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
   920
        # make sure param have been loaded
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
   921
        self.params
26404
795f02a24b87 bundle2: allow compressed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26396
diff changeset
   922
        # From there, payload need to be decompressed
30363
d045b4091197 bundle2: use compression engines API to obtain decompressor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30361
diff changeset
   923
        self._fp = self._compengine.decompressorreader(self._fp)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   924
        indebug(self.ui, b'start extraction of bundle2 parts')
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
   925
        headerblock = self._readpartheader()
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
   926
        while headerblock is not None:
35137
da91e7309daf bundle2: don't use seekable bundle2 parts by default (issue5691)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35136
diff changeset
   927
            part = cls(self.ui, headerblock, self._fp)
20802
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
   928
            yield part
35135
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35134
diff changeset
   929
            # Ensure part is fully consumed so we can start reading the next
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35134
diff changeset
   930
            # part.
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35134
diff changeset
   931
            part.consume()
35136
2b72bc88043f bundle2: only seek to beginning of part in bundlerepo
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35135
diff changeset
   932
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
   933
            headerblock = self._readpartheader()
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   934
        indebug(self.ui, b'end of bundle2 stream')
20802
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
   935
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
   936
    def _readpartheader(self):
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
   937
        """reads a part header size and return the bytes blob
20864
9a75d2559cff bundle2: support unbundling empty part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20856
diff changeset
   938
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
   939
        returns None if empty"""
20864
9a75d2559cff bundle2: support unbundling empty part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20856
diff changeset
   940
        headersize = self._unpack(_fpartheadersize)[0]
23011
006a81d07e57 bundle2: detect and disallow a negative chunk size
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23010
diff changeset
   941
        if headersize < 0:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
   942
            raise error.BundleValueError(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   943
                b'negative part header size: %i' % headersize
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
   944
            )
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   945
        indebug(self.ui, b'part header size: %i' % headersize)
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
   946
        if headersize:
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
   947
            return self._readexact(headersize)
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
   948
        return None
20864
9a75d2559cff bundle2: support unbundling empty part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20856
diff changeset
   949
24071
184a2f6f40da bundle2.unbundle20: add compressed() method
Eric Sumner <ericsumner@fb.com>
parents: 24070
diff changeset
   950
    def compressed(self):
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
   951
        self.params  # load params
26802
42f705f2c02d bundle2: make unbundle.compressed return True when compressed
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26795
diff changeset
   952
        return self._compressed
20802
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
   953
31863
cd7aaf344d83 bundle2: move the 'close' method off the unpackermixin
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31862
diff changeset
   954
    def close(self):
cd7aaf344d83 bundle2: move the 'close' method off the unpackermixin
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31862
diff changeset
   955
        """close underlying file"""
43115
4aa72cdf616f py3: delete b'' prefix from safehasattr arguments
Martin von Zweigbergk <martinvonz@google.com>
parents: 43080
diff changeset
   956
        if util.safehasattr(self._fp, 'close'):
31863
cd7aaf344d83 bundle2: move the 'close' method off the unpackermixin
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31862
diff changeset
   957
            return self._fp.close()
cd7aaf344d83 bundle2: move the 'close' method off the unpackermixin
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31862
diff changeset
   958
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
   959
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   960
formatmap = {b'20': unbundle20}
24648
5cac3accdaa1 unbundle20: allow generic dispatch between unbundlers
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24642
diff changeset
   961
26395
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26393
diff changeset
   962
b2streamparamsmap = {}
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26393
diff changeset
   963
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
   964
26395
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26393
diff changeset
   965
def b2streamparamhandler(name):
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26393
diff changeset
   966
    """register a handler for a stream level parameter"""
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
   967
26395
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26393
diff changeset
   968
    def decorator(func):
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26393
diff changeset
   969
        assert name not in formatmap
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26393
diff changeset
   970
        b2streamparamsmap[name] = func
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26393
diff changeset
   971
        return func
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
   972
26395
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26393
diff changeset
   973
    return decorator
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26393
diff changeset
   974
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
   975
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   976
@b2streamparamhandler(b'compression')
26404
795f02a24b87 bundle2: allow compressed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26396
diff changeset
   977
def processcompression(unbundler, param, value):
795f02a24b87 bundle2: allow compressed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26396
diff changeset
   978
    """read compression parameter and install payload decompression"""
30363
d045b4091197 bundle2: use compression engines API to obtain decompressor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30361
diff changeset
   979
    if value not in util.compengines.supportedbundletypes:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
   980
        raise error.BundleUnknownFeatureError(params=(param,), values=(value,))
30363
d045b4091197 bundle2: use compression engines API to obtain decompressor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30361
diff changeset
   981
    unbundler._compengine = util.compengines.forbundletype(value)
26802
42f705f2c02d bundle2: make unbundle.compressed return True when compressed
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26795
diff changeset
   982
    if value is not None:
42f705f2c02d bundle2: make unbundle.compressed return True when compressed
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26795
diff changeset
   983
        unbundler._compressed = True
26404
795f02a24b87 bundle2: allow compressed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26396
diff changeset
   984
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
   985
21005
3d38ebb586fe bundle2: rename part to bundlepart
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21004
diff changeset
   986
class bundlepart(object):
20856
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
   987
    """A bundle2 part contains application level payload
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
   988
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
   989
    The part `type` is used to route the part to the application level
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
   990
    handler.
21604
c399bf961cb9 bundle2: the ability to set ``data`` attribute of the part is now official
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21603
diff changeset
   991
c399bf961cb9 bundle2: the ability to set ``data`` attribute of the part is now official
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21603
diff changeset
   992
    The part payload is contained in ``part.data``. It could be raw bytes or a
21605
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
   993
    generator of byte chunks.
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
   994
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
   995
    You can add parameters to the part using the ``addparam`` method.
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
   996
    Parameters can be either mandatory (default) or advisory. Remote side
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
   997
    should be able to safely ignore the advisory ones.
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
   998
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
   999
    Both data and parameters cannot be modified after the generation has begun.
20856
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
  1000
    """
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
  1001
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1002
    def __init__(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1003
        self,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1004
        parttype,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1005
        mandatoryparams=(),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1006
        advisoryparams=(),
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1007
        data=b'',
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1008
        mandatory=True,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1009
    ):
23868
405eacbecc52 bundle2: enforce parttype as alphanumerical
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23590
diff changeset
  1010
        validateparttype(parttype)
20995
e995d104c87f bundle2: add an integer id to part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20952
diff changeset
  1011
        self.id = None
20856
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
  1012
        self.type = parttype
21604
c399bf961cb9 bundle2: the ability to set ``data`` attribute of the part is now official
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21603
diff changeset
  1013
        self._data = data
21605
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
  1014
        self._mandatoryparams = list(mandatoryparams)
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
  1015
        self._advisoryparams = list(advisoryparams)
21607
054fa5176fa7 bundle2: forbid duplicate parameter keys
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21606
diff changeset
  1016
        # checking for duplicated entries
054fa5176fa7 bundle2: forbid duplicate parameter keys
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21606
diff changeset
  1017
        self._seenparams = set()
054fa5176fa7 bundle2: forbid duplicate parameter keys
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21606
diff changeset
  1018
        for pname, __ in self._mandatoryparams + self._advisoryparams:
054fa5176fa7 bundle2: forbid duplicate parameter keys
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21606
diff changeset
  1019
            if pname in self._seenparams:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1020
                raise error.ProgrammingError(b'duplicated params: %s' % pname)
21607
054fa5176fa7 bundle2: forbid duplicate parameter keys
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21606
diff changeset
  1021
            self._seenparams.add(pname)
21601
7ff01befc7ec bundle2: track life cycle of parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21600
diff changeset
  1022
        # status of the part's generation:
7ff01befc7ec bundle2: track life cycle of parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21600
diff changeset
  1023
        # - None: not started,
7ff01befc7ec bundle2: track life cycle of parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21600
diff changeset
  1024
        # - False: currently generated,
7ff01befc7ec bundle2: track life cycle of parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21600
diff changeset
  1025
        # - True: generation done.
7ff01befc7ec bundle2: track life cycle of parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21600
diff changeset
  1026
        self._generated = None
23590
4440c7cc3728 bundle2.bundlepart: make mandatory part flag explicit in API
Eric Sumner <ericsumner@fb.com>
parents: 23586
diff changeset
  1027
        self.mandatory = mandatory
20856
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
  1028
30906
1f51b4658f21 bundle2: implement a basic __repr__ for bundle2 part
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 30905
diff changeset
  1029
    def __repr__(self):
44173
74172a234dd3 py3: fully fix bundlepart.__repr__ to return str not bytes
Kyle Lippincott <spectral@google.com>
parents: 44164
diff changeset
  1030
        cls = "%s.%s" % (self.__class__.__module__, self.__class__.__name__)
74172a234dd3 py3: fully fix bundlepart.__repr__ to return str not bytes
Kyle Lippincott <spectral@google.com>
parents: 44164
diff changeset
  1031
        return '<%s object at %x; id: %s; type: %s; mandatory: %s>' % (
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1032
            cls,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1033
            id(self),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1034
            self.id,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1035
            self.type,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1036
            self.mandatory,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1037
        )
30906
1f51b4658f21 bundle2: implement a basic __repr__ for bundle2 part
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 30905
diff changeset
  1038
24793
2ec894582ea2 bundle2: add a 'copy' method on parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24743
diff changeset
  1039
    def copy(self):
2ec894582ea2 bundle2: add a 'copy' method on parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24743
diff changeset
  1040
        """return a copy of the part
2ec894582ea2 bundle2: add a 'copy' method on parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24743
diff changeset
  1041
2ec894582ea2 bundle2: add a 'copy' method on parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24743
diff changeset
  1042
        The new part have the very same content but no partid assigned yet.
2ec894582ea2 bundle2: add a 'copy' method on parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24743
diff changeset
  1043
        Parts with generated data cannot be copied."""
43115
4aa72cdf616f py3: delete b'' prefix from safehasattr arguments
Martin von Zweigbergk <martinvonz@google.com>
parents: 43080
diff changeset
  1044
        assert not util.safehasattr(self.data, 'next')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1045
        return self.__class__(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1046
            self.type,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1047
            self._mandatoryparams,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1048
            self._advisoryparams,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1049
            self._data,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1050
            self.mandatory,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1051
        )
24793
2ec894582ea2 bundle2: add a 'copy' method on parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24743
diff changeset
  1052
21604
c399bf961cb9 bundle2: the ability to set ``data`` attribute of the part is now official
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21603
diff changeset
  1053
    # methods used to defines the part content
27879
52a4ad62b006 cleanup: use modern @property/@foo.setter property specification
Augie Fackler <augie@google.com>
parents: 27751
diff changeset
  1054
    @property
52a4ad62b006 cleanup: use modern @property/@foo.setter property specification
Augie Fackler <augie@google.com>
parents: 27751
diff changeset
  1055
    def data(self):
52a4ad62b006 cleanup: use modern @property/@foo.setter property specification
Augie Fackler <augie@google.com>
parents: 27751
diff changeset
  1056
        return self._data
52a4ad62b006 cleanup: use modern @property/@foo.setter property specification
Augie Fackler <augie@google.com>
parents: 27751
diff changeset
  1057
52a4ad62b006 cleanup: use modern @property/@foo.setter property specification
Augie Fackler <augie@google.com>
parents: 27751
diff changeset
  1058
    @data.setter
52a4ad62b006 cleanup: use modern @property/@foo.setter property specification
Augie Fackler <augie@google.com>
parents: 27751
diff changeset
  1059
    def data(self, data):
21604
c399bf961cb9 bundle2: the ability to set ``data`` attribute of the part is now official
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21603
diff changeset
  1060
        if self._generated is not None:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1061
            raise error.ReadOnlyPartError(b'part is being generated')
21604
c399bf961cb9 bundle2: the ability to set ``data`` attribute of the part is now official
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21603
diff changeset
  1062
        self._data = data
c399bf961cb9 bundle2: the ability to set ``data`` attribute of the part is now official
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21603
diff changeset
  1063
21605
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
  1064
    @property
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
  1065
    def mandatoryparams(self):
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
  1066
        # make it an immutable tuple to force people through ``addparam``
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
  1067
        return tuple(self._mandatoryparams)
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
  1068
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
  1069
    @property
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
  1070
    def advisoryparams(self):
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
  1071
        # make it an immutable tuple to force people through ``addparam``
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
  1072
        return tuple(self._advisoryparams)
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
  1073
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1074
    def addparam(self, name, value=b'', mandatory=True):
31861
6d055cd6669f bundle2: add documention to 'part.addparams'
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31652
diff changeset
  1075
        """add a parameter to the part
6d055cd6669f bundle2: add documention to 'part.addparams'
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31652
diff changeset
  1076
6d055cd6669f bundle2: add documention to 'part.addparams'
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31652
diff changeset
  1077
        If 'mandatory' is set to True, the remote handler must claim support
6d055cd6669f bundle2: add documention to 'part.addparams'
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31652
diff changeset
  1078
        for this parameter or the unbundling will be aborted.
6d055cd6669f bundle2: add documention to 'part.addparams'
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31652
diff changeset
  1079
6d055cd6669f bundle2: add documention to 'part.addparams'
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31652
diff changeset
  1080
        The 'name' and 'value' cannot exceed 255 bytes each.
6d055cd6669f bundle2: add documention to 'part.addparams'
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31652
diff changeset
  1081
        """
21605
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
  1082
        if self._generated is not None:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1083
            raise error.ReadOnlyPartError(b'part is being generated')
21607
054fa5176fa7 bundle2: forbid duplicate parameter keys
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21606
diff changeset
  1084
        if name in self._seenparams:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1085
            raise ValueError(b'duplicated params: %s' % name)
21607
054fa5176fa7 bundle2: forbid duplicate parameter keys
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21606
diff changeset
  1086
        self._seenparams.add(name)
21605
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
  1087
        params = self._advisoryparams
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
  1088
        if mandatory:
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
  1089
            params = self._mandatoryparams
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
  1090
        params.append((name, value))
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
  1091
21601
7ff01befc7ec bundle2: track life cycle of parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21600
diff changeset
  1092
    # methods used to generates the bundle2 stream
25321
b44ee346211f bundle2: add debug output for part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25320
diff changeset
  1093
    def getchunks(self, ui):
21601
7ff01befc7ec bundle2: track life cycle of parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21600
diff changeset
  1094
        if self._generated is not None:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1095
            raise error.ProgrammingError(b'part can only be consumed once')
21601
7ff01befc7ec bundle2: track life cycle of parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21600
diff changeset
  1096
        self._generated = False
25323
21a25fb81d2c bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25322
diff changeset
  1097
21a25fb81d2c bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25322
diff changeset
  1098
        if ui.debugflag:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1099
            msg = [b'bundle2-output-part: "%s"' % self.type]
25323
21a25fb81d2c bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25322
diff changeset
  1100
            if not self.mandatory:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1101
                msg.append(b' (advisory)')
25323
21a25fb81d2c bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25322
diff changeset
  1102
            nbmp = len(self.mandatoryparams)
21a25fb81d2c bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25322
diff changeset
  1103
            nbap = len(self.advisoryparams)
21a25fb81d2c bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25322
diff changeset
  1104
            if nbmp or nbap:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1105
                msg.append(b' (params:')
25323
21a25fb81d2c bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25322
diff changeset
  1106
                if nbmp:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1107
                    msg.append(b' %i mandatory' % nbmp)
25323
21a25fb81d2c bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25322
diff changeset
  1108
                if nbap:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1109
                    msg.append(b' %i advisory' % nbmp)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1110
                msg.append(b')')
25323
21a25fb81d2c bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25322
diff changeset
  1111
            if not self.data:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1112
                msg.append(b' empty payload')
43115
4aa72cdf616f py3: delete b'' prefix from safehasattr arguments
Martin von Zweigbergk <martinvonz@google.com>
parents: 43080
diff changeset
  1113
            elif util.safehasattr(self.data, 'next') or util.safehasattr(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1114
                self.data, b'__next__'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1115
            ):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1116
                msg.append(b' streamed payload')
25323
21a25fb81d2c bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25322
diff changeset
  1117
            else:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1118
                msg.append(b' %i bytes payload' % len(self.data))
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1119
            msg.append(b'\n')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1120
            ui.debug(b''.join(msg))
25323
21a25fb81d2c bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25322
diff changeset
  1121
20877
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
  1122
        #### header
23590
4440c7cc3728 bundle2.bundlepart: make mandatory part flag explicit in API
Eric Sumner <ericsumner@fb.com>
parents: 23586
diff changeset
  1123
        if self.mandatory:
4440c7cc3728 bundle2.bundlepart: make mandatory part flag explicit in API
Eric Sumner <ericsumner@fb.com>
parents: 23586
diff changeset
  1124
            parttype = self.type.upper()
4440c7cc3728 bundle2.bundlepart: make mandatory part flag explicit in API
Eric Sumner <ericsumner@fb.com>
parents: 23586
diff changeset
  1125
        else:
4440c7cc3728 bundle2.bundlepart: make mandatory part flag explicit in API
Eric Sumner <ericsumner@fb.com>
parents: 23586
diff changeset
  1126
            parttype = self.type.lower()
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1127
        outdebug(ui, b'part %s: "%s"' % (pycompat.bytestr(self.id), parttype))
20877
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
  1128
        ## parttype
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1129
        header = [
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1130
            _pack(_fparttypesize, len(parttype)),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1131
            parttype,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1132
            _pack(_fpartid, self.id),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1133
        ]
20877
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
  1134
        ## parameters
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
  1135
        # count
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
  1136
        manpar = self.mandatoryparams
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
  1137
        advpar = self.advisoryparams
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
  1138
        header.append(_pack(_fpartparamcount, len(manpar), len(advpar)))
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
  1139
        # size
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
  1140
        parsizes = []
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
  1141
        for key, value in manpar:
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
  1142
            parsizes.append(len(key))
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
  1143
            parsizes.append(len(value))
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
  1144
        for key, value in advpar:
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
  1145
            parsizes.append(len(key))
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
  1146
            parsizes.append(len(value))
33698
da7c285ec6da bundle2: use modern Python division
Augie Fackler <augie@google.com>
parents: 33693
diff changeset
  1147
        paramsizes = _pack(_makefpartparamsizes(len(parsizes) // 2), *parsizes)
20877
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
  1148
        header.append(paramsizes)
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
  1149
        # key, value
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
  1150
        for key, value in manpar:
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
  1151
            header.append(key)
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
  1152
            header.append(value)
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
  1153
        for key, value in advpar:
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
  1154
            header.append(key)
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
  1155
            header.append(value)
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
  1156
        ## finalize header
34256
ab379eed2e31 bundle2: raise a more helpful error if building a bundle part header fails
Augie Fackler <raf@durin42.com>
parents: 34228
diff changeset
  1157
        try:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1158
            headerchunk = b''.join(header)
34256
ab379eed2e31 bundle2: raise a more helpful error if building a bundle part header fails
Augie Fackler <raf@durin42.com>
parents: 34228
diff changeset
  1159
        except TypeError:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1160
            raise TypeError(
43554
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43543
diff changeset
  1161
                'Found a non-bytes trying to '
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43543
diff changeset
  1162
                'build bundle part header: %r' % header
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1163
            )
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1164
        outdebug(ui, b'header chunk size: %i' % len(headerchunk))
20856
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
  1165
        yield _pack(_fpartheadersize, len(headerchunk))
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
  1166
        yield headerchunk
20877
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
  1167
        ## payload
23067
420a051616ce bundle2: transmit exception during part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23066
diff changeset
  1168
        try:
420a051616ce bundle2: transmit exception during part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23066
diff changeset
  1169
            for chunk in self._payloadchunks():
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1170
                outdebug(ui, b'payload chunk size: %i' % len(chunk))
23067
420a051616ce bundle2: transmit exception during part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23066
diff changeset
  1171
                yield _pack(_fpayloadsize, len(chunk))
420a051616ce bundle2: transmit exception during part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23066
diff changeset
  1172
                yield chunk
26144
4bc3707f3e67 bundle2: don't try to recover from a GeneratorExit (issue4785)
Augie Fackler <augie@google.com>
parents: 25919
diff changeset
  1173
        except GeneratorExit:
4bc3707f3e67 bundle2: don't try to recover from a GeneratorExit (issue4785)
Augie Fackler <augie@google.com>
parents: 25919
diff changeset
  1174
            # GeneratorExit means that nobody is listening for our
4bc3707f3e67 bundle2: don't try to recover from a GeneratorExit (issue4785)
Augie Fackler <augie@google.com>
parents: 25919
diff changeset
  1175
            # results anyway, so just bail quickly rather than trying
4bc3707f3e67 bundle2: don't try to recover from a GeneratorExit (issue4785)
Augie Fackler <augie@google.com>
parents: 25919
diff changeset
  1176
            # to produce an error part.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1177
            ui.debug(b'bundle2-generatorexit\n')
26144
4bc3707f3e67 bundle2: don't try to recover from a GeneratorExit (issue4785)
Augie Fackler <augie@google.com>
parents: 25919
diff changeset
  1178
            raise
25660
328739ea70c3 global: mass rewrite to use modern exception syntax
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25641
diff changeset
  1179
        except BaseException as exc:
37087
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module
Yuya Nishihara <yuya@tcha.org>
parents: 37008
diff changeset
  1180
            bexc = stringutil.forcebytestr(exc)
23067
420a051616ce bundle2: transmit exception during part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23066
diff changeset
  1181
            # backup exception data for later
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1182
            ui.debug(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1183
                b'bundle2-input-stream-interrupt: encoding exception %s' % bexc
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1184
            )
32232
76f9a0009b4b pycompat: extract helper to raise exception with traceback
Yuya Nishihara <yuya@tcha.org>
parents: 32024
diff changeset
  1185
            tb = sys.exc_info()[2]
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1186
            msg = b'unexpected error: %s' % bexc
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1187
            interpart = bundlepart(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1188
                b'error:abort', [(b'message', msg)], mandatory=False
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1189
            )
23067
420a051616ce bundle2: transmit exception during part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23066
diff changeset
  1190
            interpart.id = 0
420a051616ce bundle2: transmit exception during part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23066
diff changeset
  1191
            yield _pack(_fpayloadsize, -1)
25321
b44ee346211f bundle2: add debug output for part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25320
diff changeset
  1192
            for chunk in interpart.getchunks(ui=ui):
23067
420a051616ce bundle2: transmit exception during part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23066
diff changeset
  1193
                yield chunk
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1194
            outdebug(ui, b'closing payload chunk')
23067
420a051616ce bundle2: transmit exception during part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23066
diff changeset
  1195
            # abort current part payload
420a051616ce bundle2: transmit exception during part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23066
diff changeset
  1196
            yield _pack(_fpayloadsize, 0)
32232
76f9a0009b4b pycompat: extract helper to raise exception with traceback
Yuya Nishihara <yuya@tcha.org>
parents: 32024
diff changeset
  1197
            pycompat.raisewithtb(exc, tb)
21000
4cae06ae1562 bundle2: extract a _payloadchunks method for part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20998
diff changeset
  1198
        # end of payload
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1199
        outdebug(ui, b'closing payload chunk')
21000
4cae06ae1562 bundle2: extract a _payloadchunks method for part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20998
diff changeset
  1200
        yield _pack(_fpayloadsize, 0)
21601
7ff01befc7ec bundle2: track life cycle of parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21600
diff changeset
  1201
        self._generated = True
21000
4cae06ae1562 bundle2: extract a _payloadchunks method for part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20998
diff changeset
  1202
4cae06ae1562 bundle2: extract a _payloadchunks method for part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20998
diff changeset
  1203
    def _payloadchunks(self):
4cae06ae1562 bundle2: extract a _payloadchunks method for part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20998
diff changeset
  1204
        """yield chunks of a the part payload
4cae06ae1562 bundle2: extract a _payloadchunks method for part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20998
diff changeset
  1205
4cae06ae1562 bundle2: extract a _payloadchunks method for part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20998
diff changeset
  1206
        Exists to handle the different methods to provide data to a part."""
20876
ddd56f3eb786 bundle2: support for bundling and unbundling payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20864
diff changeset
  1207
        # we only support fixed size data now.
ddd56f3eb786 bundle2: support for bundling and unbundling payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20864
diff changeset
  1208
        # This will be improved in the future.
43115
4aa72cdf616f py3: delete b'' prefix from safehasattr arguments
Martin von Zweigbergk <martinvonz@google.com>
parents: 43080
diff changeset
  1209
        if util.safehasattr(self.data, 'next') or util.safehasattr(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1210
            self.data, b'__next__'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1211
        ):
21001
c93bb6a08fa1 bundle2: support chunk iterator as part data
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21000
diff changeset
  1212
            buff = util.chunkbuffer(self.data)
c93bb6a08fa1 bundle2: support chunk iterator as part data
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21000
diff changeset
  1213
            chunk = buff.read(preferedchunksize)
c93bb6a08fa1 bundle2: support chunk iterator as part data
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21000
diff changeset
  1214
            while chunk:
c93bb6a08fa1 bundle2: support chunk iterator as part data
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21000
diff changeset
  1215
                yield chunk
c93bb6a08fa1 bundle2: support chunk iterator as part data
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21000
diff changeset
  1216
                chunk = buff.read(preferedchunksize)
c93bb6a08fa1 bundle2: support chunk iterator as part data
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21000
diff changeset
  1217
        elif len(self.data):
20876
ddd56f3eb786 bundle2: support for bundling and unbundling payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20864
diff changeset
  1218
            yield self.data
20802
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
  1219
23066
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
  1220
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
  1221
flaginterrupt = -1
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
  1222
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1223
23066
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
  1224
class interrupthandler(unpackermixin):
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
  1225
    """read one part and process it with restricted capability
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
  1226
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
  1227
    This allows to transmit exception raised on the producer size during part
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
  1228
    iteration while the consumer is reading a part.
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
  1229
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
  1230
    Part processed in this manner only have access to a ui object,"""
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
  1231
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
  1232
    def __init__(self, ui, fp):
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
  1233
        super(interrupthandler, self).__init__(fp)
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
  1234
        self.ui = ui
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
  1235
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
  1236
    def _readpartheader(self):
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
  1237
        """reads a part header size and return the bytes blob
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
  1238
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
  1239
        returns None if empty"""
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
  1240
        headersize = self._unpack(_fpartheadersize)[0]
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
  1241
        if headersize < 0:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1242
            raise error.BundleValueError(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1243
                b'negative part header size: %i' % headersize
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1244
            )
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1245
        indebug(self.ui, b'part header size: %i\n' % headersize)
23066
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
  1246
        if headersize:
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
  1247
            return self._readexact(headersize)
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
  1248
        return None
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
  1249
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
  1250
    def __call__(self):
25335
8f7137a85a0e bundle2: add generic debug output regarding processed interruption
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25334
diff changeset
  1251
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1252
        self.ui.debug(
43117
8ff1ecfadcd1 cleanup: join string literals that are already on one line
Martin von Zweigbergk <martinvonz@google.com>
parents: 43115
diff changeset
  1253
            b'bundle2-input-stream-interrupt: opening out of band context\n'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1254
        )
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1255
        indebug(self.ui, b'bundle2 stream interruption, looking for a part.')
23066
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
  1256
        headerblock = self._readpartheader()
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
  1257
        if headerblock is None:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1258
            indebug(self.ui, b'no part found during interruption.')
23066
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
  1259
            return
35137
da91e7309daf bundle2: don't use seekable bundle2 parts by default (issue5691)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35136
diff changeset
  1260
        part = unbundlepart(self.ui, headerblock, self._fp)
23066
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
  1261
        op = interruptoperation(self.ui)
34265
e71890f27767 bundle2: move processpart stream maintenance into part iterator
Durham Goode <durham@fb.com>
parents: 34256
diff changeset
  1262
        hardabort = False
e71890f27767 bundle2: move processpart stream maintenance into part iterator
Durham Goode <durham@fb.com>
parents: 34256
diff changeset
  1263
        try:
e71890f27767 bundle2: move processpart stream maintenance into part iterator
Durham Goode <durham@fb.com>
parents: 34256
diff changeset
  1264
            _processpart(op, part)
e71890f27767 bundle2: move processpart stream maintenance into part iterator
Durham Goode <durham@fb.com>
parents: 34256
diff changeset
  1265
        except (SystemExit, KeyboardInterrupt):
e71890f27767 bundle2: move processpart stream maintenance into part iterator
Durham Goode <durham@fb.com>
parents: 34256
diff changeset
  1266
            hardabort = True
e71890f27767 bundle2: move processpart stream maintenance into part iterator
Durham Goode <durham@fb.com>
parents: 34256
diff changeset
  1267
            raise
e71890f27767 bundle2: move processpart stream maintenance into part iterator
Durham Goode <durham@fb.com>
parents: 34256
diff changeset
  1268
        finally:
e71890f27767 bundle2: move processpart stream maintenance into part iterator
Durham Goode <durham@fb.com>
parents: 34256
diff changeset
  1269
            if not hardabort:
35135
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35134
diff changeset
  1270
                part.consume()
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1271
        self.ui.debug(
43117
8ff1ecfadcd1 cleanup: join string literals that are already on one line
Martin von Zweigbergk <martinvonz@google.com>
parents: 43115
diff changeset
  1272
            b'bundle2-input-stream-interrupt: closing out of band context\n'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1273
        )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1274
23066
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
  1275
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
  1276
class interruptoperation(object):
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
  1277
    """A limited operation to be use by part handler during interruption
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
  1278
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
  1279
    It only have access to an ui object.
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
  1280
    """
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
  1281
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
  1282
    def __init__(self, ui):
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
  1283
        self.ui = ui
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
  1284
        self.reply = None
24878
e530cde6d115 bundle2: disable ouput capture unless we use http (issue4613 issue4615)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24851
diff changeset
  1285
        self.captureoutput = False
23066
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
  1286
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
  1287
    @property
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
  1288
    def repo(self):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1289
        raise error.ProgrammingError(b'no repo access from stream interruption')
23066
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
  1290
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
  1291
    def gettransaction(self):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1292
        raise TransactionUnavailable(b'no repo access from stream interruption')
23066
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
  1293
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1294
35134
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35133
diff changeset
  1295
def decodepayloadchunks(ui, fh):
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35133
diff changeset
  1296
    """Reads bundle2 part payload data into chunks.
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35133
diff changeset
  1297
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35133
diff changeset
  1298
    Part payload data consists of framed chunks. This function takes
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35133
diff changeset
  1299
    a file handle and emits those chunks.
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35133
diff changeset
  1300
    """
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1301
    dolog = ui.configbool(b'devel', b'bundle2.debug')
35138
589eed45a452 bundle2: inline debug logging
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35137
diff changeset
  1302
    debug = ui.debug
589eed45a452 bundle2: inline debug logging
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35137
diff changeset
  1303
35140
764e3ad1cf54 bundle2: inline struct operations
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35139
diff changeset
  1304
    headerstruct = struct.Struct(_fpayloadsize)
764e3ad1cf54 bundle2: inline struct operations
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35139
diff changeset
  1305
    headersize = headerstruct.size
764e3ad1cf54 bundle2: inline struct operations
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35139
diff changeset
  1306
    unpack = headerstruct.unpack
764e3ad1cf54 bundle2: inline struct operations
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35139
diff changeset
  1307
35134
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35133
diff changeset
  1308
    readexactly = changegroup.readexactly
35139
1fb0846ad792 bundle2: inline changegroup.readexactly()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35138
diff changeset
  1309
    read = fh.read
35134
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35133
diff changeset
  1310
35140
764e3ad1cf54 bundle2: inline struct operations
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35139
diff changeset
  1311
    chunksize = unpack(readexactly(fh, headersize))[0]
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1312
    indebug(ui, b'payload chunk size: %i' % chunksize)
35134
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35133
diff changeset
  1313
35139
1fb0846ad792 bundle2: inline changegroup.readexactly()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35138
diff changeset
  1314
    # changegroup.readexactly() is inlined below for performance.
35134
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35133
diff changeset
  1315
    while chunksize:
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35133
diff changeset
  1316
        if chunksize >= 0:
35139
1fb0846ad792 bundle2: inline changegroup.readexactly()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35138
diff changeset
  1317
            s = read(chunksize)
1fb0846ad792 bundle2: inline changegroup.readexactly()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35138
diff changeset
  1318
            if len(s) < chunksize:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1319
                raise error.Abort(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1320
                    _(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1321
                        b'stream ended unexpectedly '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1322
                        b' (got %d bytes, expected %d)'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1323
                    )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1324
                    % (len(s), chunksize)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1325
                )
35139
1fb0846ad792 bundle2: inline changegroup.readexactly()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35138
diff changeset
  1326
1fb0846ad792 bundle2: inline changegroup.readexactly()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35138
diff changeset
  1327
            yield s
35134
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35133
diff changeset
  1328
        elif chunksize == flaginterrupt:
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35133
diff changeset
  1329
            # Interrupt "signal" detected. The regular stream is interrupted
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35133
diff changeset
  1330
            # and a bundle2 part follows. Consume it.
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35133
diff changeset
  1331
            interrupthandler(ui, fh)()
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35133
diff changeset
  1332
        else:
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35133
diff changeset
  1333
            raise error.BundleValueError(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1334
                b'negative payload chunk size: %s' % chunksize
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1335
            )
35134
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35133
diff changeset
  1336
35139
1fb0846ad792 bundle2: inline changegroup.readexactly()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35138
diff changeset
  1337
        s = read(headersize)
1fb0846ad792 bundle2: inline changegroup.readexactly()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35138
diff changeset
  1338
        if len(s) < headersize:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1339
            raise error.Abort(
43117
8ff1ecfadcd1 cleanup: join string literals that are already on one line
Martin von Zweigbergk <martinvonz@google.com>
parents: 43115
diff changeset
  1340
                _(b'stream ended unexpectedly  (got %d bytes, expected %d)')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1341
                % (len(s), chunksize)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1342
            )
35139
1fb0846ad792 bundle2: inline changegroup.readexactly()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35138
diff changeset
  1343
35140
764e3ad1cf54 bundle2: inline struct operations
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35139
diff changeset
  1344
        chunksize = unpack(s)[0]
35138
589eed45a452 bundle2: inline debug logging
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35137
diff changeset
  1345
589eed45a452 bundle2: inline debug logging
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35137
diff changeset
  1346
        # indebug() inlined for performance.
589eed45a452 bundle2: inline debug logging
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35137
diff changeset
  1347
        if dolog:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1348
            debug(b'bundle2-input: payload chunk size: %i\n' % chunksize)
35134
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35133
diff changeset
  1349
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1350
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
  1351
class unbundlepart(unpackermixin):
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
  1352
    """a bundle part read from a bundle"""
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
  1353
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
  1354
    def __init__(self, ui, header, fp):
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
  1355
        super(unbundlepart, self).__init__(fp)
43115
4aa72cdf616f py3: delete b'' prefix from safehasattr arguments
Martin von Zweigbergk <martinvonz@google.com>
parents: 43080
diff changeset
  1356
        self._seekable = util.safehasattr(fp, 'seek') and util.safehasattr(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1357
            fp, b'tell'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1358
        )
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
  1359
        self.ui = ui
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
  1360
        # unbundle state attr
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
  1361
        self._headerdata = header
21015
14dd49260246 bundle2: move the fromheader closure into the class itself
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21014
diff changeset
  1362
        self._headeroffset = 0
21019
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
  1363
        self._initialized = False
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
  1364
        self.consumed = False
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
  1365
        # part data
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
  1366
        self.id = None
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
  1367
        self.type = None
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
  1368
        self.mandatoryparams = None
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
  1369
        self.advisoryparams = None
21610
d6056805f8f4 bundle2: introduce a ``params`` dictionary on unbundled parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21609
diff changeset
  1370
        self.params = None
21612
f221eb0531d9 bundle2: expose mandatory params in a mandatorykeys attribute
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21611
diff changeset
  1371
        self.mandatorykeys = ()
21019
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
  1372
        self._readheader()
23585
94b25d71dd0f bundle2.unbundlepart: decouple mandatory from parttype
Eric Sumner <ericsumner@fb.com>
parents: 23439
diff changeset
  1373
        self._mandatory = None
24036
c7601086338a bundle2.unbundlepart: tell() implementation
Eric Sumner <ericsumner@fb.com>
parents: 24035
diff changeset
  1374
        self._pos = 0
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
  1375
21015
14dd49260246 bundle2: move the fromheader closure into the class itself
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21014
diff changeset
  1376
    def _fromheader(self, size):
14dd49260246 bundle2: move the fromheader closure into the class itself
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21014
diff changeset
  1377
        """return the next <size> byte from the header"""
14dd49260246 bundle2: move the fromheader closure into the class itself
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21014
diff changeset
  1378
        offset = self._headeroffset
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1379
        data = self._headerdata[offset : (offset + size)]
21019
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
  1380
        self._headeroffset = offset + size
21015
14dd49260246 bundle2: move the fromheader closure into the class itself
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21014
diff changeset
  1381
        return data
14dd49260246 bundle2: move the fromheader closure into the class itself
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21014
diff changeset
  1382
21016
b477afb1c81e bundle2: move unpackheader closure into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21015
diff changeset
  1383
    def _unpackheader(self, format):
b477afb1c81e bundle2: move unpackheader closure into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21015
diff changeset
  1384
        """read given format from header
b477afb1c81e bundle2: move unpackheader closure into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21015
diff changeset
  1385
b477afb1c81e bundle2: move unpackheader closure into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21015
diff changeset
  1386
        This automatically compute the size of the format to read."""
b477afb1c81e bundle2: move unpackheader closure into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21015
diff changeset
  1387
        data = self._fromheader(struct.calcsize(format))
b477afb1c81e bundle2: move unpackheader closure into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21015
diff changeset
  1388
        return _unpack(format, data)
b477afb1c81e bundle2: move unpackheader closure into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21015
diff changeset
  1389
21608
3cb96ca90c17 bundle2: introduce an ``_initparams`` method
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21607
diff changeset
  1390
    def _initparams(self, mandatoryparams, advisoryparams):
3cb96ca90c17 bundle2: introduce an ``_initparams`` method
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21607
diff changeset
  1391
        """internal function to setup all logic related parameters"""
21609
63cc2594ef8a bundle2: make sure unbundled part param are read-only
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21608
diff changeset
  1392
        # make it read only to prevent people touching it by mistake.
63cc2594ef8a bundle2: make sure unbundled part param are read-only
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21608
diff changeset
  1393
        self.mandatoryparams = tuple(mandatoryparams)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1394
        self.advisoryparams = tuple(advisoryparams)
21610
d6056805f8f4 bundle2: introduce a ``params`` dictionary on unbundled parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21609
diff changeset
  1395
        # user friendly UI
29591
6215b5537ba5 bundle2: use a sorted dict for holding parameters
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29294
diff changeset
  1396
        self.params = util.sortdict(self.mandatoryparams)
6215b5537ba5 bundle2: use a sorted dict for holding parameters
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29294
diff changeset
  1397
        self.params.update(self.advisoryparams)
21612
f221eb0531d9 bundle2: expose mandatory params in a mandatorykeys attribute
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21611
diff changeset
  1398
        self.mandatorykeys = frozenset(p[0] for p in mandatoryparams)
21608
3cb96ca90c17 bundle2: introduce an ``_initparams`` method
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21607
diff changeset
  1399
21019
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
  1400
    def _readheader(self):
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
  1401
        """read the header and setup the object"""
21016
b477afb1c81e bundle2: move unpackheader closure into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21015
diff changeset
  1402
        typesize = self._unpackheader(_fparttypesize)[0]
21015
14dd49260246 bundle2: move the fromheader closure into the class itself
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21014
diff changeset
  1403
        self.type = self._fromheader(typesize)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1404
        indebug(self.ui, b'part type: "%s"' % self.type)
21016
b477afb1c81e bundle2: move unpackheader closure into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21015
diff changeset
  1405
        self.id = self._unpackheader(_fpartid)[0]
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1406
        indebug(self.ui, b'part id: "%s"' % pycompat.bytestr(self.id))
23585
94b25d71dd0f bundle2.unbundlepart: decouple mandatory from parttype
Eric Sumner <ericsumner@fb.com>
parents: 23439
diff changeset
  1407
        # extract mandatory bit from type
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1408
        self.mandatory = self.type != self.type.lower()
23585
94b25d71dd0f bundle2.unbundlepart: decouple mandatory from parttype
Eric Sumner <ericsumner@fb.com>
parents: 23439
diff changeset
  1409
        self.type = self.type.lower()
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
  1410
        ## reading parameters
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
  1411
        # param count
21016
b477afb1c81e bundle2: move unpackheader closure into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21015
diff changeset
  1412
        mancount, advcount = self._unpackheader(_fpartparamcount)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1413
        indebug(self.ui, b'part parameters: %i' % (mancount + advcount))
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
  1414
        # param size
21016
b477afb1c81e bundle2: move unpackheader closure into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21015
diff changeset
  1415
        fparamsizes = _makefpartparamsizes(mancount + advcount)
b477afb1c81e bundle2: move unpackheader closure into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21015
diff changeset
  1416
        paramsizes = self._unpackheader(fparamsizes)
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
  1417
        # make it a list of couple again
33700
9a323773216c bundle2: work around zip() being lazy in Python 3
Augie Fackler <augie@google.com>
parents: 33699
diff changeset
  1418
        paramsizes = list(zip(paramsizes[::2], paramsizes[1::2]))
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
  1419
        # split mandatory from advisory
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
  1420
        mansizes = paramsizes[:mancount]
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
  1421
        advsizes = paramsizes[mancount:]
23139
e53f6b72a0e4 spelling: fixes from proofreading of spell checker issues
Mads Kiilerich <madski@unity3d.com>
parents: 23067
diff changeset
  1422
        # retrieve param value
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
  1423
        manparams = []
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
  1424
        for key, value in mansizes:
21015
14dd49260246 bundle2: move the fromheader closure into the class itself
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21014
diff changeset
  1425
            manparams.append((self._fromheader(key), self._fromheader(value)))
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
  1426
        advparams = []
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
  1427
        for key, value in advsizes:
21015
14dd49260246 bundle2: move the fromheader closure into the class itself
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21014
diff changeset
  1428
            advparams.append((self._fromheader(key), self._fromheader(value)))
21608
3cb96ca90c17 bundle2: introduce an ``_initparams`` method
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21607
diff changeset
  1429
        self._initparams(manparams, advparams)
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
  1430
        ## part payload
24034
9881a1437799 bundle2.unbundlepart: raise payloadchunks from a closure to a method
Eric Sumner <ericsumner@fb.com>
parents: 24026
diff changeset
  1431
        self._payloadstream = util.chunkbuffer(self._payloadchunks())
21019
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
  1432
        # we read the data, tell it
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
  1433
        self._initialized = True
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
  1434
35134
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35133
diff changeset
  1435
    def _payloadchunks(self):
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35133
diff changeset
  1436
        """Generator of decoded chunks in the payload."""
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35133
diff changeset
  1437
        return decodepayloadchunks(self.ui, self._fp)
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35133
diff changeset
  1438
35135
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35134
diff changeset
  1439
    def consume(self):
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35134
diff changeset
  1440
        """Read the part payload until completion.
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35134
diff changeset
  1441
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35134
diff changeset
  1442
        By consuming the part data, the underlying stream read offset will
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35134
diff changeset
  1443
        be advanced to the next part (or end of stream).
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35134
diff changeset
  1444
        """
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35134
diff changeset
  1445
        if self.consumed:
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35134
diff changeset
  1446
            return
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35134
diff changeset
  1447
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35134
diff changeset
  1448
        chunk = self.read(32768)
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35134
diff changeset
  1449
        while chunk:
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35134
diff changeset
  1450
            self._pos += len(chunk)
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35134
diff changeset
  1451
            chunk = self.read(32768)
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35134
diff changeset
  1452
21019
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
  1453
    def read(self, size=None):
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
  1454
        """read payload data"""
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
  1455
        if not self._initialized:
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
  1456
            self._readheader()
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
  1457
        if size is None:
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
  1458
            data = self._payloadstream.read()
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
  1459
        else:
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
  1460
            data = self._payloadstream.read(size)
25334
76cba1ecf55f bundle2: add generic debug output regarding processed part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25333
diff changeset
  1461
        self._pos += len(data)
21019
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
  1462
        if size is None or len(data) < size:
25334
76cba1ecf55f bundle2: add generic debug output regarding processed part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25333
diff changeset
  1463
            if not self.consumed and self._pos:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1464
                self.ui.debug(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1465
                    b'bundle2-input-part: total payload size %i\n' % self._pos
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1466
                )
21019
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
  1467
            self.consumed = True
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
  1468
        return data
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
  1469
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1470
35133
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
  1471
class seekableunbundlepart(unbundlepart):
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
  1472
    """A bundle2 part in a bundle that is seekable.
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
  1473
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
  1474
    Regular ``unbundlepart`` instances can only be read once. This class
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
  1475
    extends ``unbundlepart`` to enable bi-directional seeking within the
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
  1476
    part.
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
  1477
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
  1478
    Bundle2 part data consists of framed chunks. Offsets when seeking
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
  1479
    refer to the decoded data, not the offsets in the underlying bundle2
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
  1480
    stream.
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
  1481
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
  1482
    To facilitate quickly seeking within the decoded data, instances of this
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
  1483
    class maintain a mapping between offsets in the underlying stream and
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
  1484
    the decoded payload. This mapping will consume memory in proportion
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
  1485
    to the number of chunks within the payload (which almost certainly
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
  1486
    increases in proportion with the size of the part).
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
  1487
    """
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1488
35133
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
  1489
    def __init__(self, ui, header, fp):
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
  1490
        # (payload, file) offsets for chunk starts.
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
  1491
        self._chunkindex = []
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
  1492
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
  1493
        super(seekableunbundlepart, self).__init__(ui, header, fp)
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
  1494
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
  1495
    def _payloadchunks(self, chunknum=0):
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
  1496
        '''seek to specified chunk and start yielding data'''
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
  1497
        if len(self._chunkindex) == 0:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1498
            assert chunknum == 0, b'Must start with chunk 0'
35133
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
  1499
            self._chunkindex.append((0, self._tellfp()))
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
  1500
        else:
41768
aaad36b88298 cleanup: use () to wrap long lines instead of \
Augie Fackler <augie@google.com>
parents: 40416
diff changeset
  1501
            assert chunknum < len(self._chunkindex), (
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1502
                b'Unknown chunk %d' % chunknum
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1503
            )
35133
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
  1504
            self._seekfp(self._chunkindex[chunknum][1])
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
  1505
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
  1506
        pos = self._chunkindex[chunknum][0]
35134
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35133
diff changeset
  1507
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35133
diff changeset
  1508
        for chunk in decodepayloadchunks(self.ui, self._fp):
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35133
diff changeset
  1509
            chunknum += 1
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35133
diff changeset
  1510
            pos += len(chunk)
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35133
diff changeset
  1511
            if chunknum == len(self._chunkindex):
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35133
diff changeset
  1512
                self._chunkindex.append((pos, self._tellfp()))
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35133
diff changeset
  1513
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35133
diff changeset
  1514
            yield chunk
35133
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
  1515
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
  1516
    def _findchunk(self, pos):
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
  1517
        '''for a given payload position, return a chunk number and offset'''
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
  1518
        for chunk, (ppos, fpos) in enumerate(self._chunkindex):
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
  1519
            if ppos == pos:
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
  1520
                return chunk, 0
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
  1521
            elif ppos > pos:
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
  1522
                return chunk - 1, pos - self._chunkindex[chunk - 1][0]
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1523
        raise ValueError(b'Unknown chunk')
35133
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
  1524
24036
c7601086338a bundle2.unbundlepart: tell() implementation
Eric Sumner <ericsumner@fb.com>
parents: 24035
diff changeset
  1525
    def tell(self):
c7601086338a bundle2.unbundlepart: tell() implementation
Eric Sumner <ericsumner@fb.com>
parents: 24035
diff changeset
  1526
        return self._pos
c7601086338a bundle2.unbundlepart: tell() implementation
Eric Sumner <ericsumner@fb.com>
parents: 24035
diff changeset
  1527
35069
241d9caca11e bundle2: use os.SEEK_* constants
Gregory Szorc <gregory.szorc@gmail.com>
parents: 34820
diff changeset
  1528
    def seek(self, offset, whence=os.SEEK_SET):
241d9caca11e bundle2: use os.SEEK_* constants
Gregory Szorc <gregory.szorc@gmail.com>
parents: 34820
diff changeset
  1529
        if whence == os.SEEK_SET:
24037
f0b498cfc5c8 bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents: 24036
diff changeset
  1530
            newpos = offset
35069
241d9caca11e bundle2: use os.SEEK_* constants
Gregory Szorc <gregory.szorc@gmail.com>
parents: 34820
diff changeset
  1531
        elif whence == os.SEEK_CUR:
24037
f0b498cfc5c8 bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents: 24036
diff changeset
  1532
            newpos = self._pos + offset
35069
241d9caca11e bundle2: use os.SEEK_* constants
Gregory Szorc <gregory.szorc@gmail.com>
parents: 34820
diff changeset
  1533
        elif whence == os.SEEK_END:
24037
f0b498cfc5c8 bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents: 24036
diff changeset
  1534
            if not self.consumed:
35141
699b2a759319 bundle2: avoid unbound read when seeking
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35140
diff changeset
  1535
                # Can't use self.consume() here because it advances self._pos.
699b2a759319 bundle2: avoid unbound read when seeking
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35140
diff changeset
  1536
                chunk = self.read(32768)
699b2a759319 bundle2: avoid unbound read when seeking
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35140
diff changeset
  1537
                while chunk:
699b2a759319 bundle2: avoid unbound read when seeking
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35140
diff changeset
  1538
                    chunk = self.read(32768)
24037
f0b498cfc5c8 bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents: 24036
diff changeset
  1539
            newpos = self._chunkindex[-1][0] - offset
f0b498cfc5c8 bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents: 24036
diff changeset
  1540
        else:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1541
            raise ValueError(b'Unknown whence value: %r' % (whence,))
24037
f0b498cfc5c8 bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents: 24036
diff changeset
  1542
f0b498cfc5c8 bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents: 24036
diff changeset
  1543
        if newpos > self._chunkindex[-1][0] and not self.consumed:
35141
699b2a759319 bundle2: avoid unbound read when seeking
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35140
diff changeset
  1544
            # Can't use self.consume() here because it advances self._pos.
699b2a759319 bundle2: avoid unbound read when seeking
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35140
diff changeset
  1545
            chunk = self.read(32768)
699b2a759319 bundle2: avoid unbound read when seeking
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35140
diff changeset
  1546
            while chunk:
699b2a759319 bundle2: avoid unbound read when seeking
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35140
diff changeset
  1547
                chunk = self.read(32668)
699b2a759319 bundle2: avoid unbound read when seeking
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35140
diff changeset
  1548
24037
f0b498cfc5c8 bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents: 24036
diff changeset
  1549
        if not 0 <= newpos <= self._chunkindex[-1][0]:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1550
            raise ValueError(b'Offset out of range')
24037
f0b498cfc5c8 bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents: 24036
diff changeset
  1551
f0b498cfc5c8 bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents: 24036
diff changeset
  1552
        if self._pos != newpos:
f0b498cfc5c8 bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents: 24036
diff changeset
  1553
            chunk, internaloffset = self._findchunk(newpos)
f0b498cfc5c8 bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents: 24036
diff changeset
  1554
            self._payloadstream = util.chunkbuffer(self._payloadchunks(chunk))
f0b498cfc5c8 bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents: 24036
diff changeset
  1555
            adjust = self.read(internaloffset)
f0b498cfc5c8 bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents: 24036
diff changeset
  1556
            if len(adjust) != internaloffset:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1557
                raise error.Abort(_(b'Seek failed\n'))
24037
f0b498cfc5c8 bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents: 24036
diff changeset
  1558
            self._pos = newpos
f0b498cfc5c8 bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents: 24036
diff changeset
  1559
31889
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
  1560
    def _seekfp(self, offset, whence=0):
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
  1561
        """move the underlying file pointer
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
  1562
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
  1563
        This method is meant for internal usage by the bundle2 protocol only.
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
  1564
        They directly manipulate the low level stream including bundle2 level
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
  1565
        instruction.
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
  1566
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
  1567
        Do not use it to implement higher-level logic or methods."""
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
  1568
        if self._seekable:
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
  1569
            return self._fp.seek(offset, whence)
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
  1570
        else:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1571
            raise NotImplementedError(_(b'File pointer is not seekable'))
31889
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
  1572
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
  1573
    def _tellfp(self):
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
  1574
        """return the file offset, or None if file is not seekable
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
  1575
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
  1576
        This method is meant for internal usage by the bundle2 protocol only.
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
  1577
        They directly manipulate the low level stream including bundle2 level
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
  1578
        instruction.
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
  1579
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
  1580
        Do not use it to implement higher-level logic or methods."""
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
  1581
        if self._seekable:
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
  1582
            try:
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
  1583
                return self._fp.tell()
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
  1584
            except IOError as e:
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
  1585
                if e.errno == errno.ESPIPE:
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
  1586
                    self._seekable = False
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
  1587
                else:
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
  1588
                    raise
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
  1589
        return None
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
  1590
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1591
25317
5a5b7046d00e bundle2: add an informative comment to the capability dict
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25315
diff changeset
  1592
# These are only the static capabilities.
5a5b7046d00e bundle2: add an informative comment to the capability dict
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25315
diff changeset
  1593
# Check the 'getrepocaps' function for the rest.
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1594
capabilities = {
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1595
    b'HG20': (),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1596
    b'bookmarks': (),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1597
    b'error': (b'abort', b'unsupportedcontent', b'pushraced', b'pushkey'),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1598
    b'listkeys': (),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1599
    b'pushkey': (),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1600
    b'digests': tuple(sorted(util.DIGESTS.keys())),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1601
    b'remote-changegroup': (b'http', b'https'),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1602
    b'hgtagsfnodes': (),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1603
    b'phases': (b'heads',),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1604
    b'stream': (b'v2',),
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1605
}
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1606
22341
2d16b39601b5 obsmarker: move bundle2caps from the localrepo class to the bundle2 module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22340
diff changeset
  1607
35783
c97639ad6874 bundle2: specify what capabilities will be used for
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35763
diff changeset
  1608
def getrepocaps(repo, allowpushback=False, role=None):
22342
262c5cc126c1 bundle2: introduce a `getrepocaps` to retrieve the bundle2 caps of a repo
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22341
diff changeset
  1609
    """return the bundle2 capabilities for a given repo
262c5cc126c1 bundle2: introduce a `getrepocaps` to retrieve the bundle2 caps of a repo
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22341
diff changeset
  1610
22343
76677a2c1cfd bundle2: advertise the obsmarker part in bundle2 capabilities
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22342
diff changeset
  1611
    Exists to allow extensions (like evolution) to mutate the capabilities.
35783
c97639ad6874 bundle2: specify what capabilities will be used for
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35763
diff changeset
  1612
c97639ad6874 bundle2: specify what capabilities will be used for
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35763
diff changeset
  1613
    The returned value is used for servers advertising their capabilities as
c97639ad6874 bundle2: specify what capabilities will be used for
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35763
diff changeset
  1614
    well as clients advertising their capabilities to servers as part of
c97639ad6874 bundle2: specify what capabilities will be used for
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35763
diff changeset
  1615
    bundle2 requests. The ``role`` argument specifies which is which.
22342
262c5cc126c1 bundle2: introduce a `getrepocaps` to retrieve the bundle2 caps of a repo
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22341
diff changeset
  1616
    """
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1617
    if role not in (b'client', b'server'):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1618
        raise error.ProgrammingError(b'role argument must be client or server')
35783
c97639ad6874 bundle2: specify what capabilities will be used for
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35763
diff changeset
  1619
22343
76677a2c1cfd bundle2: advertise the obsmarker part in bundle2 capabilities
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22342
diff changeset
  1620
    caps = capabilities.copy()
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1621
    caps[b'changegroup'] = tuple(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1622
        sorted(changegroup.supportedincomingversions(repo))
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1623
    )
22953
b1d694d3975e obsolete: add exchange option
Durham Goode <durham@fb.com>
parents: 22661
diff changeset
  1624
    if obsolete.isenabled(repo, obsolete.exchangeopt):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1625
        supportedformat = tuple(b'V%i' % v for v in obsolete.formats)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1626
        caps[b'obsmarkers'] = supportedformat
23439
743736fc7c41 bundle2-push: provide transaction to reply unbundler
Eric Sumner <ericsumner@fb.com>
parents: 23438
diff changeset
  1627
    if allowpushback:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1628
        caps[b'pushback'] = ()
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1629
    cpmode = repo.ui.config(b'server', b'concurrent-push-mode')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1630
    if cpmode == b'check-related':
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1631
        caps[b'checkheads'] = (b'related',)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1632
    if b'phases' in repo.ui.configlist(b'devel', b'legacy.exchange'):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1633
        caps.pop(b'phases')
35790
c1b9eb15a51c bundle2: don't advertise stream bundle2 capability when feature disabled
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35788
diff changeset
  1634
c1b9eb15a51c bundle2: don't advertise stream bundle2 capability when feature disabled
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35788
diff changeset
  1635
    # Don't advertise stream clone support in server mode if not configured.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1636
    if role == b'server':
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1637
        streamsupported = repo.ui.configbool(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1638
            b'server', b'uncompressed', untrusted=True
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1639
        )
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1640
        featuresupported = repo.ui.configbool(b'server', b'bundle2.stream')
35790
c1b9eb15a51c bundle2: don't advertise stream bundle2 capability when feature disabled
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35788
diff changeset
  1641
c1b9eb15a51c bundle2: don't advertise stream bundle2 capability when feature disabled
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35788
diff changeset
  1642
        if not streamsupported or not featuresupported:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1643
            caps.pop(b'stream')
35792
1d118f9f4f57 bundle2: always advertise client support for stream parts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35790
diff changeset
  1644
    # Else always advertise support on client, because payload support
1d118f9f4f57 bundle2: always advertise client support for stream parts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35790
diff changeset
  1645
    # should always be advertised.
35790
c1b9eb15a51c bundle2: don't advertise stream bundle2 capability when feature disabled
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35788
diff changeset
  1646
46672
7015b0232c5e exchange: stop advertising rev-branch-cache bundle capability
Joerg Sonnenberger <joerg@bec.de>
parents: 46571
diff changeset
  1647
    # b'rev-branch-cache is no longer advertised, but still supported
7015b0232c5e exchange: stop advertising rev-branch-cache bundle capability
Joerg Sonnenberger <joerg@bec.de>
parents: 46571
diff changeset
  1648
    # for legacy clients.
7015b0232c5e exchange: stop advertising rev-branch-cache bundle capability
Joerg Sonnenberger <joerg@bec.de>
parents: 46571
diff changeset
  1649
22343
76677a2c1cfd bundle2: advertise the obsmarker part in bundle2 capabilities
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22342
diff changeset
  1650
    return caps
22342
262c5cc126c1 bundle2: introduce a `getrepocaps` to retrieve the bundle2 caps of a repo
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22341
diff changeset
  1651
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1652
21644
17755dd8c509 bundle2: introduce a bundle2caps function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21628
diff changeset
  1653
def bundle2caps(remote):
23139
e53f6b72a0e4 spelling: fixes from proofreading of spell checker issues
Mads Kiilerich <madski@unity3d.com>
parents: 23067
diff changeset
  1654
    """return the bundle capabilities of a peer as dict"""
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1655
    raw = remote.capable(b'bundle2')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1656
    if not raw and raw != b'':
21644
17755dd8c509 bundle2: introduce a bundle2caps function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21628
diff changeset
  1657
        return {}
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1658
    capsblob = urlreq.unquote(remote.capable(b'bundle2'))
21644
17755dd8c509 bundle2: introduce a bundle2caps function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21628
diff changeset
  1659
    return decodecaps(capsblob)
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
  1660
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1661
22344
9829b7948100 bundle2: add a `obsmarkersversion` function to extract supported version
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22343
diff changeset
  1662
def obsmarkersversion(caps):
45957
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45682
diff changeset
  1663
    """extract the list of supported obsmarkers versions from a bundle2caps dict"""
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1664
    obscaps = caps.get(b'obsmarkers', ())
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1665
    return [int(c[1:]) for c in obscaps if c.startswith(b'V')]
22344
9829b7948100 bundle2: add a `obsmarkersversion` function to extract supported version
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22343
diff changeset
  1666
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1667
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1668
def writenewbundle(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1669
    ui,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1670
    repo,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1671
    source,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1672
    filename,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1673
    bundletype,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1674
    outgoing,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1675
    opts,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1676
    vfs=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1677
    compression=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1678
    compopts=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1679
):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1680
    if bundletype.startswith(b'HG10'):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1681
        cg = changegroup.makechangegroup(repo, outgoing, b'01', source)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1682
        return writebundle(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1683
            ui,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1684
            cg,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1685
            filename,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1686
            bundletype,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1687
            vfs=vfs,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1688
            compression=compression,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1689
            compopts=compopts,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1690
        )
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1691
    elif not bundletype.startswith(b'HG20'):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1692
        raise error.ProgrammingError(b'unknown bundle type: %s' % bundletype)
32261
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32232
diff changeset
  1693
32549
37d70ba1d9d1 bundle: add an experimental knob to include obsmarkers in bundle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32548
diff changeset
  1694
    caps = {}
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1695
    if b'obsolescence' in opts:
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1696
        caps[b'obsmarkers'] = (b'V1',)
32549
37d70ba1d9d1 bundle: add an experimental knob to include obsmarkers in bundle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32548
diff changeset
  1697
    bundle = bundle20(ui, caps)
32261
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32232
diff changeset
  1698
    bundle.setcompression(compression, compopts)
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32232
diff changeset
  1699
    _addpartsfromopts(ui, repo, bundle, source, outgoing, opts)
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32232
diff changeset
  1700
    chunkiter = bundle.getchunks()
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32232
diff changeset
  1701
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32232
diff changeset
  1702
    return changegroup.writechunks(ui, chunkiter, filename, vfs=vfs)
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32232
diff changeset
  1703
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1704
32261
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32232
diff changeset
  1705
def _addpartsfromopts(ui, repo, bundler, source, outgoing, opts):
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32232
diff changeset
  1706
    # We should eventually reconcile this logic with the one behind
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32232
diff changeset
  1707
    # 'exchange.getbundle2partsgenerator'.
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32232
diff changeset
  1708
    #
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32232
diff changeset
  1709
    # The type of input from 'getbundle' and 'writenewbundle' are a bit
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32232
diff changeset
  1710
    # different right now. So we keep them separated for now for the sake of
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32232
diff changeset
  1711
    # simplicity.
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32232
diff changeset
  1712
37008
66c0ff381cfc bundle: condition the changegroup part when creating a new bundle
Boris Feld <boris.feld@octobus.net>
parents: 36974
diff changeset
  1713
    # we might not always want a changegroup in such bundle, for example in
66c0ff381cfc bundle: condition the changegroup part when creating a new bundle
Boris Feld <boris.feld@octobus.net>
parents: 36974
diff changeset
  1714
    # stream bundles
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1715
    if opts.get(b'changegroup', True):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1716
        cgversion = opts.get(b'cg.version')
37008
66c0ff381cfc bundle: condition the changegroup part when creating a new bundle
Boris Feld <boris.feld@octobus.net>
parents: 36974
diff changeset
  1717
        if cgversion is None:
66c0ff381cfc bundle: condition the changegroup part when creating a new bundle
Boris Feld <boris.feld@octobus.net>
parents: 36974
diff changeset
  1718
            cgversion = changegroup.safeversion(repo)
66c0ff381cfc bundle: condition the changegroup part when creating a new bundle
Boris Feld <boris.feld@octobus.net>
parents: 36974
diff changeset
  1719
        cg = changegroup.makechangegroup(repo, outgoing, cgversion, source)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1720
        part = bundler.newpart(b'changegroup', data=cg.getchunks())
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1721
        part.addparam(b'version', cg.version)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1722
        if b'clcount' in cg.extras:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1723
            part.addparam(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1724
                b'nbchanges', b'%d' % cg.extras[b'clcount'], mandatory=False
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1725
            )
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1726
        if opts.get(b'phases') and repo.revs(
45144
c93dd9d9f1e6 discovery: change users of `outgoing.missingheads` to `outgoing.ancestorsof`
Manuel Jacob <me@manueljacob.de>
parents: 45117
diff changeset
  1727
            b'%ln and secret()', outgoing.ancestorsof
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1728
        ):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1729
            part.addparam(
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1730
                b'targetphase', b'%d' % phases.secret, mandatory=False
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1731
            )
43131
c17a63eb5d4c sidedata: apply basic but tight security around exchange
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43117
diff changeset
  1732
        if b'exp-sidedata-flag' in repo.requirements:
c17a63eb5d4c sidedata: apply basic but tight security around exchange
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43117
diff changeset
  1733
            part.addparam(b'exp-sidedata', b'1')
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1734
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1735
    if opts.get(b'streamv2', False):
37169
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part
Boris Feld <boris.feld@octobus.net>
parents: 37167
diff changeset
  1736
        addpartbundlestream2(bundler, repo, stream=True)
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part
Boris Feld <boris.feld@octobus.net>
parents: 37167
diff changeset
  1737
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1738
    if opts.get(b'tagsfnodescache', True):
37167
6c7a6b04b274 bundlespec: move computing the bundle contentops in parsebundlespec
Boris Feld <boris.feld@octobus.net>
parents: 37087
diff changeset
  1739
        addparttagsfnodescache(repo, bundler, outgoing)
6c7a6b04b274 bundlespec: move computing the bundle contentops in parsebundlespec
Boris Feld <boris.feld@octobus.net>
parents: 37087
diff changeset
  1740
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1741
    if opts.get(b'revbranchcache', True):
37167
6c7a6b04b274 bundlespec: move computing the bundle contentops in parsebundlespec
Boris Feld <boris.feld@octobus.net>
parents: 37087
diff changeset
  1742
        addpartrevbranchcache(repo, bundler, outgoing)
32263
d7f93ebbbbdf bundle: add optional 'tagsfnodecache' data to on disk bundle (issue5543)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32262
diff changeset
  1743
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1744
    if opts.get(b'obsolescence', False):
32549
37d70ba1d9d1 bundle: add an experimental knob to include obsmarkers in bundle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32548
diff changeset
  1745
        obsmarkers = repo.obsstore.relevantmarkers(outgoing.missing)
46165
41d695a08e90 bundle: optional advisory obsolescence parts
Joerg Sonnenberger <joerg@bec.de>
parents: 46114
diff changeset
  1746
        buildobsmarkerspart(
41d695a08e90 bundle: optional advisory obsolescence parts
Joerg Sonnenberger <joerg@bec.de>
parents: 46114
diff changeset
  1747
            bundler,
41d695a08e90 bundle: optional advisory obsolescence parts
Joerg Sonnenberger <joerg@bec.de>
parents: 46114
diff changeset
  1748
            obsmarkers,
41d695a08e90 bundle: optional advisory obsolescence parts
Joerg Sonnenberger <joerg@bec.de>
parents: 46114
diff changeset
  1749
            mandatory=opts.get(b'obsolescence-mandatory', True),
41d695a08e90 bundle: optional advisory obsolescence parts
Joerg Sonnenberger <joerg@bec.de>
parents: 46114
diff changeset
  1750
        )
32549
37d70ba1d9d1 bundle: add an experimental knob to include obsmarkers in bundle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32548
diff changeset
  1751
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1752
    if opts.get(b'phases', False):
33043
e8c8d81eb864 bundle: add config option to include phases
Martin von Zweigbergk <martinvonz@google.com>
parents: 33042
diff changeset
  1753
        headsbyphase = phases.subsetphaseheads(repo, outgoing.missing)
34326
5779d096a696 phases: move binary encoding into a reusable function
Boris Feld <boris.feld@octobus.net>
parents: 34325
diff changeset
  1754
        phasedata = phases.binaryencode(headsbyphase)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1755
        bundler.newpart(b'phase-heads', data=phasedata)
33043
e8c8d81eb864 bundle: add config option to include phases
Martin von Zweigbergk <martinvonz@google.com>
parents: 33042
diff changeset
  1756
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1757
32262
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32261
diff changeset
  1758
def addparttagsfnodescache(repo, bundler, outgoing):
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32261
diff changeset
  1759
    # we include the tags fnode cache for the bundle changeset
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32261
diff changeset
  1760
    # (as an optional parts)
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32261
diff changeset
  1761
    cache = tags.hgtagsfnodescache(repo.unfiltered())
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32261
diff changeset
  1762
    chunks = []
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32261
diff changeset
  1763
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32261
diff changeset
  1764
    # .hgtags fnodes are only relevant for head changesets. While we could
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32261
diff changeset
  1765
    # transfer values for all known nodes, there will likely be little to
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32261
diff changeset
  1766
    # no benefit.
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32261
diff changeset
  1767
    #
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32261
diff changeset
  1768
    # We don't bother using a generator to produce output data because
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32261
diff changeset
  1769
    # a) we only have 40 bytes per head and even esoteric numbers of heads
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32261
diff changeset
  1770
    # consume little memory (1M heads is 40MB) b) we don't want to send the
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32261
diff changeset
  1771
    # part if we don't have entries and knowing if we have entries requires
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32261
diff changeset
  1772
    # cache lookups.
45144
c93dd9d9f1e6 discovery: change users of `outgoing.missingheads` to `outgoing.ancestorsof`
Manuel Jacob <me@manueljacob.de>
parents: 45117
diff changeset
  1773
    for node in outgoing.ancestorsof:
32262
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32261
diff changeset
  1774
        # Don't compute missing, as this may slow down serving.
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32261
diff changeset
  1775
        fnode = cache.getfnode(node, computemissing=False)
46547
5aac1a1a5beb tagcache: distinguish between invalid and missing entries
Matt Harbison <matt_harbison@yahoo.com>
parents: 46481
diff changeset
  1776
        if fnode:
32262
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32261
diff changeset
  1777
            chunks.extend([node, fnode])
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32261
diff changeset
  1778
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32261
diff changeset
  1779
    if chunks:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1780
        bundler.newpart(b'hgtagsfnodes', data=b''.join(chunks))
32262
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32261
diff changeset
  1781
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1782
36970
79b73be4dda5 rev-branch-cache: add a function to generate a part
Boris Feld <boris.feld@octobus.net>
parents: 36969
diff changeset
  1783
def addpartrevbranchcache(repo, bundler, outgoing):
79b73be4dda5 rev-branch-cache: add a function to generate a part
Boris Feld <boris.feld@octobus.net>
parents: 36969
diff changeset
  1784
    # we include the rev branch cache for the bundle changeset
79b73be4dda5 rev-branch-cache: add a function to generate a part
Boris Feld <boris.feld@octobus.net>
parents: 36969
diff changeset
  1785
    # (as an optional parts)
79b73be4dda5 rev-branch-cache: add a function to generate a part
Boris Feld <boris.feld@octobus.net>
parents: 36969
diff changeset
  1786
    cache = repo.revbranchcache()
79b73be4dda5 rev-branch-cache: add a function to generate a part
Boris Feld <boris.feld@octobus.net>
parents: 36969
diff changeset
  1787
    cl = repo.unfiltered().changelog
79b73be4dda5 rev-branch-cache: add a function to generate a part
Boris Feld <boris.feld@octobus.net>
parents: 36969
diff changeset
  1788
    branchesdata = collections.defaultdict(lambda: (set(), set()))
79b73be4dda5 rev-branch-cache: add a function to generate a part
Boris Feld <boris.feld@octobus.net>
parents: 36969
diff changeset
  1789
    for node in outgoing.missing:
79b73be4dda5 rev-branch-cache: add a function to generate a part
Boris Feld <boris.feld@octobus.net>
parents: 36969
diff changeset
  1790
        branch, close = cache.branchinfo(cl.rev(node))
79b73be4dda5 rev-branch-cache: add a function to generate a part
Boris Feld <boris.feld@octobus.net>
parents: 36969
diff changeset
  1791
        branchesdata[branch][close].add(node)
79b73be4dda5 rev-branch-cache: add a function to generate a part
Boris Feld <boris.feld@octobus.net>
parents: 36969
diff changeset
  1792
79b73be4dda5 rev-branch-cache: add a function to generate a part
Boris Feld <boris.feld@octobus.net>
parents: 36969
diff changeset
  1793
    def generate():
79b73be4dda5 rev-branch-cache: add a function to generate a part
Boris Feld <boris.feld@octobus.net>
parents: 36969
diff changeset
  1794
        for branch, (nodes, closed) in sorted(branchesdata.items()):
79b73be4dda5 rev-branch-cache: add a function to generate a part
Boris Feld <boris.feld@octobus.net>
parents: 36969
diff changeset
  1795
            utf8branch = encoding.fromlocal(branch)
79b73be4dda5 rev-branch-cache: add a function to generate a part
Boris Feld <boris.feld@octobus.net>
parents: 36969
diff changeset
  1796
            yield rbcstruct.pack(len(utf8branch), len(nodes), len(closed))
79b73be4dda5 rev-branch-cache: add a function to generate a part
Boris Feld <boris.feld@octobus.net>
parents: 36969
diff changeset
  1797
            yield utf8branch
79b73be4dda5 rev-branch-cache: add a function to generate a part
Boris Feld <boris.feld@octobus.net>
parents: 36969
diff changeset
  1798
            for n in sorted(nodes):
79b73be4dda5 rev-branch-cache: add a function to generate a part
Boris Feld <boris.feld@octobus.net>
parents: 36969
diff changeset
  1799
                yield n
79b73be4dda5 rev-branch-cache: add a function to generate a part
Boris Feld <boris.feld@octobus.net>
parents: 36969
diff changeset
  1800
            for n in sorted(closed):
79b73be4dda5 rev-branch-cache: add a function to generate a part
Boris Feld <boris.feld@octobus.net>
parents: 36969
diff changeset
  1801
                yield n
79b73be4dda5 rev-branch-cache: add a function to generate a part
Boris Feld <boris.feld@octobus.net>
parents: 36969
diff changeset
  1802
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1803
    bundler.newpart(b'cache:rev-branch-cache', data=generate(), mandatory=False)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1804
36970
79b73be4dda5 rev-branch-cache: add a function to generate a part
Boris Feld <boris.feld@octobus.net>
parents: 36969
diff changeset
  1805
37169
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part
Boris Feld <boris.feld@octobus.net>
parents: 37167
diff changeset
  1806
def _formatrequirementsspec(requirements):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1807
    requirements = [req for req in requirements if req != b"shared"]
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1808
    return urlreq.quote(b','.join(sorted(requirements)))
37169
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part
Boris Feld <boris.feld@octobus.net>
parents: 37167
diff changeset
  1809
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1810
37169
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part
Boris Feld <boris.feld@octobus.net>
parents: 37167
diff changeset
  1811
def _formatrequirementsparams(requirements):
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part
Boris Feld <boris.feld@octobus.net>
parents: 37167
diff changeset
  1812
    requirements = _formatrequirementsspec(requirements)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1813
    params = b"%s%s" % (urlreq.quote(b"requirements="), requirements)
37169
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part
Boris Feld <boris.feld@octobus.net>
parents: 37167
diff changeset
  1814
    return params
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part
Boris Feld <boris.feld@octobus.net>
parents: 37167
diff changeset
  1815
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1816
46726
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos
Raphaël Gomès <rgomes@octobus.net>
parents: 46672
diff changeset
  1817
def format_remote_wanted_sidedata(repo):
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos
Raphaël Gomès <rgomes@octobus.net>
parents: 46672
diff changeset
  1818
    """Formats a repo's wanted sidedata categories into a bytestring for
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos
Raphaël Gomès <rgomes@octobus.net>
parents: 46672
diff changeset
  1819
    capabilities exchange."""
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos
Raphaël Gomès <rgomes@octobus.net>
parents: 46672
diff changeset
  1820
    wanted = b""
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos
Raphaël Gomès <rgomes@octobus.net>
parents: 46672
diff changeset
  1821
    if repo._wanted_sidedata:
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos
Raphaël Gomès <rgomes@octobus.net>
parents: 46672
diff changeset
  1822
        wanted = b','.join(
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos
Raphaël Gomès <rgomes@octobus.net>
parents: 46672
diff changeset
  1823
            pycompat.bytestr(c) for c in sorted(repo._wanted_sidedata)
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos
Raphaël Gomès <rgomes@octobus.net>
parents: 46672
diff changeset
  1824
        )
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos
Raphaël Gomès <rgomes@octobus.net>
parents: 46672
diff changeset
  1825
    return wanted
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos
Raphaël Gomès <rgomes@octobus.net>
parents: 46672
diff changeset
  1826
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos
Raphaël Gomès <rgomes@octobus.net>
parents: 46672
diff changeset
  1827
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos
Raphaël Gomès <rgomes@octobus.net>
parents: 46672
diff changeset
  1828
def read_remote_wanted_sidedata(remote):
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos
Raphaël Gomès <rgomes@octobus.net>
parents: 46672
diff changeset
  1829
    sidedata_categories = remote.capable(b'exp-wanted-sidedata')
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos
Raphaël Gomès <rgomes@octobus.net>
parents: 46672
diff changeset
  1830
    return read_wanted_sidedata(sidedata_categories)
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos
Raphaël Gomès <rgomes@octobus.net>
parents: 46672
diff changeset
  1831
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos
Raphaël Gomès <rgomes@octobus.net>
parents: 46672
diff changeset
  1832
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos
Raphaël Gomès <rgomes@octobus.net>
parents: 46672
diff changeset
  1833
def read_wanted_sidedata(formatted):
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos
Raphaël Gomès <rgomes@octobus.net>
parents: 46672
diff changeset
  1834
    if formatted:
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos
Raphaël Gomès <rgomes@octobus.net>
parents: 46672
diff changeset
  1835
        return set(formatted.split(b','))
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos
Raphaël Gomès <rgomes@octobus.net>
parents: 46672
diff changeset
  1836
    return set()
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos
Raphaël Gomès <rgomes@octobus.net>
parents: 46672
diff changeset
  1837
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos
Raphaël Gomès <rgomes@octobus.net>
parents: 46672
diff changeset
  1838
37169
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part
Boris Feld <boris.feld@octobus.net>
parents: 37167
diff changeset
  1839
def addpartbundlestream2(bundler, repo, **kwargs):
43554
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43543
diff changeset
  1840
    if not kwargs.get('stream', False):
37169
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part
Boris Feld <boris.feld@octobus.net>
parents: 37167
diff changeset
  1841
        return
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part
Boris Feld <boris.feld@octobus.net>
parents: 37167
diff changeset
  1842
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part
Boris Feld <boris.feld@octobus.net>
parents: 37167
diff changeset
  1843
    if not streamclone.allowservergeneration(repo):
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1844
        raise error.Abort(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1845
            _(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1846
                b'stream data requested but server does not allow '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1847
                b'this feature'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1848
            ),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1849
            hint=_(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1850
                b'well-behaved clients should not be '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1851
                b'requesting stream data from servers not '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1852
                b'advertising it; the client may be buggy'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1853
            ),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1854
        )
37169
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part
Boris Feld <boris.feld@octobus.net>
parents: 37167
diff changeset
  1855
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part
Boris Feld <boris.feld@octobus.net>
parents: 37167
diff changeset
  1856
    # Stream clones don't compress well. And compression undermines a
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part
Boris Feld <boris.feld@octobus.net>
parents: 37167
diff changeset
  1857
    # goal of stream clones, which is to be fast. Communicate the desire
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part
Boris Feld <boris.feld@octobus.net>
parents: 37167
diff changeset
  1858
    # to avoid compression to consumers of the bundle.
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part
Boris Feld <boris.feld@octobus.net>
parents: 37167
diff changeset
  1859
    bundler.prefercompressed = False
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part
Boris Feld <boris.feld@octobus.net>
parents: 37167
diff changeset
  1860
40338
af62936c2508 streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40147
diff changeset
  1861
    # get the includes and excludes
43554
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43543
diff changeset
  1862
    includepats = kwargs.get('includepats')
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43543
diff changeset
  1863
    excludepats = kwargs.get('excludepats')
40338
af62936c2508 streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40147
diff changeset
  1864
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1865
    narrowstream = repo.ui.configbool(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1866
        b'experimental', b'server.stream-narrow-clones'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1867
    )
40338
af62936c2508 streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40147
diff changeset
  1868
af62936c2508 streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40147
diff changeset
  1869
    if (includepats or excludepats) and not narrowstream:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1870
        raise error.Abort(_(b'server does not support narrow stream clones'))
40338
af62936c2508 streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40147
diff changeset
  1871
40398
0ac794e0e285 streamclone: include obsstore file into stream bundle if client can read it
Anton Shestakov <av6@dwimlabs.net>
parents: 40372
diff changeset
  1872
    includeobsmarkers = False
0ac794e0e285 streamclone: include obsstore file into stream bundle if client can read it
Anton Shestakov <av6@dwimlabs.net>
parents: 40372
diff changeset
  1873
    if repo.obsstore:
0ac794e0e285 streamclone: include obsstore file into stream bundle if client can read it
Anton Shestakov <av6@dwimlabs.net>
parents: 40372
diff changeset
  1874
        remoteversions = obsmarkersversion(bundler.capabilities)
40399
4ab6e7b4fe8a streamclone: abort when client needs to handle obsmarkers, but doesn't
Anton Shestakov <av6@dwimlabs.net>
parents: 40398
diff changeset
  1875
        if not remoteversions:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1876
            raise error.Abort(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1877
                _(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1878
                    b'server has obsolescence markers, but client '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1879
                    b'cannot receive them via stream clone'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1880
                )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1881
            )
40399
4ab6e7b4fe8a streamclone: abort when client needs to handle obsmarkers, but doesn't
Anton Shestakov <av6@dwimlabs.net>
parents: 40398
diff changeset
  1882
        elif repo.obsstore._version in remoteversions:
40398
0ac794e0e285 streamclone: include obsstore file into stream bundle if client can read it
Anton Shestakov <av6@dwimlabs.net>
parents: 40372
diff changeset
  1883
            includeobsmarkers = True
0ac794e0e285 streamclone: include obsstore file into stream bundle if client can read it
Anton Shestakov <av6@dwimlabs.net>
parents: 40372
diff changeset
  1884
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1885
    filecount, bytecount, it = streamclone.generatev2(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1886
        repo, includepats, excludepats, includeobsmarkers
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1887
    )
37169
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part
Boris Feld <boris.feld@octobus.net>
parents: 37167
diff changeset
  1888
    requirements = _formatrequirementsspec(repo.requirements)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1889
    part = bundler.newpart(b'stream2', data=it)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1890
    part.addparam(b'bytecount', b'%d' % bytecount, mandatory=True)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1891
    part.addparam(b'filecount', b'%d' % filecount, mandatory=True)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1892
    part.addparam(b'requirements', requirements, mandatory=True)
37169
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part
Boris Feld <boris.feld@octobus.net>
parents: 37167
diff changeset
  1893
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1894
46165
41d695a08e90 bundle: optional advisory obsolescence parts
Joerg Sonnenberger <joerg@bec.de>
parents: 46114
diff changeset
  1895
def buildobsmarkerspart(bundler, markers, mandatory=True):
32548
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32425
diff changeset
  1896
    """add an obsmarker part to the bundler with <markers>
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32425
diff changeset
  1897
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32425
diff changeset
  1898
    No part is created if markers is empty.
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32425
diff changeset
  1899
    Raises ValueError if the bundler doesn't support any known obsmarker format.
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32425
diff changeset
  1900
    """
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32425
diff changeset
  1901
    if not markers:
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32425
diff changeset
  1902
        return None
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32425
diff changeset
  1903
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32425
diff changeset
  1904
    remoteversions = obsmarkersversion(bundler.capabilities)
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32425
diff changeset
  1905
    version = obsolete.commonversion(remoteversions)
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32425
diff changeset
  1906
    if version is None:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1907
        raise ValueError(b'bundler does not support common obsmarker format')
32548
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32425
diff changeset
  1908
    stream = obsolete.encodemarkers(markers, True, version=version)
46165
41d695a08e90 bundle: optional advisory obsolescence parts
Joerg Sonnenberger <joerg@bec.de>
parents: 46114
diff changeset
  1909
    return bundler.newpart(b'obsmarkers', data=stream, mandatory=mandatory)
32548
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32425
diff changeset
  1910
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1911
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1912
def writebundle(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1913
    ui, cg, filename, bundletype, vfs=None, compression=None, compopts=None
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1914
):
28666
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
  1915
    """Write a bundle file and return its filename.
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
  1916
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
  1917
    Existing files will not be overwritten.
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
  1918
    If no filename is specified, a temporary file is created.
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
  1919
    bz2 compression can be turned off.
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
  1920
    The bundle file will be deleted in case of errors.
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
  1921
    """
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
  1922
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1923
    if bundletype == b"HG20":
28666
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
  1924
        bundle = bundle20(ui)
30757
511a4bf52754 bundle2: allow compression options to be passed to compressor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30448
diff changeset
  1925
        bundle.setcompression(compression, compopts)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1926
        part = bundle.newpart(b'changegroup', data=cg.getchunks())
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1927
        part.addparam(b'version', cg.version)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1928
        if b'clcount' in cg.extras:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1929
            part.addparam(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1930
                b'nbchanges', b'%d' % cg.extras[b'clcount'], mandatory=False
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1931
            )
28666
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
  1932
        chunkiter = bundle.getchunks()
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
  1933
    else:
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
  1934
        # compression argument is only for the bundle2 case
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
  1935
        assert compression is None
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1936
        if cg.version != b'01':
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1937
            raise error.Abort(
43117
8ff1ecfadcd1 cleanup: join string literals that are already on one line
Martin von Zweigbergk <martinvonz@google.com>
parents: 43115
diff changeset
  1938
                _(b'old bundle types only supports v1 changegroups')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1939
            )
28666
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
  1940
        header, comp = bundletypes[bundletype]
30361
f81002f736d7 bundle2: use new compression engine API for compression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30177
diff changeset
  1941
        if comp not in util.compengines.supportedbundletypes:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1942
            raise error.Abort(_(b'unknown stream compression type: %s') % comp)
30361
f81002f736d7 bundle2: use new compression engine API for compression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30177
diff changeset
  1943
        compengine = util.compengines.forbundletype(comp)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1944
28666
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
  1945
        def chunkiter():
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
  1946
            yield header
30757
511a4bf52754 bundle2: allow compression options to be passed to compressor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30448
diff changeset
  1947
            for chunk in compengine.compressstream(cg.getchunks(), compopts):
30367
5925bda42dbd bundle2: use compressstream compression engine API
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30363
diff changeset
  1948
                yield chunk
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1949
28666
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
  1950
        chunkiter = chunkiter()
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
  1951
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
  1952
    # parse the changegroup data, otherwise we will block
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
  1953
    # in case of sshrepo because we don't know the end of the stream
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
  1954
    return changegroup.writechunks(ui, chunkiter, filename, vfs=vfs)
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
  1955
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1956
33049
d765ad56081f bundle: make combinechangegroupresults() take a bundleoperation
Martin von Zweigbergk <martinvonz@google.com>
parents: 33048
diff changeset
  1957
def combinechangegroupresults(op):
33048
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33047
diff changeset
  1958
    """logic to combine 0 or more addchangegroup results into one"""
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1959
    results = [r.get(b'return', 0) for r in op.records[b'changegroup']]
33048
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33047
diff changeset
  1960
    changedheads = 0
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33047
diff changeset
  1961
    result = 1
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33047
diff changeset
  1962
    for ret in results:
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33047
diff changeset
  1963
        # If any changegroup result is 0, return 0
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33047
diff changeset
  1964
        if ret == 0:
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33047
diff changeset
  1965
            result = 0
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33047
diff changeset
  1966
            break
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33047
diff changeset
  1967
        if ret < -1:
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33047
diff changeset
  1968
            changedheads += ret + 1
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33047
diff changeset
  1969
        elif ret > 1:
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33047
diff changeset
  1970
            changedheads += ret - 1
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33047
diff changeset
  1971
    if changedheads > 0:
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33047
diff changeset
  1972
        result = 1 + changedheads
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33047
diff changeset
  1973
    elif changedheads < 0:
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33047
diff changeset
  1974
        result = -1 + changedheads
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33047
diff changeset
  1975
    return result
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33047
diff changeset
  1976
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1977
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1978
@parthandler(
43131
c17a63eb5d4c sidedata: apply basic but tight security around exchange
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43117
diff changeset
  1979
    b'changegroup',
c17a63eb5d4c sidedata: apply basic but tight security around exchange
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43117
diff changeset
  1980
    (
c17a63eb5d4c sidedata: apply basic but tight security around exchange
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43117
diff changeset
  1981
        b'version',
c17a63eb5d4c sidedata: apply basic but tight security around exchange
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43117
diff changeset
  1982
        b'nbchanges',
c17a63eb5d4c sidedata: apply basic but tight security around exchange
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43117
diff changeset
  1983
        b'exp-sidedata',
46726
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos
Raphaël Gomès <rgomes@octobus.net>
parents: 46672
diff changeset
  1984
        b'exp-wanted-sidedata',
43131
c17a63eb5d4c sidedata: apply basic but tight security around exchange
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43117
diff changeset
  1985
        b'treemanifest',
c17a63eb5d4c sidedata: apply basic but tight security around exchange
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43117
diff changeset
  1986
        b'targetphase',
c17a63eb5d4c sidedata: apply basic but tight security around exchange
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43117
diff changeset
  1987
    ),
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  1988
)
20998
93a3c5b58635 bundle2: use reply part to return result of addchangegroup
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20997
diff changeset
  1989
def handlechangegroup(op, inpart):
46166
c511fef30290 bundle2: drop some outdated comment
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46165
diff changeset
  1990
    """apply a changegroup part on the repo"""
39716
b10d145837bc localrepo: extract resolving of opener options to standalone functions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38823
diff changeset
  1991
    from . import localrepo
b10d145837bc localrepo: extract resolving of opener options to standalone functions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38823
diff changeset
  1992
32948
af31d531dda0 changegroup: let callers pass in transaction to apply() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 32910
diff changeset
  1993
    tr = op.gettransaction()
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1994
    unpackerversion = inpart.params.get(b'version', b'01')
23170
02e8f9b60052 bundle2: support a "version" argument in `changegroup` part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23169
diff changeset
  1995
    # We should raise an appropriate exception here
27751
a40e2f7fe49d changegroup: hide packermap behind methods
Martin von Zweigbergk <martinvonz@google.com>
parents: 27734
diff changeset
  1996
    cg = changegroup.getunbundler(unpackerversion, inpart, None)
23001
4df9b5e62f70 bundle2: add a comment about addchangegroup source and url
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22953
diff changeset
  1997
    # the source and url passed here are overwritten by the one contained in
4df9b5e62f70 bundle2: add a comment about addchangegroup source and url
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22953
diff changeset
  1998
    # the transaction.hookargs argument. So 'bundle2' is a placeholder
25518
ca656f3dffd7 bundle2: provide number of changesets information to 'addchangegroup'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25507
diff changeset
  1999
    nbchangesets = None
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2000
    if b'nbchanges' in inpart.params:
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2001
        nbchangesets = int(inpart.params.get(b'nbchanges'))
45558
10284ce3d5ed scmutil: introduce function to check whether repo uses treemanifest or not
Pulkit Goyal <7895pulkit@gmail.com>
parents: 45392
diff changeset
  2002
    if b'treemanifest' in inpart.params and not scmutil.istreemanifest(op.repo):
27734
5c0fd878779c treemanifests: set bundle2 part parameter indicating treemanifest
Martin von Zweigbergk <martinvonz@google.com>
parents: 26829
diff changeset
  2003
        if len(op.repo.changelog) != 0:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2004
            raise error.Abort(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2005
                _(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2006
                    b"bundle contains tree manifests, but local repo is "
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2007
                    b"non-empty and does not use tree manifests"
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2008
                )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2009
            )
45392
77b8588dd84e requirements: introduce new requirements related module
Pulkit Goyal <7895pulkit@gmail.com>
parents: 45391
diff changeset
  2010
        op.repo.requirements.add(requirements.TREEMANIFEST_REQUIREMENT)
39716
b10d145837bc localrepo: extract resolving of opener options to standalone functions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38823
diff changeset
  2011
        op.repo.svfs.options = localrepo.resolvestorevfsoptions(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2012
            op.repo.ui, op.repo.requirements, op.repo.features
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2013
        )
45106
a03c177a4679 scmutil: add writereporequirements() and route requires writing through it
Pulkit Goyal <7895pulkit@gmail.com>
parents: 44406
diff changeset
  2014
        scmutil.writereporequirements(op.repo)
43131
c17a63eb5d4c sidedata: apply basic but tight security around exchange
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43117
diff changeset
  2015
c17a63eb5d4c sidedata: apply basic but tight security around exchange
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43117
diff changeset
  2016
    bundlesidedata = bool(b'exp-sidedata' in inpart.params)
c17a63eb5d4c sidedata: apply basic but tight security around exchange
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43117
diff changeset
  2017
    reposidedata = bool(b'exp-sidedata-flag' in op.repo.requirements)
c17a63eb5d4c sidedata: apply basic but tight security around exchange
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43117
diff changeset
  2018
    if reposidedata and not bundlesidedata:
c17a63eb5d4c sidedata: apply basic but tight security around exchange
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43117
diff changeset
  2019
        msg = b"repository is using sidedata but the bundle source do not"
c17a63eb5d4c sidedata: apply basic but tight security around exchange
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43117
diff changeset
  2020
        hint = b'this is currently unsupported'
c17a63eb5d4c sidedata: apply basic but tight security around exchange
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43117
diff changeset
  2021
        raise error.Abort(msg, hint=hint)
c17a63eb5d4c sidedata: apply basic but tight security around exchange
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43117
diff changeset
  2022
33407
39d4e5a66f5a bundle2: support the 'targetphase' parameter for the changegroup part
Boris Feld <boris.feld@octobus.net>
parents: 33252
diff changeset
  2023
    extrakwargs = {}
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2024
    targetphase = inpart.params.get(b'targetphase')
33407
39d4e5a66f5a bundle2: support the 'targetphase' parameter for the changegroup part
Boris Feld <boris.feld@octobus.net>
parents: 33252
diff changeset
  2025
    if targetphase is not None:
43554
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43543
diff changeset
  2026
        extrakwargs['targetphase'] = int(targetphase)
46726
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos
Raphaël Gomès <rgomes@octobus.net>
parents: 46672
diff changeset
  2027
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos
Raphaël Gomès <rgomes@octobus.net>
parents: 46672
diff changeset
  2028
    remote_sidedata = inpart.params.get(b'exp-wanted-sidedata')
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos
Raphaël Gomès <rgomes@octobus.net>
parents: 46672
diff changeset
  2029
    extrakwargs['sidedata_categories'] = read_wanted_sidedata(remote_sidedata)
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos
Raphaël Gomès <rgomes@octobus.net>
parents: 46672
diff changeset
  2030
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2031
    ret = _processchangegroup(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2032
        op,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2033
        cg,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2034
        tr,
46571
83ac7c91e172 bundle2: pass the operation source down to the changegroup
Raphaël Gomès <rgomes@octobus.net>
parents: 46547
diff changeset
  2035
        op.source,
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2036
        b'bundle2',
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2037
        expectedtotal=nbchangesets,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2038
        **extrakwargs
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2039
    )
20998
93a3c5b58635 bundle2: use reply part to return result of addchangegroup
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20997
diff changeset
  2040
    if op.reply is not None:
23139
e53f6b72a0e4 spelling: fixes from proofreading of spell checker issues
Mads Kiilerich <madski@unity3d.com>
parents: 23067
diff changeset
  2041
        # This is definitely not the final form of this
20998
93a3c5b58635 bundle2: use reply part to return result of addchangegroup
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20997
diff changeset
  2042
        # return. But one need to start somewhere.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2043
        part = op.reply.newpart(b'reply:changegroup', mandatory=False)
33701
24b7c75c6441 bundle2: use bytestr() instead of str() to convert part id to bytes
Augie Fackler <augie@google.com>
parents: 33700
diff changeset
  2044
        part.addparam(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2045
            b'in-reply-to', pycompat.bytestr(inpart.id), mandatory=False
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2046
        )
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2047
        part.addparam(b'return', b'%i' % ret, mandatory=False)
21019
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
  2048
    assert not inpart.read()
20950
c7ceae0faf69 bundle2: first crude version of bundling changeset with bundle2
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20949
diff changeset
  2049
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2050
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2051
_remotechangegroupparams = tuple(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2052
    [b'url', b'size', b'digests']
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2053
    + [b'digest:%s' % k for k in util.DIGESTS.keys()]
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2054
)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2055
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2056
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2057
@parthandler(b'remote-changegroup', _remotechangegroupparams)
23029
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
  2058
def handleremotechangegroup(op, inpart):
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
  2059
    """apply a bundle10 on the repo, given an url and validation information
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
  2060
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
  2061
    All the information about the remote bundle to import are given as
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
  2062
    parameters. The parameters include:
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
  2063
      - url: the url to the bundle10.
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
  2064
      - size: the bundle10 file size. It is used to validate what was
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
  2065
        retrieved by the client matches the server knowledge about the bundle.
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
  2066
      - digests: a space separated list of the digest types provided as
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
  2067
        parameters.
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
  2068
      - digest:<digest-type>: the hexadecimal representation of the digest with
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
  2069
        that name. Like the size, it is used to validate what was retrieved by
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
  2070
        the client matches what the server knows about the bundle.
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
  2071
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
  2072
    When multiple digest types are given, all of them are checked.
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
  2073
    """
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
  2074
    try:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2075
        raw_url = inpart.params[b'url']
23029
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
  2076
    except KeyError:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2077
        raise error.Abort(_(b'remote-changegroup: missing "%s" param') % b'url')
46907
ffd3e823a7e5 urlutil: extract `url` related code from `util` into the new module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46793
diff changeset
  2078
    parsed_url = urlutil.url(raw_url)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2079
    if parsed_url.scheme not in capabilities[b'remote-changegroup']:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2080
        raise error.Abort(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2081
            _(b'remote-changegroup does not support %s urls')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2082
            % parsed_url.scheme
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2083
        )
23029
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
  2084
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
  2085
    try:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2086
        size = int(inpart.params[b'size'])
23029
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
  2087
    except ValueError:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2088
        raise error.Abort(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2089
            _(b'remote-changegroup: invalid value for param "%s"') % b'size'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2090
        )
23029
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
  2091
    except KeyError:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2092
        raise error.Abort(
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2093
            _(b'remote-changegroup: missing "%s" param') % b'size'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2094
        )
23029
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
  2095
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
  2096
    digests = {}
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2097
    for typ in inpart.params.get(b'digests', b'').split():
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2098
        param = b'digest:%s' % typ
23029
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
  2099
        try:
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
  2100
            value = inpart.params[param]
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
  2101
        except KeyError:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2102
            raise error.Abort(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2103
                _(b'remote-changegroup: missing "%s" param') % param
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2104
            )
23029
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
  2105
        digests[typ] = value
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
  2106
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
  2107
    real_part = util.digestchecker(url.open(op.ui, raw_url), size, digests)
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
  2108
32948
af31d531dda0 changegroup: let callers pass in transaction to apply() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 32910
diff changeset
  2109
    tr = op.gettransaction()
25919
8221fefaea08 bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
  2110
    from . import exchange
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2111
23029
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
  2112
    cg = exchange.readbundle(op.repo.ui, real_part, raw_url)
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
  2113
    if not isinstance(cg, changegroup.cg1unpacker):
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2114
        raise error.Abort(
46907
ffd3e823a7e5 urlutil: extract `url` related code from `util` into the new module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46793
diff changeset
  2115
            _(b'%s: not a bundle version 1.0') % urlutil.hidepassword(raw_url)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2116
        )
46571
83ac7c91e172 bundle2: pass the operation source down to the changegroup
Raphaël Gomès <rgomes@octobus.net>
parents: 46547
diff changeset
  2117
    ret = _processchangegroup(op, cg, tr, op.source, b'bundle2')
23029
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
  2118
    if op.reply is not None:
23139
e53f6b72a0e4 spelling: fixes from proofreading of spell checker issues
Mads Kiilerich <madski@unity3d.com>
parents: 23067
diff changeset
  2119
        # This is definitely not the final form of this
23029
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
  2120
        # return. But one need to start somewhere.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2121
        part = op.reply.newpart(b'reply:changegroup')
33701
24b7c75c6441 bundle2: use bytestr() instead of str() to convert part id to bytes
Augie Fackler <augie@google.com>
parents: 33700
diff changeset
  2122
        part.addparam(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2123
            b'in-reply-to', pycompat.bytestr(inpart.id), mandatory=False
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2124
        )
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2125
        part.addparam(b'return', b'%i' % ret, mandatory=False)
23029
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
  2126
    try:
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
  2127
        real_part.validate()
26587
56b2bcea2529 error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26565
diff changeset
  2128
    except error.Abort as e:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2129
        raise error.Abort(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2130
            _(b'bundle at %s is corrupted:\n%s')
46907
ffd3e823a7e5 urlutil: extract `url` related code from `util` into the new module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46793
diff changeset
  2131
            % (urlutil.hidepassword(raw_url), e.message)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2132
        )
23029
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
  2133
    assert not inpart.read()
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
  2134
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2135
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2136
@parthandler(b'reply:changegroup', (b'return', b'in-reply-to'))
22548
8a1ae133770a bundle2: rename functions that have the same name
Mike Hommey <mh@glandium.org>
parents: 22390
diff changeset
  2137
def handlereplychangegroup(op, inpart):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2138
    ret = int(inpart.params[b'return'])
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2139
    replyto = int(inpart.params[b'in-reply-to'])
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2140
    op.records.add(b'changegroup', {b'return': ret}, replyto)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2141
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2142
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2143
@parthandler(b'check:bookmarks')
35267
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35141
diff changeset
  2144
def handlecheckbookmarks(op, inpart):
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35141
diff changeset
  2145
    """check location of bookmarks
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35141
diff changeset
  2146
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35141
diff changeset
  2147
    This part is to be used to detect push race regarding bookmark, it
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35141
diff changeset
  2148
    contains binary encoded (bookmark, node) tuple. If the local state does
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35141
diff changeset
  2149
    not marks the one in the part, a PushRaced exception is raised
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35141
diff changeset
  2150
    """
46793
6266d19556ad node: introduce nodeconstants class
Joerg Sonnenberger <joerg@bec.de>
parents: 46726
diff changeset
  2151
    bookdata = bookmarks.binarydecode(op.repo, inpart)
35267
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35141
diff changeset
  2152
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2153
    msgstandard = (
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2154
        b'remote repository changed while pushing - please try again '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2155
        b'(bookmark "%s" move from %s to %s)'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2156
    )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2157
    msgmissing = (
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2158
        b'remote repository changed while pushing - please try again '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2159
        b'(bookmark "%s" is missing, expected %s)'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2160
    )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2161
    msgexist = (
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2162
        b'remote repository changed while pushing - please try again '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2163
        b'(bookmark "%s" set on %s, expected missing)'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2164
    )
35267
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35141
diff changeset
  2165
    for book, node in bookdata:
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35141
diff changeset
  2166
        currentnode = op.repo._bookmarks.get(book)
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35141
diff changeset
  2167
        if currentnode != node:
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35141
diff changeset
  2168
            if node is None:
46114
59fa3890d40a node: import symbols explicitly
Joerg Sonnenberger <joerg@bec.de>
parents: 45957
diff changeset
  2169
                finalmsg = msgexist % (book, short(currentnode))
35267
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35141
diff changeset
  2170
            elif currentnode is None:
46114
59fa3890d40a node: import symbols explicitly
Joerg Sonnenberger <joerg@bec.de>
parents: 45957
diff changeset
  2171
                finalmsg = msgmissing % (book, short(node))
35267
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35141
diff changeset
  2172
            else:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2173
                finalmsg = msgstandard % (
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2174
                    book,
46114
59fa3890d40a node: import symbols explicitly
Joerg Sonnenberger <joerg@bec.de>
parents: 45957
diff changeset
  2175
                    short(node),
59fa3890d40a node: import symbols explicitly
Joerg Sonnenberger <joerg@bec.de>
parents: 45957
diff changeset
  2176
                    short(currentnode),
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2177
                )
35267
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35141
diff changeset
  2178
            raise error.PushRaced(finalmsg)
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35141
diff changeset
  2179
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2180
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2181
@parthandler(b'check:heads')
22548
8a1ae133770a bundle2: rename functions that have the same name
Mike Hommey <mh@glandium.org>
parents: 22390
diff changeset
  2182
def handlecheckheads(op, inpart):
21060
0bea9db7543b bundle2: add a "check:heads" handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21024
diff changeset
  2183
    """check that head of the repo did not change
0bea9db7543b bundle2: add a "check:heads" handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21024
diff changeset
  2184
0bea9db7543b bundle2: add a "check:heads" handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21024
diff changeset
  2185
    This is used to detect a push race when using unbundle.
0bea9db7543b bundle2: add a "check:heads" handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21024
diff changeset
  2186
    This replaces the "heads" argument of unbundle."""
0bea9db7543b bundle2: add a "check:heads" handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21024
diff changeset
  2187
    h = inpart.read(20)
0bea9db7543b bundle2: add a "check:heads" handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21024
diff changeset
  2188
    heads = []
0bea9db7543b bundle2: add a "check:heads" handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21024
diff changeset
  2189
    while len(h) == 20:
0bea9db7543b bundle2: add a "check:heads" handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21024
diff changeset
  2190
        heads.append(h)
0bea9db7543b bundle2: add a "check:heads" handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21024
diff changeset
  2191
        h = inpart.read(20)
0bea9db7543b bundle2: add a "check:heads" handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21024
diff changeset
  2192
    assert not h
26565
ee1bcb9aa6e4 bundle2: add op.gettransaction() to handlers that need the lock
Durham Goode <durham@fb.com>
parents: 26542
diff changeset
  2193
    # Trigger a transaction so that we are guaranteed to have the lock now.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2194
    if op.ui.configbool(b'experimental', b'bundle2lazylocking'):
26565
ee1bcb9aa6e4 bundle2: add op.gettransaction() to handlers that need the lock
Durham Goode <durham@fb.com>
parents: 26542
diff changeset
  2195
        op.gettransaction()
29294
077d0535f51f bundle2: don't assume ordering of heads checked after push
Mads Kiilerich <madski@unity3d.com>
parents: 28883
diff changeset
  2196
    if sorted(heads) != sorted(op.repo.heads()):
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2197
        raise error.PushRaced(
43117
8ff1ecfadcd1 cleanup: join string literals that are already on one line
Martin von Zweigbergk <martinvonz@google.com>
parents: 43115
diff changeset
  2198
            b'remote repository changed while pushing - please try again'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2199
        )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2200
21130
1ff06386217f bundle2: introduce `replycaps` part for on-demand reply
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21129
diff changeset
  2201
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2202
@parthandler(b'check:updated-heads')
32729
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32549
diff changeset
  2203
def handlecheckupdatedheads(op, inpart):
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32549
diff changeset
  2204
    """check for race on the heads touched by a push
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32549
diff changeset
  2205
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32549
diff changeset
  2206
    This is similar to 'check:heads' but focus on the heads actually updated
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32549
diff changeset
  2207
    during the push. If other activities happen on unrelated heads, it is
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32549
diff changeset
  2208
    ignored.
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32549
diff changeset
  2209
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32549
diff changeset
  2210
    This allow server with high traffic to avoid push contention as long as
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32549
diff changeset
  2211
    unrelated parts of the graph are involved."""
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32549
diff changeset
  2212
    h = inpart.read(20)
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32549
diff changeset
  2213
    heads = []
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32549
diff changeset
  2214
    while len(h) == 20:
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32549
diff changeset
  2215
        heads.append(h)
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32549
diff changeset
  2216
        h = inpart.read(20)
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32549
diff changeset
  2217
    assert not h
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32549
diff changeset
  2218
    # trigger a transaction so that we are guaranteed to have the lock now.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2219
    if op.ui.configbool(b'experimental', b'bundle2lazylocking'):
32729
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32549
diff changeset
  2220
        op.gettransaction()
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32549
diff changeset
  2221
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32549
diff changeset
  2222
    currentheads = set()
42002
662ffdde5adf branchcache: rename itervalues() to iterheads()
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41801
diff changeset
  2223
    for ls in op.repo.branchmap().iterheads():
32729
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32549
diff changeset
  2224
        currentheads.update(ls)
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32549
diff changeset
  2225
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32549
diff changeset
  2226
    for h in heads:
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32549
diff changeset
  2227
        if h not in currentheads:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2228
            raise error.PushRaced(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2229
                b'remote repository changed while pushing - '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2230
                b'please try again'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2231
            )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2232
32729
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32549
diff changeset
  2233
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2234
@parthandler(b'check:phases')
34820
a95067b1dca6 phase: introduce a new 'check:phases' part
Boris Feld <boris.feld@octobus.net>
parents: 34637
diff changeset
  2235
def handlecheckphases(op, inpart):
a95067b1dca6 phase: introduce a new 'check:phases' part
Boris Feld <boris.feld@octobus.net>
parents: 34637
diff changeset
  2236
    """check that phase boundaries of the repository did not change
a95067b1dca6 phase: introduce a new 'check:phases' part
Boris Feld <boris.feld@octobus.net>
parents: 34637
diff changeset
  2237
a95067b1dca6 phase: introduce a new 'check:phases' part
Boris Feld <boris.feld@octobus.net>
parents: 34637
diff changeset
  2238
    This is used to detect a push race.
a95067b1dca6 phase: introduce a new 'check:phases' part
Boris Feld <boris.feld@octobus.net>
parents: 34637
diff changeset
  2239
    """
a95067b1dca6 phase: introduce a new 'check:phases' part
Boris Feld <boris.feld@octobus.net>
parents: 34637
diff changeset
  2240
    phasetonodes = phases.binarydecode(inpart)
a95067b1dca6 phase: introduce a new 'check:phases' part
Boris Feld <boris.feld@octobus.net>
parents: 34637
diff changeset
  2241
    unfi = op.repo.unfiltered()
a95067b1dca6 phase: introduce a new 'check:phases' part
Boris Feld <boris.feld@octobus.net>
parents: 34637
diff changeset
  2242
    cl = unfi.changelog
a95067b1dca6 phase: introduce a new 'check:phases' part
Boris Feld <boris.feld@octobus.net>
parents: 34637
diff changeset
  2243
    phasecache = unfi._phasecache
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2244
    msg = (
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2245
        b'remote repository changed while pushing - please try again '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2246
        b'(%s is %s expected %s)'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2247
    )
45117
b1e51ef4e536 phases: sparsify phase lists
Joerg Sonnenberger <joerg@bec.de>
parents: 45106
diff changeset
  2248
    for expectedphase, nodes in pycompat.iteritems(phasetonodes):
34820
a95067b1dca6 phase: introduce a new 'check:phases' part
Boris Feld <boris.feld@octobus.net>
parents: 34637
diff changeset
  2249
        for n in nodes:
a95067b1dca6 phase: introduce a new 'check:phases' part
Boris Feld <boris.feld@octobus.net>
parents: 34637
diff changeset
  2250
            actualphase = phasecache.phase(unfi, cl.rev(n))
a95067b1dca6 phase: introduce a new 'check:phases' part
Boris Feld <boris.feld@octobus.net>
parents: 34637
diff changeset
  2251
            if actualphase != expectedphase:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2252
                finalmsg = msg % (
46114
59fa3890d40a node: import symbols explicitly
Joerg Sonnenberger <joerg@bec.de>
parents: 45957
diff changeset
  2253
                    short(n),
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2254
                    phases.phasenames[actualphase],
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2255
                    phases.phasenames[expectedphase],
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2256
                )
34820
a95067b1dca6 phase: introduce a new 'check:phases' part
Boris Feld <boris.feld@octobus.net>
parents: 34637
diff changeset
  2257
                raise error.PushRaced(finalmsg)
a95067b1dca6 phase: introduce a new 'check:phases' part
Boris Feld <boris.feld@octobus.net>
parents: 34637
diff changeset
  2258
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2259
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2260
@parthandler(b'output')
21131
b7435117d951 bundle2: capture remote stdout while unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21130
diff changeset
  2261
def handleoutput(op, inpart):
b7435117d951 bundle2: capture remote stdout while unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21130
diff changeset
  2262
    """forward output captured on the server to the client"""
b7435117d951 bundle2: capture remote stdout while unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21130
diff changeset
  2263
    for line in inpart.read().splitlines():
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2264
        op.ui.status(_(b'remote: %s\n') % line)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2265
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2266
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2267
@parthandler(b'replycaps')
21130
1ff06386217f bundle2: introduce `replycaps` part for on-demand reply
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21129
diff changeset
  2268
def handlereplycaps(op, inpart):
1ff06386217f bundle2: introduce `replycaps` part for on-demand reply
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21129
diff changeset
  2269
    """Notify that a reply bundle should be created
1ff06386217f bundle2: introduce `replycaps` part for on-demand reply
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21129
diff changeset
  2270
21138
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
  2271
    The payload contains the capabilities information for the reply"""
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
  2272
    caps = decodecaps(inpart.read())
21130
1ff06386217f bundle2: introduce `replycaps` part for on-demand reply
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21129
diff changeset
  2273
    if op.reply is None:
21135
98fbf3adfd83 bundle2: add capabilities support in `replycaps` part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21134
diff changeset
  2274
        op.reply = bundle20(op.ui, caps)
21131
b7435117d951 bundle2: capture remote stdout while unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21130
diff changeset
  2275
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2276
26829
58f1645f72c3 bundle2: attribute remote failures to remote (issue4788)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26802
diff changeset
  2277
class AbortFromPart(error.Abort):
58f1645f72c3 bundle2: attribute remote failures to remote (issue4788)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26802
diff changeset
  2278
    """Sub-class of Abort that denotes an error from a bundle2 part."""
58f1645f72c3 bundle2: attribute remote failures to remote (issue4788)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26802
diff changeset
  2279
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2280
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2281
@parthandler(b'error:abort', (b'message', b'hint'))
24741
bb67e52362d6 bundle2: fix names for error part handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24733
diff changeset
  2282
def handleerrorabort(op, inpart):
21177
952af771bc17 bundle2: gracefully handle abort during unbundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21176
diff changeset
  2283
    """Used to transmit abort error over the wire"""
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2284
    raise AbortFromPart(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2285
        inpart.params[b'message'], hint=inpart.params.get(b'hint')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2286
    )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2287
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2288
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2289
@parthandler(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2290
    b'error:pushkey',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2291
    (b'namespace', b'key', b'new', b'old', b'ret', b'in-reply-to'),
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2292
)
25493
d8e7b0781ad7 bundle2: convey PushkeyFailed error over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25492
diff changeset
  2293
def handleerrorpushkey(op, inpart):
d8e7b0781ad7 bundle2: convey PushkeyFailed error over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25492
diff changeset
  2294
    """Used to transmit failure of a mandatory pushkey over the wire"""
d8e7b0781ad7 bundle2: convey PushkeyFailed error over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25492
diff changeset
  2295
    kwargs = {}
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2296
    for name in (b'namespace', b'key', b'new', b'old', b'ret'):
25493
d8e7b0781ad7 bundle2: convey PushkeyFailed error over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25492
diff changeset
  2297
        value = inpart.params.get(name)
d8e7b0781ad7 bundle2: convey PushkeyFailed error over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25492
diff changeset
  2298
        if value is not None:
d8e7b0781ad7 bundle2: convey PushkeyFailed error over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25492
diff changeset
  2299
            kwargs[name] = value
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2300
    raise error.PushkeyFailed(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2301
        inpart.params[b'in-reply-to'], **pycompat.strkwargs(kwargs)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2302
    )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2303
25493
d8e7b0781ad7 bundle2: convey PushkeyFailed error over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25492
diff changeset
  2304
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2305
@parthandler(b'error:unsupportedcontent', (b'parttype', b'params'))
24741
bb67e52362d6 bundle2: fix names for error part handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24733
diff changeset
  2306
def handleerrorunsupportedcontent(op, inpart):
21619
292331e906d7 bundle2: rename b2x:error:unknownpart to b2x:error:unsupportedcontent
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21618
diff changeset
  2307
    """Used to transmit unknown content error over the wire"""
21622
457492741007 bundle2: support transmission of params error over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21620
diff changeset
  2308
    kwargs = {}
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2309
    parttype = inpart.params.get(b'parttype')
21627
3e8bcc90f07c bundle2: support None parttype in BundleValueError
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21626
diff changeset
  2310
    if parttype is not None:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2311
        kwargs[b'parttype'] = parttype
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2312
    params = inpart.params.get(b'params')
21622
457492741007 bundle2: support transmission of params error over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21620
diff changeset
  2313
    if params is not None:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2314
        kwargs[b'params'] = params.split(b'\0')
21622
457492741007 bundle2: support transmission of params error over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21620
diff changeset
  2315
36457
247b473f408e bundle2: **strkwargs love on various kwargs constructions
Augie Fackler <augie@google.com>
parents: 36306
diff changeset
  2316
    raise error.BundleUnknownFeatureError(**pycompat.strkwargs(kwargs))
21186
9f3652e851f8 bundle2: gracefully handle PushRaced error during unbundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21185
diff changeset
  2317
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2318
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2319
@parthandler(b'error:pushraced', (b'message',))
24741
bb67e52362d6 bundle2: fix names for error part handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24733
diff changeset
  2320
def handleerrorpushraced(op, inpart):
21186
9f3652e851f8 bundle2: gracefully handle PushRaced error during unbundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21185
diff changeset
  2321
    """Used to transmit push race error over the wire"""
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2322
    raise error.ResponseError(_(b'push failed:'), inpart.params[b'message'])
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2323
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2324
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2325
@parthandler(b'listkeys', (b'namespace',))
21655
35095f332846 bundle: introduce a listkey handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21644
diff changeset
  2326
def handlelistkeys(op, inpart):
35095f332846 bundle: introduce a listkey handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21644
diff changeset
  2327
    """retrieve pushkey namespace content stored in a bundle2"""
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2328
    namespace = inpart.params[b'namespace']
21655
35095f332846 bundle: introduce a listkey handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21644
diff changeset
  2329
    r = pushkey.decodekeys(inpart.read())
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2330
    op.records.add(b'listkeys', (namespace, r))
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2331
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2332
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2333
@parthandler(b'pushkey', (b'namespace', b'key', b'old', b'new'))
21660
e87d2a12d41b bundle2: add ``pushkey`` support
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21655
diff changeset
  2334
def handlepushkey(op, inpart):
e87d2a12d41b bundle2: add ``pushkey`` support
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21655
diff changeset
  2335
    """process a pushkey request"""
e87d2a12d41b bundle2: add ``pushkey`` support
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21655
diff changeset
  2336
    dec = pushkey.decode
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2337
    namespace = dec(inpart.params[b'namespace'])
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2338
    key = dec(inpart.params[b'key'])
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2339
    old = dec(inpart.params[b'old'])
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2340
    new = dec(inpart.params[b'new'])
26565
ee1bcb9aa6e4 bundle2: add op.gettransaction() to handlers that need the lock
Durham Goode <durham@fb.com>
parents: 26542
diff changeset
  2341
    # Grab the transaction to ensure that we have the lock before performing the
ee1bcb9aa6e4 bundle2: add op.gettransaction() to handlers that need the lock
Durham Goode <durham@fb.com>
parents: 26542
diff changeset
  2342
    # pushkey.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2343
    if op.ui.configbool(b'experimental', b'bundle2lazylocking'):
26565
ee1bcb9aa6e4 bundle2: add op.gettransaction() to handlers that need the lock
Durham Goode <durham@fb.com>
parents: 26542
diff changeset
  2344
        op.gettransaction()
21660
e87d2a12d41b bundle2: add ``pushkey`` support
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21655
diff changeset
  2345
    ret = op.repo.pushkey(namespace, key, old, new)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2346
    record = {b'namespace': namespace, b'key': key, b'old': old, b'new': new}
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2347
    op.records.add(b'pushkey', record)
21660
e87d2a12d41b bundle2: add ``pushkey`` support
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21655
diff changeset
  2348
    if op.reply is not None:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2349
        rpart = op.reply.newpart(b'reply:pushkey')
33701
24b7c75c6441 bundle2: use bytestr() instead of str() to convert part id to bytes
Augie Fackler <augie@google.com>
parents: 33700
diff changeset
  2350
        rpart.addparam(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2351
            b'in-reply-to', pycompat.bytestr(inpart.id), mandatory=False
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2352
        )
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2353
        rpart.addparam(b'return', b'%i' % ret, mandatory=False)
25481
6de96cb31176 bundle2: abort when a mandatory pushkey part fails
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25401
diff changeset
  2354
    if inpart.mandatory and not ret:
25484
a5192774e925 bundle2: introduce a PushkeyFail error to abort unbundle on pushkey error
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25481
diff changeset
  2355
        kwargs = {}
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2356
        for key in (b'namespace', b'key', b'new', b'old', b'ret'):
25484
a5192774e925 bundle2: introduce a PushkeyFail error to abort unbundle on pushkey error
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25481
diff changeset
  2357
            if key in inpart.params:
a5192774e925 bundle2: introduce a PushkeyFail error to abort unbundle on pushkey error
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25481
diff changeset
  2358
                kwargs[key] = inpart.params[key]
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2359
        raise error.PushkeyFailed(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2360
            partid=b'%d' % inpart.id, **pycompat.strkwargs(kwargs)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2361
        )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2362
21660
e87d2a12d41b bundle2: add ``pushkey`` support
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21655
diff changeset
  2363
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2364
@parthandler(b'bookmarks')
35269
af5507203d01 bookmark: introduce a 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35268
diff changeset
  2365
def handlebookmark(op, inpart):
af5507203d01 bookmark: introduce a 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35268
diff changeset
  2366
    """transmit bookmark information
af5507203d01 bookmark: introduce a 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35268
diff changeset
  2367
35275
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35274
diff changeset
  2368
    The part contains binary encoded bookmark information.
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35274
diff changeset
  2369
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35274
diff changeset
  2370
    The exact behavior of this part can be controlled by the 'bookmarks' mode
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35274
diff changeset
  2371
    on the bundle operation.
35269
af5507203d01 bookmark: introduce a 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35268
diff changeset
  2372
35275
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35274
diff changeset
  2373
    When mode is 'apply' (the default) the bookmark information is applied as
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35274
diff changeset
  2374
    is to the unbundling repository. Make sure a 'check:bookmarks' part is
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35274
diff changeset
  2375
    issued earlier to check for push races in such update. This behavior is
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35274
diff changeset
  2376
    suitable for pushing.
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35274
diff changeset
  2377
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35274
diff changeset
  2378
    When mode is 'records', the information is recorded into the 'bookmarks'
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35274
diff changeset
  2379
    records of the bundle operation. This behavior is suitable for pulling.
35269
af5507203d01 bookmark: introduce a 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35268
diff changeset
  2380
    """
46793
6266d19556ad node: introduce nodeconstants class
Joerg Sonnenberger <joerg@bec.de>
parents: 46726
diff changeset
  2381
    changes = bookmarks.binarydecode(op.repo, inpart)
35270
f392066d127c bookmark: add pushkey hook compatiblity to the bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35269
diff changeset
  2382
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2383
    pushkeycompat = op.repo.ui.configbool(
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2384
        b'server', b'bookmarks-pushkey-compat'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2385
    )
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2386
    bookmarksmode = op.modes.get(b'bookmarks', b'apply')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2387
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2388
    if bookmarksmode == b'apply':
35275
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35274
diff changeset
  2389
        tr = op.gettransaction()
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35274
diff changeset
  2390
        bookstore = op.repo._bookmarks
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35274
diff changeset
  2391
        if pushkeycompat:
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35274
diff changeset
  2392
            allhooks = []
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35274
diff changeset
  2393
            for book, node in changes:
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35274
diff changeset
  2394
                hookargs = tr.hookargs.copy()
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2395
                hookargs[b'pushkeycompat'] = b'1'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2396
                hookargs[b'namespace'] = b'bookmarks'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2397
                hookargs[b'key'] = book
46114
59fa3890d40a node: import symbols explicitly
Joerg Sonnenberger <joerg@bec.de>
parents: 45957
diff changeset
  2398
                hookargs[b'old'] = hex(bookstore.get(book, b''))
59fa3890d40a node: import symbols explicitly
Joerg Sonnenberger <joerg@bec.de>
parents: 45957
diff changeset
  2399
                hookargs[b'new'] = hex(node if node is not None else b'')
35275
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35274
diff changeset
  2400
                allhooks.append(hookargs)
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35274
diff changeset
  2401
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35274
diff changeset
  2402
            for hookargs in allhooks:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2403
                op.repo.hook(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2404
                    b'prepushkey', throw=True, **pycompat.strkwargs(hookargs)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2405
                )
35275
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35274
diff changeset
  2406
44406
8407031f195f bookmarks: prevent pushes of divergent bookmarks (foo@remote)
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 44173
diff changeset
  2407
        for book, node in changes:
8407031f195f bookmarks: prevent pushes of divergent bookmarks (foo@remote)
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 44173
diff changeset
  2408
            if bookmarks.isdivergent(book):
8407031f195f bookmarks: prevent pushes of divergent bookmarks (foo@remote)
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 44173
diff changeset
  2409
                msg = _(b'cannot accept divergent bookmark %s!') % book
8407031f195f bookmarks: prevent pushes of divergent bookmarks (foo@remote)
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 44173
diff changeset
  2410
                raise error.Abort(msg)
8407031f195f bookmarks: prevent pushes of divergent bookmarks (foo@remote)
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 44173
diff changeset
  2411
35275
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35274
diff changeset
  2412
        bookstore.applychanges(op.repo, op.gettransaction(), changes)
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35274
diff changeset
  2413
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35274
diff changeset
  2414
        if pushkeycompat:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2415
43798
888bd39ed555 lock: pass "success" boolean to _afterlock callbacks
Kyle Lippincott <spectral@google.com>
parents: 43554
diff changeset
  2416
            def runhook(unused_success):
35275
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35274
diff changeset
  2417
                for hookargs in allhooks:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2418
                    op.repo.hook(b'pushkey', **pycompat.strkwargs(hookargs))
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2419
35275
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35274
diff changeset
  2420
            op.repo._afterlock(runhook)
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35274
diff changeset
  2421
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2422
    elif bookmarksmode == b'records':
35270
f392066d127c bookmark: add pushkey hook compatiblity to the bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35269
diff changeset
  2423
        for book, node in changes:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2424
            record = {b'bookmark': book, b'node': node}
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2425
            op.records.add(b'bookmarks', record)
35275
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35274
diff changeset
  2426
    else:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2427
        raise error.ProgrammingError(
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2428
            b'unkown bookmark mode: %s' % bookmarksmode
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2429
        )
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2430
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2431
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2432
@parthandler(b'phase-heads')
33043
e8c8d81eb864 bundle: add config option to include phases
Martin von Zweigbergk <martinvonz@google.com>
parents: 33042
diff changeset
  2433
def handlephases(op, inpart):
e8c8d81eb864 bundle: add config option to include phases
Martin von Zweigbergk <martinvonz@google.com>
parents: 33042
diff changeset
  2434
    """apply phases from bundle part to repo"""
34327
12c42bcd4133 phases: move the binary decoding function in the phases module
Boris Feld <boris.feld@octobus.net>
parents: 34326
diff changeset
  2435
    headsbyphase = phases.binarydecode(inpart)
34328
4ef472b975ff bundle2: only grab a transaction when 'phase-heads' affect the repository
Boris Feld <boris.feld@octobus.net>
parents: 34327
diff changeset
  2436
    phases.updatephases(op.repo.unfiltered(), op.gettransaction, headsbyphase)
33043
e8c8d81eb864 bundle: add config option to include phases
Martin von Zweigbergk <martinvonz@google.com>
parents: 33042
diff changeset
  2437
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2438
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2439
@parthandler(b'reply:pushkey', (b'return', b'in-reply-to'))
21660
e87d2a12d41b bundle2: add ``pushkey`` support
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21655
diff changeset
  2440
def handlepushkeyreply(op, inpart):
e87d2a12d41b bundle2: add ``pushkey`` support
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21655
diff changeset
  2441
    """retrieve the result of a pushkey request"""
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2442
    ret = int(inpart.params[b'return'])
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2443
    partid = int(inpart.params[b'in-reply-to'])
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2444
    op.records.add(b'pushkey', {b'return': ret}, partid)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2445
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2446
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2447
@parthandler(b'obsmarkers')
22336
60786c8a2f70 bundle2: add an obsmarkers part handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21900
diff changeset
  2448
def handleobsmarker(op, inpart):
60786c8a2f70 bundle2: add an obsmarkers part handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21900
diff changeset
  2449
    """add a stream of obsmarkers to the repo"""
60786c8a2f70 bundle2: add an obsmarkers part handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21900
diff changeset
  2450
    tr = op.gettransaction()
24733
c00e4338fa4b obsolete: experimental flag to get debug about obsmarkers exchange
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24686
diff changeset
  2451
    markerdata = inpart.read()
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2452
    if op.ui.config(b'experimental', b'obsmarkers-exchange-debug'):
43080
86e4daa2d54c cleanup: mark some ui.(status|note|warn|write) calls as not needing i18n
Augie Fackler <augie@google.com>
parents: 43077
diff changeset
  2453
        op.ui.writenoi18n(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2454
            b'obsmarker-exchange: %i bytes received\n' % len(markerdata)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2455
        )
26685
ea390d889d3a bundle2: gracefully skip 'obsmarkers' part if evolution is disabled
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26587
diff changeset
  2456
    # The mergemarkers call will crash if marker creation is not enabled.
ea390d889d3a bundle2: gracefully skip 'obsmarkers' part if evolution is disabled
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26587
diff changeset
  2457
    # we want to avoid this if the part is advisory.
ea390d889d3a bundle2: gracefully skip 'obsmarkers' part if evolution is disabled
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26587
diff changeset
  2458
    if not inpart.mandatory and op.repo.obsstore.readonly:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2459
        op.repo.ui.debug(
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2460
            b'ignoring obsolescence markers, feature not enabled\n'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2461
        )
26685
ea390d889d3a bundle2: gracefully skip 'obsmarkers' part if evolution is disabled
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26587
diff changeset
  2462
        return
24733
c00e4338fa4b obsolete: experimental flag to get debug about obsmarkers exchange
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24686
diff changeset
  2463
    new = op.repo.obsstore.mergemarkers(tr, markerdata)
32124
99515353c72a obsolete: invalidate "volatile" set cache after merging marker
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32024
diff changeset
  2464
    op.repo.invalidatevolatilesets()
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2465
    op.records.add(b'obsmarkers', {b'new': new})
22340
394a17de6a2d obsmarker: produce a reply part for markers received through bundle2
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22338
diff changeset
  2466
    if op.reply is not None:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2467
        rpart = op.reply.newpart(b'reply:obsmarkers')
33701
24b7c75c6441 bundle2: use bytestr() instead of str() to convert part id to bytes
Augie Fackler <augie@google.com>
parents: 33700
diff changeset
  2468
        rpart.addparam(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2469
            b'in-reply-to', pycompat.bytestr(inpart.id), mandatory=False
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2470
        )
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2471
        rpart.addparam(b'new', b'%i' % new, mandatory=False)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2472
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2473
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2474
@parthandler(b'reply:obsmarkers', (b'new', b'in-reply-to'))
25506
86472038dfd2 bundle2: rename duplicate handlepushkeyreply to handleobsmarkerreply
Martin von Zweigbergk <martinvonz@google.com>
parents: 25493
diff changeset
  2475
def handleobsmarkerreply(op, inpart):
22340
394a17de6a2d obsmarker: produce a reply part for markers received through bundle2
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22338
diff changeset
  2476
    """retrieve the result of a pushkey request"""
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2477
    ret = int(inpart.params[b'new'])
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2478
    partid = int(inpart.params[b'in-reply-to'])
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2479
    op.records.add(b'obsmarkers', {b'new': ret}, partid)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2480
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2481
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2482
@parthandler(b'hgtagsfnodes')
25401
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
  2483
def handlehgtagsfnodes(op, inpart):
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
  2484
    """Applies .hgtags fnodes cache entries to the local repo.
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
  2485
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
  2486
    Payload is pairs of 20 byte changeset nodes and filenodes.
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
  2487
    """
26565
ee1bcb9aa6e4 bundle2: add op.gettransaction() to handlers that need the lock
Durham Goode <durham@fb.com>
parents: 26542
diff changeset
  2488
    # Grab the transaction so we ensure that we have the lock at this point.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2489
    if op.ui.configbool(b'experimental', b'bundle2lazylocking'):
26565
ee1bcb9aa6e4 bundle2: add op.gettransaction() to handlers that need the lock
Durham Goode <durham@fb.com>
parents: 26542
diff changeset
  2490
        op.gettransaction()
25401
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
  2491
    cache = tags.hgtagsfnodescache(op.repo.unfiltered())
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
  2492
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
  2493
    count = 0
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
  2494
    while True:
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
  2495
        node = inpart.read(20)
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
  2496
        fnode = inpart.read(20)
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
  2497
        if len(node) < 20 or len(fnode) < 20:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2498
            op.ui.debug(b'ignoring incomplete received .hgtags fnodes data\n')
25401
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
  2499
            break
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
  2500
        cache.setfnode(node, fnode)
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
  2501
        count += 1
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
  2502
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
  2503
    cache.write()
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2504
    op.ui.debug(b'applied %i hgtags fnodes cache entries\n' % count)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2505
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2506
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2507
rbcstruct = struct.Struct(b'>III')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2508
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2509
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2510
@parthandler(b'cache:rev-branch-cache')
36969
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part
Boris Feld <boris.feld@octobus.net>
parents: 36458
diff changeset
  2511
def handlerbc(op, inpart):
46444
3e91d9978bec branchmap: update rev-branch-cache incrementally
Joerg Sonnenberger <joerg@bec.de>
parents: 46166
diff changeset
  2512
    """Legacy part, ignored for compatibility with bundles from or
3e91d9978bec branchmap: update rev-branch-cache incrementally
Joerg Sonnenberger <joerg@bec.de>
parents: 46166
diff changeset
  2513
    for Mercurial before 5.7. Newer Mercurial computes the cache
3e91d9978bec branchmap: update rev-branch-cache incrementally
Joerg Sonnenberger <joerg@bec.de>
parents: 46166
diff changeset
  2514
    efficiently enough during unbundling that the additional transfer
3e91d9978bec branchmap: update rev-branch-cache incrementally
Joerg Sonnenberger <joerg@bec.de>
parents: 46166
diff changeset
  2515
    is unnecessary."""
36969
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part
Boris Feld <boris.feld@octobus.net>
parents: 36458
diff changeset
  2516
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2517
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2518
@parthandler(b'pushvars')
33719
db3dc11356ed pushvars: move fb extension pushvars to core
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33708
diff changeset
  2519
def bundle2getvars(op, part):
db3dc11356ed pushvars: move fb extension pushvars to core
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33708
diff changeset
  2520
    '''unbundle a bundle2 containing shellvars on the server'''
db3dc11356ed pushvars: move fb extension pushvars to core
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33708
diff changeset
  2521
    # An option to disable unbundling on server-side for security reasons
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2522
    if op.ui.configbool(b'push', b'pushvars.server'):
33719
db3dc11356ed pushvars: move fb extension pushvars to core
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33708
diff changeset
  2523
        hookargs = {}
db3dc11356ed pushvars: move fb extension pushvars to core
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33708
diff changeset
  2524
        for key, value in part.advisoryparams:
db3dc11356ed pushvars: move fb extension pushvars to core
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33708
diff changeset
  2525
            key = key.upper()
db3dc11356ed pushvars: move fb extension pushvars to core
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33708
diff changeset
  2526
            # We want pushed variables to have USERVAR_ prepended so we know
db3dc11356ed pushvars: move fb extension pushvars to core
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33708
diff changeset
  2527
            # they came from the --pushvar flag.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2528
            key = b"USERVAR_" + key
33719
db3dc11356ed pushvars: move fb extension pushvars to core
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33708
diff changeset
  2529
            hookargs[key] = value
db3dc11356ed pushvars: move fb extension pushvars to core
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33708
diff changeset
  2530
        op.addhookargs(hookargs)
35758
b996ddf5963d bundle2: add a 'stream' part handler for stream cloning
Boris Feld <boris.feld@octobus.net>
parents: 35655
diff changeset
  2531
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2532
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2533
@parthandler(b'stream2', (b'requirements', b'filecount', b'bytecount'))
35788
b116a66bcc44 bundle2: move version of stream clone into part name
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35787
diff changeset
  2534
def handlestreamv2bundle(op, part):
35758
b996ddf5963d bundle2: add a 'stream' part handler for stream cloning
Boris Feld <boris.feld@octobus.net>
parents: 35655
diff changeset
  2535
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2536
    requirements = urlreq.unquote(part.params[b'requirements']).split(b',')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2537
    filecount = int(part.params[b'filecount'])
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2538
    bytecount = int(part.params[b'bytecount'])
35758
b996ddf5963d bundle2: add a 'stream' part handler for stream cloning
Boris Feld <boris.feld@octobus.net>
parents: 35655
diff changeset
  2539
b996ddf5963d bundle2: add a 'stream' part handler for stream cloning
Boris Feld <boris.feld@octobus.net>
parents: 35655
diff changeset
  2540
    repo = op.repo
b996ddf5963d bundle2: add a 'stream' part handler for stream cloning
Boris Feld <boris.feld@octobus.net>
parents: 35655
diff changeset
  2541
    if len(repo):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2542
        msg = _(b'cannot apply stream clone to non empty repository')
35758
b996ddf5963d bundle2: add a 'stream' part handler for stream cloning
Boris Feld <boris.feld@octobus.net>
parents: 35655
diff changeset
  2543
        raise error.Abort(msg)
b996ddf5963d bundle2: add a 'stream' part handler for stream cloning
Boris Feld <boris.feld@octobus.net>
parents: 35655
diff changeset
  2544
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2545
    repo.ui.debug(b'applying stream bundle\n')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2546
    streamclone.applybundlev2(repo, part, filecount, bytecount, requirements)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2547
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2548
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2549
def widen_bundle(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2550
    bundler, repo, oldmatcher, newmatcher, common, known, cgversion, ellipses
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2551
):
40073
1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39855
diff changeset
  2552
    """generates bundle2 for widening a narrow clone
1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39855
diff changeset
  2553
42434
a97b12f726e4 narrow: pass the bundle to bundle2.widen_bundle() instead of generating there
Pulkit Goyal <7895pulkit@gmail.com>
parents: 42143
diff changeset
  2554
    bundler is the bundle to which data should be added
40073
1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39855
diff changeset
  2555
    repo is the localrepository instance
40344
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40338
diff changeset
  2556
    oldmatcher matches what the client already has
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40338
diff changeset
  2557
    newmatcher matches what the client needs (including what it already has)
40073
1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39855
diff changeset
  2558
    common is set of common heads between server and client
1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39855
diff changeset
  2559
    known is a set of revs known on the client side (used in ellipses)
1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39855
diff changeset
  2560
    cgversion is the changegroup version to send
1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39855
diff changeset
  2561
    ellipses is boolean value telling whether to send ellipses data or not
1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39855
diff changeset
  2562
1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39855
diff changeset
  2563
    returns bundle2 of the data required for extending
1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39855
diff changeset
  2564
    """
1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39855
diff changeset
  2565
    commonnodes = set()
1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39855
diff changeset
  2566
    cl = repo.changelog
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2567
    for r in repo.revs(b"::%ln", common):
40073
1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39855
diff changeset
  2568
        commonnodes.add(cl.node(r))
1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39855
diff changeset
  2569
    if commonnodes:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2570
        packer = changegroup.getbundler(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2571
            cgversion,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2572
            repo,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2573
            oldmatcher=oldmatcher,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2574
            matcher=newmatcher,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2575
            fullnodes=commonnodes,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2576
        )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2577
        cgdata = packer.generate(
47055
d55b71393907 node: replace nullid and friends with nodeconstants class
Joerg Sonnenberger <joerg@bec.de>
parents: 46907
diff changeset
  2578
            {repo.nullid},
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2579
            list(commonnodes),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2580
            False,
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2581
            b'narrow_widen',
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2582
            changelog=False,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42935
diff changeset
  2583
        )
40073
1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39855
diff changeset
  2584
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2585
        part = bundler.newpart(b'changegroup', data=cgdata)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2586
        part.addparam(b'version', cgversion)
45558
10284ce3d5ed scmutil: introduce function to check whether repo uses treemanifest or not
Pulkit Goyal <7895pulkit@gmail.com>
parents: 45392
diff changeset
  2587
        if scmutil.istreemanifest(repo):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  2588
            part.addparam(b'treemanifest', b'1')
43131
c17a63eb5d4c sidedata: apply basic but tight security around exchange
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43117
diff changeset
  2589
        if b'exp-sidedata-flag' in repo.requirements:
c17a63eb5d4c sidedata: apply basic but tight security around exchange
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43117
diff changeset
  2590
            part.addparam(b'exp-sidedata', b'1')
46726
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos
Raphaël Gomès <rgomes@octobus.net>
parents: 46672
diff changeset
  2591
            wanted = format_remote_wanted_sidedata(repo)
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos
Raphaël Gomès <rgomes@octobus.net>
parents: 46672
diff changeset
  2592
            part.addparam(b'exp-wanted-sidedata', wanted)
40073
1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39855
diff changeset
  2593
1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39855
diff changeset
  2594
    return bundler