hgext/clonebundles.py
author Matt Harbison <matt_harbison@yahoo.com>
Thu, 10 Oct 2024 17:52:26 -0400
changeset 52007 521df44118a8
parent 51863 f4733654f144
child 52096 93484d43be22
permissions -rw-r--r--
tests: allow optional output when `test-lfs-server` is sent SIGTERM I guess this has only ever been tested on Windows, but on macOS, the test was failing with changes like: +++ /private/tmp/mercurial-ci/tests/test-lfs-test-server.t#git-server.err @@ -858,6 +858,7 @@ (Restart the server in a different location so it no longer has the content) $ "$PYTHON" $RUNTESTDIR/killdaemons.py $DAEMON_PIDS + $TESTTMP.sh: line 153: 38639 Terminated: 15 lfs-test-server > lfs-server.log 2>&1 #if hg-server $ cat $TESTTMP/access.log $TESTTMP/errors.log
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
26623
5a95fe44121d clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
     1
# This software may be used and distributed according to the terms of the
5a95fe44121d clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
     2
# GNU General Public License version 2 or any later version.
5a95fe44121d clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
     3
27738
a0e783d26e81 exchange: make clone bundles non-experimental and enabled by default
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27737
diff changeset
     4
"""advertise pre-generated bundles to seed clones
26762
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
     5
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
     6
"clonebundles" is a server-side extension used to advertise the existence
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
     7
of pre-generated, externally hosted bundle files to clients that are
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
     8
cloning so that cloning can be faster, more reliable, and require less
37498
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 32773
diff changeset
     9
resources on the server. "pullbundles" is a related feature for sending
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 32773
diff changeset
    10
pre-generated bundle files to clients as part of pull operations.
26762
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
    11
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
    12
Cloning can be a CPU and I/O intensive operation on servers. Traditionally,
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
    13
the server, in response to a client's request to clone, dynamically generates
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
    14
a bundle containing the entire repository content and sends it to the client.
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
    15
There is no caching on the server and the server will have to redundantly
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
    16
generate the same outgoing bundle in response to each clone request. For
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
    17
servers with large repositories or with high clone volume, the load from
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
    18
clones can make scaling the server challenging and costly.
26623
5a95fe44121d clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    19
37498
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 32773
diff changeset
    20
This extension provides server operators the ability to offload
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 32773
diff changeset
    21
potentially expensive clone load to an external service. Pre-generated
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 32773
diff changeset
    22
bundles also allow using more CPU intensive compression, reducing the
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 32773
diff changeset
    23
effective bandwidth requirements.
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 32773
diff changeset
    24
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 32773
diff changeset
    25
Here's how clone bundles work:
26762
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
    26
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
    27
1. A server operator establishes a mechanism for making bundle files available
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
    28
   on a hosting service where Mercurial clients can fetch them.
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
    29
2. A manifest file listing available bundle URLs and some optional metadata
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
    30
   is added to the Mercurial repository on the server.
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
    31
3. A client initiates a clone against a clone bundles aware server.
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
    32
4. The client sees the server is advertising clone bundles and fetches the
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
    33
   manifest listing available bundles.
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
    34
5. The client filters and sorts the available bundles based on what it
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
    35
   supports and prefers.
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
    36
6. The client downloads and applies an available bundle from the
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
    37
   server-specified URL.
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
    38
7. The client reconnects to the original server and performs the equivalent
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
    39
   of :hg:`pull` to retrieve all repository data not in the bundle. (The
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
    40
   repository could have been updated between when the bundle was created
37498
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 32773
diff changeset
    41
   and when the client started the clone.) This may use "pullbundles".
26623
5a95fe44121d clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    42
26762
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
    43
Instead of the server generating full repository bundles for every clone
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
    44
request, it generates full bundles once and they are subsequently reused to
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
    45
bootstrap new clones. The server may still transfer data at clone time.
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
    46
However, this is only data that has been added/changed since the bundle was
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
    47
created. For large, established repositories, this can reduce server load for
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
    48
clones to less than 1% of original.
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
    49
37498
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 32773
diff changeset
    50
Here's how pullbundles work:
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 32773
diff changeset
    51
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 32773
diff changeset
    52
1. A manifest file listing available bundles and describing the revisions
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 32773
diff changeset
    53
   is added to the Mercurial repository on the server.
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 32773
diff changeset
    54
2. A new-enough client informs the server that it supports partial pulls
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 32773
diff changeset
    55
   and initiates a pull.
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 32773
diff changeset
    56
3. If the server has pull bundles enabled and sees the client advertising
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 32773
diff changeset
    57
   partial pulls, it checks for a matching pull bundle in the manifest.
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 32773
diff changeset
    58
   A bundle matches if the format is supported by the client, the client
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 32773
diff changeset
    59
   has the required revisions already and needs something from the bundle.
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 32773
diff changeset
    60
4. If there is at least one matching bundle, the server sends it to the client.
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 32773
diff changeset
    61
5. The client applies the bundle and notices that the server reply was
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 32773
diff changeset
    62
   incomplete. It initiates another pull.
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 32773
diff changeset
    63
26762
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
    64
To work, this extension requires the following of server operators:
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
    65
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
    66
* Generating bundle files of repository content (typically periodically,
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
    67
  such as once per day).
37498
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 32773
diff changeset
    68
* Clone bundles: A file server that clients have network access to and that
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 32773
diff changeset
    69
  Python knows how to talk to through its normal URL handling facility
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 32773
diff changeset
    70
  (typically an HTTP/HTTPS server).
26762
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
    71
* A process for keeping the bundles manifest in sync with available bundle
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
    72
  files.
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
    73
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
    74
Strictly speaking, using a static file hosting server isn't required: a server
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
    75
operator could use a dynamic service for retrieving bundle data. However,
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
    76
static file hosting services are simple and scalable and should be sufficient
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
    77
for most needs.
26623
5a95fe44121d clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    78
26884
762bf510b42c clonebundles: fix typo s/comand/command/
Javi Merino <merino.jav@gmail.com>
parents: 26857
diff changeset
    79
Bundle files can be generated with the :hg:`bundle` command. Typically
26762
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
    80
:hg:`bundle --all` is used to produce a bundle of the entire repository.
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
    81
50645
bf7404f2e22d bundles: clarify streaming v2 bundle usage
Mathias De Mare <mathias.de_mare@nokia.com>
parents: 48875
diff changeset
    82
The bundlespec option `stream` (see :hg:`help bundlespec`)
bf7404f2e22d bundles: clarify streaming v2 bundle usage
Mathias De Mare <mathias.de_mare@nokia.com>
parents: 48875
diff changeset
    83
can be used to produce a special *streaming clonebundle*, typically using
bf7404f2e22d bundles: clarify streaming v2 bundle usage
Mathias De Mare <mathias.de_mare@nokia.com>
parents: 48875
diff changeset
    84
:hg:`bundle --all --type="none-streamv2"`.
bf7404f2e22d bundles: clarify streaming v2 bundle usage
Mathias De Mare <mathias.de_mare@nokia.com>
parents: 48875
diff changeset
    85
These are bundle files that are extremely efficient
26762
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
    86
to produce and consume (read: fast). However, they are larger than
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
    87
traditional bundle formats and require that clients support the exact set
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
    88
of repository data store formats in use by the repository that created them.
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
    89
Typically, a newer server can serve data that is compatible with older clients.
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
    90
However, *streaming clone bundles* don't have this guarantee. **Server
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
    91
operators need to be aware that newer versions of Mercurial may produce
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
    92
streaming clone bundles incompatible with older Mercurial versions.**
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
    93
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
    94
A server operator is responsible for creating a ``.hg/clonebundles.manifest``
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
    95
file containing the list of available bundle files suitable for seeding
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
    96
clones. If this file does not exist, the repository will not advertise the
37498
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 32773
diff changeset
    97
existence of clone bundles when clients connect. For pull bundles,
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 32773
diff changeset
    98
``.hg/pullbundles.manifest`` is used.
26623
5a95fe44121d clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    99
32454
702af1ad3b18 clonebundles: fix missing newline character
Matt Harbison <matt_harbison@yahoo.com>
parents: 31146
diff changeset
   100
The manifest file contains a newline (\\n) delimited list of entries.
26623
5a95fe44121d clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   101
5a95fe44121d clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   102
Each line in this file defines an available bundle. Lines have the format:
5a95fe44121d clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   103
26762
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
   104
    <URL> [<key>=<value>[ <key>=<value>]]
26623
5a95fe44121d clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   105
26762
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
   106
That is, a URL followed by an optional, space-delimited list of key=value
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
   107
pairs describing additional properties of this bundle. Both keys and values
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
   108
are URI encoded.
26623
5a95fe44121d clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   109
37498
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 32773
diff changeset
   110
For pull bundles, the URL is a path under the ``.hg`` directory of the
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 32773
diff changeset
   111
repository.
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 32773
diff changeset
   112
26762
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
   113
Keys in UPPERCASE are reserved for use by Mercurial and are defined below.
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
   114
All non-uppercase keys can be used by site installations. An example use
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
   115
for custom properties is to use the *datacenter* attribute to define which
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
   116
data center a file is hosted in. Clients could then prefer a server in the
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
   117
data center closest to them.
26623
5a95fe44121d clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   118
26762
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
   119
The following reserved keys are currently defined:
26623
5a95fe44121d clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   120
26644
74de1c59f71c clonebundles: filter on bundle specification
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26623
diff changeset
   121
BUNDLESPEC
74de1c59f71c clonebundles: filter on bundle specification
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26623
diff changeset
   122
   A "bundle specification" string that describes the type of the bundle.
74de1c59f71c clonebundles: filter on bundle specification
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26623
diff changeset
   123
74de1c59f71c clonebundles: filter on bundle specification
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26623
diff changeset
   124
   These are string values that are accepted by the "--type" argument of
26762
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
   125
   :hg:`bundle`.
26644
74de1c59f71c clonebundles: filter on bundle specification
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26623
diff changeset
   126
74de1c59f71c clonebundles: filter on bundle specification
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26623
diff changeset
   127
   The values are parsed in strict mode, which means they must be of the
74de1c59f71c clonebundles: filter on bundle specification
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26623
diff changeset
   128
   "<compression>-<type>" form. See
74de1c59f71c clonebundles: filter on bundle specification
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26623
diff changeset
   129
   mercurial.exchange.parsebundlespec() for more details.
74de1c59f71c clonebundles: filter on bundle specification
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26623
diff changeset
   130
27886
0288e63ea3be clonebundles: improve BUNDLESPEC documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27738
diff changeset
   131
   :hg:`debugbundle --spec` can be used to print the bundle specification
0288e63ea3be clonebundles: improve BUNDLESPEC documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27738
diff changeset
   132
   string for a bundle file. The output of this command can be used verbatim
0288e63ea3be clonebundles: improve BUNDLESPEC documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27738
diff changeset
   133
   for the value of ``BUNDLESPEC`` (it is already escaped).
0288e63ea3be clonebundles: improve BUNDLESPEC documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27738
diff changeset
   134
26644
74de1c59f71c clonebundles: filter on bundle specification
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26623
diff changeset
   135
   Clients will automatically filter out specifications that are unknown or
74de1c59f71c clonebundles: filter on bundle specification
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26623
diff changeset
   136
   unsupported so they won't attempt to download something that likely won't
74de1c59f71c clonebundles: filter on bundle specification
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26623
diff changeset
   137
   apply.
74de1c59f71c clonebundles: filter on bundle specification
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26623
diff changeset
   138
74de1c59f71c clonebundles: filter on bundle specification
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26623
diff changeset
   139
   The actual value doesn't impact client behavior beyond filtering:
74de1c59f71c clonebundles: filter on bundle specification
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26623
diff changeset
   140
   clients will still sniff the bundle type from the header of downloaded
74de1c59f71c clonebundles: filter on bundle specification
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26623
diff changeset
   141
   files.
26645
2faa7671a4b3 clonebundles: filter on SNI requirement
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26644
diff changeset
   142
26762
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
   143
   **Use of this key is highly recommended**, as it allows clients to
27886
0288e63ea3be clonebundles: improve BUNDLESPEC documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27738
diff changeset
   144
   easily skip unsupported bundles. If this key is not defined, an old
0288e63ea3be clonebundles: improve BUNDLESPEC documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27738
diff changeset
   145
   client may attempt to apply a bundle that it is incapable of reading.
26762
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
   146
26645
2faa7671a4b3 clonebundles: filter on SNI requirement
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26644
diff changeset
   147
REQUIRESNI
2faa7671a4b3 clonebundles: filter on SNI requirement
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26644
diff changeset
   148
   Whether Server Name Indication (SNI) is required to connect to the URL.
2faa7671a4b3 clonebundles: filter on SNI requirement
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26644
diff changeset
   149
   SNI allows servers to use multiple certificates on the same IP. It is
2faa7671a4b3 clonebundles: filter on SNI requirement
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26644
diff changeset
   150
   somewhat common in CDNs and other hosting providers. Older Python
2faa7671a4b3 clonebundles: filter on SNI requirement
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26644
diff changeset
   151
   versions do not support SNI. Defining this attribute enables clients
26762
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
   152
   with older Python versions to filter this entry without experiencing
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
   153
   an opaque SSL failure at connection time.
26645
2faa7671a4b3 clonebundles: filter on SNI requirement
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26644
diff changeset
   154
2faa7671a4b3 clonebundles: filter on SNI requirement
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26644
diff changeset
   155
   If this is defined, it is important to advertise a non-SNI fallback
2faa7671a4b3 clonebundles: filter on SNI requirement
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26644
diff changeset
   156
   URL or clients running old Python releases may not be able to clone
2faa7671a4b3 clonebundles: filter on SNI requirement
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26644
diff changeset
   157
   with the clonebundles facility.
2faa7671a4b3 clonebundles: filter on SNI requirement
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26644
diff changeset
   158
2faa7671a4b3 clonebundles: filter on SNI requirement
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26644
diff changeset
   159
   Value should be "true".
26762
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
   160
45252
20d110e6eea6 clonebundles: document REQUIREDRAM key
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43077
diff changeset
   161
REQUIREDRAM
20d110e6eea6 clonebundles: document REQUIREDRAM key
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43077
diff changeset
   162
   Value specifies expected memory requirements to decode the payload.
20d110e6eea6 clonebundles: document REQUIREDRAM key
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43077
diff changeset
   163
   Values can have suffixes for common bytes sizes. e.g. "64MB".
20d110e6eea6 clonebundles: document REQUIREDRAM key
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43077
diff changeset
   164
20d110e6eea6 clonebundles: document REQUIREDRAM key
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43077
diff changeset
   165
   This key is often used with zstd-compressed bundles using a high
20d110e6eea6 clonebundles: document REQUIREDRAM key
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43077
diff changeset
   166
   compression level / window size, which can require 100+ MB of memory
20d110e6eea6 clonebundles: document REQUIREDRAM key
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43077
diff changeset
   167
   to decode.
20d110e6eea6 clonebundles: document REQUIREDRAM key
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43077
diff changeset
   168
37498
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 32773
diff changeset
   169
heads
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 32773
diff changeset
   170
   Used for pull bundles. This contains the ``;`` separated changeset
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 32773
diff changeset
   171
   hashes of the heads of the bundle content.
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 32773
diff changeset
   172
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 32773
diff changeset
   173
bases
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 32773
diff changeset
   174
   Used for pull bundles. This contains the ``;`` separated changeset
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 32773
diff changeset
   175
   hashes of the roots of the bundle content. This can be skipped if
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 32773
diff changeset
   176
   the bundle was created without ``--base``.
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 32773
diff changeset
   177
26762
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
   178
Manifests can contain multiple entries. Assuming metadata is defined, clients
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
   179
will filter entries from the manifest that they don't support. The remaining
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
   180
entries are optionally sorted by client preferences
32773
d25802b0eef5 clonebundles: reference correct config option
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32454
diff changeset
   181
(``ui.clonebundleprefers`` config option). The client then attempts
26762
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
   182
to fetch the bundle at the first URL in the remaining list.
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
   183
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
   184
**Errors when downloading a bundle will fail the entire clone operation:
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
   185
clients do not automatically fall back to a traditional clone.** The reason
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
   186
for this is that if a server is using clone bundles, it is probably doing so
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
   187
because the feature is necessary to help it scale. In other words, there
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
   188
is an assumption that clone load will be offloaded to another service and
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
   189
that the Mercurial server isn't responsible for serving this clone load.
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
   190
If that other service experiences issues and clients start mass falling back to
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
   191
the original Mercurial server, the added clone load could overwhelm the server
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
   192
due to unexpected load and effectively take it offline. Not having clients
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
   193
automatically fall back to cloning from the original server mitigates this
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
   194
scenario.
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
   195
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
   196
Because there is no automatic Mercurial server fallback on failure of the
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
   197
bundle hosting service, it is important for server operators to view the bundle
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
   198
hosting service as an extension of the Mercurial server in terms of
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
   199
availability and service level agreements: if the bundle hosting service goes
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
   200
down, so does the ability for clients to clone. Note: clients will see a
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
   201
message informing them how to bypass the clone bundles facility when a failure
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
   202
occurs. So server operators should prepare for some people to follow these
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
   203
instructions when a failure occurs, thus driving more load to the original
26f622859288 clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26691
diff changeset
   204
Mercurial server when the bundle hosting service fails.
50686
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   205
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   206
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   207
inline clonebundles
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   208
-------------------
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   209
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   210
It is possible to transmit clonebundles inline in case repositories are
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   211
accessed over SSH. This avoids having to setup an external HTTPS server
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   212
and results in the same access control as already present for the SSH setup.
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   213
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   214
Inline clonebundles should be placed into the `.hg/bundle-cache` directory.
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   215
A clonebundle at `.hg/bundle-cache/mybundle.bundle` is referred to
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   216
in the `clonebundles.manifest` file as `peer-bundle-cache://mybundle.bundle`.
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   217
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   218
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   219
auto-generation of clone bundles
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   220
--------------------------------
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   221
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   222
It is possible to set Mercurial to automatically re-generate clone bundles when
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   223
enough new content is available.
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   224
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   225
Mercurial will take care of the process asynchronously. The defined list of
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   226
bundle-type will be generated, uploaded, and advertised. Older bundles will get
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   227
decommissioned as newer ones replace them.
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   228
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   229
Bundles Generation:
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   230
...................
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   231
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   232
The extension can generate multiple variants of the clone bundle. Each
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   233
different variant will be defined by the "bundle-spec" they use::
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   234
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   235
    [clone-bundles]
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   236
    auto-generate.formats= zstd-v2, gzip-v2
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   237
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   238
See `hg help bundlespec` for details about available options.
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   239
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   240
By default, new bundles are generated when 5% of the repository contents or at
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   241
least 1000 revisions are not contained in the cached bundles. This option can
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   242
be controlled by the `clone-bundles.trigger.below-bundled-ratio` option
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   243
(default 0.95) and the `clone-bundles.trigger.revs` option (default 1000)::
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   244
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   245
    [clone-bundles]
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   246
    trigger.below-bundled-ratio=0.95
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   247
    trigger.revs=1000
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   248
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   249
This logic can be manually triggered using the `admin::clone-bundles-refresh`
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   250
command, or automatically on each repository change if
50735
2b0598121a71 clonebundles: fix display of auto-generate.on-change lines
Mathias De Mare <mathias.de_mare@nokia.com>
parents: 50713
diff changeset
   251
`clone-bundles.auto-generate.on-change` is set to `yes`::
50686
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   252
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   253
    [clone-bundles]
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   254
    auto-generate.on-change=yes
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   255
    auto-generate.formats= zstd-v2, gzip-v2
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   256
50711
40638610c6ee clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50710
diff changeset
   257
Automatic Inline serving
40638610c6ee clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50710
diff changeset
   258
........................
40638610c6ee clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50710
diff changeset
   259
40638610c6ee clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50710
diff changeset
   260
The simplest way to serve the generated bundle is through the Mercurial
40638610c6ee clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50710
diff changeset
   261
protocol. However it is not the most efficient as request will still be served
40638610c6ee clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50710
diff changeset
   262
by that main server. It is useful in case where authentication is complexe or
40638610c6ee clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50710
diff changeset
   263
when an efficient mirror system is already in use anyway. See the `inline
40638610c6ee clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50710
diff changeset
   264
clonebundles` section above for details about inline clonebundles
40638610c6ee clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50710
diff changeset
   265
40638610c6ee clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50710
diff changeset
   266
To automatically serve generated bundle through inline clonebundle, simply set
40638610c6ee clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50710
diff changeset
   267
the following option::
40638610c6ee clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50710
diff changeset
   268
40638610c6ee clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50710
diff changeset
   269
    auto-generate.serve-inline=yes
40638610c6ee clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50710
diff changeset
   270
40638610c6ee clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50710
diff changeset
   271
Enabling this option disable the managed upload and serving explained below.
40638610c6ee clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50710
diff changeset
   272
50686
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   273
Bundles Upload and Serving:
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   274
...........................
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   275
50711
40638610c6ee clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50710
diff changeset
   276
This is the most efficient way to serve automatically generated clone bundles,
40638610c6ee clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50710
diff changeset
   277
but requires some setup.
40638610c6ee clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50710
diff changeset
   278
50686
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   279
The generated bundles need to be made available to users through a "public" URL.
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   280
This should be donne through `clone-bundles.upload-command` configuration. The
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   281
value of this command should be a shell command. It will have access to the
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   282
bundle file path through the `$HGCB_BUNDLE_PATH` variable. And the expected
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   283
basename in the "public" URL is accessible at::
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   284
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   285
  [clone-bundles]
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   286
  upload-command=sftp put $HGCB_BUNDLE_PATH \
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   287
      sftp://bundles.host/clone-bundles/$HGCB_BUNDLE_BASENAME
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   288
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   289
If the file was already uploaded, the command must still succeed.
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   290
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   291
After upload, the file should be available at an url defined by
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   292
`clone-bundles.url-template`.
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   293
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   294
  [clone-bundles]
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   295
  url-template=https://bundles.host/cache/clone-bundles/{basename}
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   296
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   297
Old bundles cleanup:
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   298
....................
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   299
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   300
When new bundles are generated, the older ones are no longer necessary and can
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   301
be removed from storage. This is done through the `clone-bundles.delete-command`
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   302
configuration. The command is given the url of the artifact to delete through
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   303
the `$HGCB_BUNDLE_URL` environment variable.
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   304
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   305
  [clone-bundles]
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   306
  delete-command=sftp rm sftp://bundles.host/clone-bundles/$HGCB_BUNDLE_BASENAME
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   307
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   308
If the file was already deleted, the command must still succeed.
26623
5a95fe44121d clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   309
"""
5a95fe44121d clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   310
51863
f4733654f144 typing: add `from __future__ import annotations` to most files
Matt Harbison <matt_harbison@yahoo.com>
parents: 50928
diff changeset
   311
from __future__ import annotations
28095
7fa139eaebb4 clonebundles: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27886
diff changeset
   312
50686
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   313
import os
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   314
import weakref
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   315
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   316
from mercurial.i18n import _
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   317
26623
5a95fe44121d clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   318
from mercurial import (
45785
80f32ec8653a clonebundle: move the manifest filename to a constant
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45252
diff changeset
   319
    bundlecaches,
50686
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   320
    commands,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   321
    error,
26623
5a95fe44121d clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   322
    extensions,
50686
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   323
    localrepo,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   324
    lock,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   325
    node,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   326
    registrar,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   327
    util,
37785
b4d85bc122bd wireproto: rename wireproto to wireprotov1server (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37498
diff changeset
   328
    wireprotov1server,
26623
5a95fe44121d clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   329
)
5a95fe44121d clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   330
50686
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   331
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   332
from mercurial.utils import (
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   333
    procutil,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   334
)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   335
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   336
testedwith = b'ships-with-hg-core'
26623
5a95fe44121d clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   337
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 37785
diff changeset
   338
26623
5a95fe44121d clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   339
def capabilities(orig, repo, proto):
5a95fe44121d clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   340
    caps = orig(repo, proto)
5a95fe44121d clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   341
5a95fe44121d clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   342
    # Only advertise if a manifest exists. This does add some I/O to requests.
5a95fe44121d clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   343
    # But this should be cheaper than a wasted network round trip due to
5a95fe44121d clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   344
    # missing file.
45785
80f32ec8653a clonebundle: move the manifest filename to a constant
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45252
diff changeset
   345
    if repo.vfs.exists(bundlecaches.CB_MANIFEST_FILE):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   346
        caps.append(b'clonebundles')
50701
4238e6b22fc4 clonebundles: introduce a new write protocol command
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50686
diff changeset
   347
        caps.append(b'clonebundles_manifest')
26623
5a95fe44121d clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   348
5a95fe44121d clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   349
    return caps
5a95fe44121d clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   350
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 37785
diff changeset
   351
26623
5a95fe44121d clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   352
def extsetup(ui):
50776
335aeb3d670b wrapfunction: use sysstr instead of bytes as argument in "clonebundles"
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50736
diff changeset
   353
    extensions.wrapfunction(wireprotov1server, '_capabilities', capabilities)
50686
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   354
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   355
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   356
# logic for bundle auto-generation
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   357
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   358
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   359
configtable = {}
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   360
configitem = registrar.configitem(configtable)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   361
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   362
cmdtable = {}
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   363
command = registrar.command(cmdtable)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   364
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   365
configitem(b'clone-bundles', b'auto-generate.on-change', default=False)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   366
configitem(b'clone-bundles', b'auto-generate.formats', default=list)
50711
40638610c6ee clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50710
diff changeset
   367
configitem(b'clone-bundles', b'auto-generate.serve-inline', default=False)
50686
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   368
configitem(b'clone-bundles', b'trigger.below-bundled-ratio', default=0.95)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   369
configitem(b'clone-bundles', b'trigger.revs', default=1000)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   370
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   371
configitem(b'clone-bundles', b'upload-command', default=None)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   372
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   373
configitem(b'clone-bundles', b'delete-command', default=None)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   374
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   375
configitem(b'clone-bundles', b'url-template', default=None)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   376
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   377
configitem(b'devel', b'debug.clonebundles', default=False)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   378
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   379
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   380
# category for the post-close transaction hooks
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   381
CAT_POSTCLOSE = b"clonebundles-autobundles"
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   382
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   383
# template for bundle file names
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   384
BUNDLE_MASK = (
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   385
    b"full-%(bundle_type)s-%(revs)d_revs-%(tip_short)s_tip-%(op_id)s.hg"
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   386
)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   387
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   388
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   389
# file in .hg/ use to track clonebundles being auto-generated
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   390
AUTO_GEN_FILE = b'clonebundles.auto-gen'
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   391
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   392
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   393
class BundleBase(object):
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   394
    """represents the core of properties that matters for us in a bundle
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   395
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   396
    :bundle_type: the bundlespec (see hg help bundlespec)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   397
    :revs:        the number of revisions in the repo at bundle creation time
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   398
    :tip_rev:     the rev-num of the tip revision
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   399
    :tip_node:    the node id of the tip-most revision in the bundle
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   400
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   401
    :ready:       True if the bundle is ready to be served
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   402
    """
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   403
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   404
    ready = False
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   405
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   406
    def __init__(self, bundle_type, revs, tip_rev, tip_node):
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   407
        self.bundle_type = bundle_type
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   408
        self.revs = revs
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   409
        self.tip_rev = tip_rev
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   410
        self.tip_node = tip_node
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   411
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   412
    def valid_for(self, repo):
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   413
        """is this bundle applicable to the current repository
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   414
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   415
        This is useful for detecting bundles made irrelevant by stripping.
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   416
        """
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   417
        tip_node = node.bin(self.tip_node)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   418
        return repo.changelog.index.get_rev(tip_node) == self.tip_rev
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   419
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   420
    def __eq__(self, other):
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   421
        left = (self.ready, self.bundle_type, self.tip_rev, self.tip_node)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   422
        right = (other.ready, other.bundle_type, other.tip_rev, other.tip_node)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   423
        return left == right
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   424
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   425
    def __neq__(self, other):
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   426
        return not self == other
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   427
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   428
    def __cmp__(self, other):
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   429
        if self == other:
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   430
            return 0
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   431
        return -1
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   432
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   433
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   434
class RequestedBundle(BundleBase):
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   435
    """A bundle that should be generated.
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   436
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   437
    Additional attributes compared to BundleBase
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   438
    :heads:       list of head revisions (as rev-num)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   439
    :op_id:       a "unique" identifier for the operation triggering the change
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   440
    """
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   441
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   442
    def __init__(self, bundle_type, revs, tip_rev, tip_node, head_revs, op_id):
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   443
        self.head_revs = head_revs
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   444
        self.op_id = op_id
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   445
        super(RequestedBundle, self).__init__(
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   446
            bundle_type,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   447
            revs,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   448
            tip_rev,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   449
            tip_node,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   450
        )
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   451
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   452
    @property
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   453
    def suggested_filename(self):
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   454
        """A filename that can be used for the generated bundle"""
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   455
        data = {
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   456
            b'bundle_type': self.bundle_type,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   457
            b'revs': self.revs,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   458
            b'heads': self.head_revs,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   459
            b'tip_rev': self.tip_rev,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   460
            b'tip_node': self.tip_node,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   461
            b'tip_short': self.tip_node[:12],
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   462
            b'op_id': self.op_id,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   463
        }
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   464
        return BUNDLE_MASK % data
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   465
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   466
    def generate_bundle(self, repo, file_path):
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   467
        """generate the bundle at `filepath`"""
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   468
        commands.bundle(
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   469
            repo.ui,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   470
            repo,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   471
            file_path,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   472
            base=[b"null"],
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   473
            rev=self.head_revs,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   474
            type=self.bundle_type,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   475
            quiet=True,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   476
        )
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   477
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   478
    def generating(self, file_path, hostname=None, pid=None):
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   479
        """return a GeneratingBundle object from this object"""
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   480
        if pid is None:
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   481
            pid = os.getpid()
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   482
        if hostname is None:
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   483
            hostname = lock._getlockprefix()
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   484
        return GeneratingBundle(
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   485
            self.bundle_type,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   486
            self.revs,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   487
            self.tip_rev,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   488
            self.tip_node,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   489
            hostname,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   490
            pid,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   491
            file_path,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   492
        )
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   493
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   494
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   495
class GeneratingBundle(BundleBase):
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   496
    """A bundle being generated
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   497
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   498
    extra attributes compared to BundleBase:
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   499
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   500
    :hostname: the hostname of the machine generating the bundle
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   501
    :pid:      the pid of the process generating the bundle
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   502
    :filepath: the target filename of the bundle
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   503
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   504
    These attributes exist to help detect stalled generation processes.
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   505
    """
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   506
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   507
    ready = False
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   508
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   509
    def __init__(
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   510
        self, bundle_type, revs, tip_rev, tip_node, hostname, pid, filepath
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   511
    ):
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   512
        self.hostname = hostname
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   513
        self.pid = pid
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   514
        self.filepath = filepath
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   515
        super(GeneratingBundle, self).__init__(
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   516
            bundle_type, revs, tip_rev, tip_node
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   517
        )
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   518
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   519
    @classmethod
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   520
    def from_line(cls, line):
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   521
        """create an object by deserializing a line from AUTO_GEN_FILE"""
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   522
        assert line.startswith(b'PENDING-v1 ')
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   523
        (
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   524
            __,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   525
            bundle_type,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   526
            revs,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   527
            tip_rev,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   528
            tip_node,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   529
            hostname,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   530
            pid,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   531
            filepath,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   532
        ) = line.split()
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   533
        hostname = util.urlreq.unquote(hostname)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   534
        filepath = util.urlreq.unquote(filepath)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   535
        revs = int(revs)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   536
        tip_rev = int(tip_rev)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   537
        pid = int(pid)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   538
        return cls(
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   539
            bundle_type, revs, tip_rev, tip_node, hostname, pid, filepath
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   540
        )
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   541
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   542
    def to_line(self):
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   543
        """serialize the object to include as a line in AUTO_GEN_FILE"""
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   544
        templ = b"PENDING-v1 %s %d %d %s %s %d %s"
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   545
        data = (
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   546
            self.bundle_type,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   547
            self.revs,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   548
            self.tip_rev,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   549
            self.tip_node,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   550
            util.urlreq.quote(self.hostname),
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   551
            self.pid,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   552
            util.urlreq.quote(self.filepath),
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   553
        )
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   554
        return templ % data
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   555
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   556
    def __eq__(self, other):
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   557
        if not super(GeneratingBundle, self).__eq__(other):
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   558
            return False
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   559
        left = (self.hostname, self.pid, self.filepath)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   560
        right = (other.hostname, other.pid, other.filepath)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   561
        return left == right
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   562
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   563
    def uploaded(self, url, basename):
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   564
        """return a GeneratedBundle from this object"""
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   565
        return GeneratedBundle(
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   566
            self.bundle_type,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   567
            self.revs,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   568
            self.tip_rev,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   569
            self.tip_node,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   570
            url,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   571
            basename,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   572
        )
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   573
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   574
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   575
class GeneratedBundle(BundleBase):
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   576
    """A bundle that is done being generated and can be served
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   577
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   578
    extra attributes compared to BundleBase:
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   579
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   580
    :file_url: the url where the bundle is available.
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   581
    :basename: the "basename" used to upload (useful for deletion)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   582
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   583
    These attributes exist to generate a bundle manifest
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   584
    (.hg/pullbundles.manifest)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   585
    """
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   586
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   587
    ready = True
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   588
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   589
    def __init__(
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   590
        self, bundle_type, revs, tip_rev, tip_node, file_url, basename
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   591
    ):
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   592
        self.file_url = file_url
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   593
        self.basename = basename
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   594
        super(GeneratedBundle, self).__init__(
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   595
            bundle_type, revs, tip_rev, tip_node
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   596
        )
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   597
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   598
    @classmethod
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   599
    def from_line(cls, line):
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   600
        """create an object by deserializing a line from AUTO_GEN_FILE"""
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   601
        assert line.startswith(b'DONE-v1 ')
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   602
        (
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   603
            __,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   604
            bundle_type,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   605
            revs,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   606
            tip_rev,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   607
            tip_node,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   608
            file_url,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   609
            basename,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   610
        ) = line.split()
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   611
        revs = int(revs)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   612
        tip_rev = int(tip_rev)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   613
        file_url = util.urlreq.unquote(file_url)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   614
        return cls(bundle_type, revs, tip_rev, tip_node, file_url, basename)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   615
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   616
    def to_line(self):
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   617
        """serialize the object to include as a line in AUTO_GEN_FILE"""
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   618
        templ = b"DONE-v1 %s %d %d %s %s %s"
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   619
        data = (
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   620
            self.bundle_type,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   621
            self.revs,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   622
            self.tip_rev,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   623
            self.tip_node,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   624
            util.urlreq.quote(self.file_url),
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   625
            self.basename,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   626
        )
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   627
        return templ % data
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   628
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   629
    def manifest_line(self):
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   630
        """serialize the object to include as a line in pullbundles.manifest"""
50710
1299525832d0 clone-bundle: only add the REQUIRESNI bit for http(s)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50701
diff changeset
   631
        templ = b"%s BUNDLESPEC=%s"
1299525832d0 clone-bundle: only add the REQUIRESNI bit for http(s)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50701
diff changeset
   632
        if self.file_url.startswith(b'http'):
1299525832d0 clone-bundle: only add the REQUIRESNI bit for http(s)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50701
diff changeset
   633
            templ += b" REQUIRESNI=true"
50686
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   634
        return templ % (self.file_url, self.bundle_type)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   635
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   636
    def __eq__(self, other):
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   637
        if not super(GeneratedBundle, self).__eq__(other):
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   638
            return False
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   639
        return self.file_url == other.file_url
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   640
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   641
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   642
def parse_auto_gen(content):
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   643
    """parse the AUTO_GEN_FILE to return a list of Bundle object"""
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   644
    bundles = []
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   645
    for line in content.splitlines():
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   646
        if line.startswith(b'PENDING-v1 '):
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   647
            bundles.append(GeneratingBundle.from_line(line))
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   648
        elif line.startswith(b'DONE-v1 '):
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   649
            bundles.append(GeneratedBundle.from_line(line))
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   650
    return bundles
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   651
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   652
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   653
def dumps_auto_gen(bundles):
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   654
    """serialize a list of Bundle as a AUTO_GEN_FILE content"""
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   655
    lines = []
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   656
    for b in bundles:
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   657
        lines.append(b"%s\n" % b.to_line())
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   658
    lines.sort()
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   659
    return b"".join(lines)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   660
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   661
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   662
def read_auto_gen(repo):
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   663
    """read the AUTO_GEN_FILE for the <repo> a list of Bundle object"""
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   664
    data = repo.vfs.tryread(AUTO_GEN_FILE)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   665
    if not data:
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   666
        return []
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   667
    return parse_auto_gen(data)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   668
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   669
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   670
def write_auto_gen(repo, bundles):
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   671
    """write a list of Bundle objects into the repo's AUTO_GEN_FILE"""
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   672
    assert repo._cb_lock_ref is not None
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   673
    data = dumps_auto_gen(bundles)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   674
    with repo.vfs(AUTO_GEN_FILE, mode=b'wb', atomictemp=True) as f:
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   675
        f.write(data)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   676
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   677
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   678
def generate_manifest(bundles):
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   679
    """write a list of Bundle objects into the repo's AUTO_GEN_FILE"""
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   680
    bundles = list(bundles)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   681
    bundles.sort(key=lambda b: b.bundle_type)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   682
    lines = []
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   683
    for b in bundles:
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   684
        lines.append(b"%s\n" % b.manifest_line())
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   685
    return b"".join(lines)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   686
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   687
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   688
def update_ondisk_manifest(repo):
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   689
    """update the clonebundle manifest with latest url"""
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   690
    with repo.clonebundles_lock():
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   691
        bundles = read_auto_gen(repo)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   692
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   693
        per_types = {}
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   694
        for b in bundles:
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   695
            if not (b.ready and b.valid_for(repo)):
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   696
                continue
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   697
            current = per_types.get(b.bundle_type)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   698
            if current is not None and current.revs >= b.revs:
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   699
                continue
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   700
            per_types[b.bundle_type] = b
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   701
        manifest = generate_manifest(per_types.values())
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   702
        with repo.vfs(
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   703
            bundlecaches.CB_MANIFEST_FILE, mode=b"wb", atomictemp=True
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   704
        ) as f:
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   705
            f.write(manifest)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   706
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   707
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   708
def update_bundle_list(repo, new_bundles=(), del_bundles=()):
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   709
    """modify the repo's AUTO_GEN_FILE
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   710
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   711
    This method also regenerates the clone bundle manifest when needed"""
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   712
    with repo.clonebundles_lock():
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   713
        bundles = read_auto_gen(repo)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   714
        if del_bundles:
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   715
            bundles = [b for b in bundles if b not in del_bundles]
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   716
        new_bundles = [b for b in new_bundles if b not in bundles]
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   717
        bundles.extend(new_bundles)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   718
        write_auto_gen(repo, bundles)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   719
        all_changed = []
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   720
        all_changed.extend(new_bundles)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   721
        all_changed.extend(del_bundles)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   722
        if any(b.ready for b in all_changed):
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   723
            update_ondisk_manifest(repo)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   724
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   725
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   726
def cleanup_tmp_bundle(repo, target):
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   727
    """remove a GeneratingBundle file and entry"""
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   728
    assert not target.ready
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   729
    with repo.clonebundles_lock():
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   730
        repo.vfs.tryunlink(target.filepath)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   731
        update_bundle_list(repo, del_bundles=[target])
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   732
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   733
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   734
def finalize_one_bundle(repo, target):
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   735
    """upload a generated bundle and advertise it in the clonebundles.manifest"""
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   736
    with repo.clonebundles_lock():
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   737
        bundles = read_auto_gen(repo)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   738
        if target in bundles and target.valid_for(repo):
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   739
            result = upload_bundle(repo, target)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   740
            update_bundle_list(repo, new_bundles=[result])
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   741
    cleanup_tmp_bundle(repo, target)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   742
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   743
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   744
def find_outdated_bundles(repo, bundles):
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   745
    """finds outdated bundles"""
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   746
    olds = []
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   747
    per_types = {}
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   748
    for b in bundles:
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   749
        if not b.valid_for(repo):
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   750
            olds.append(b)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   751
            continue
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   752
        l = per_types.setdefault(b.bundle_type, [])
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   753
        l.append(b)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   754
    for key in sorted(per_types):
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   755
        all = per_types[key]
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   756
        if len(all) > 1:
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   757
            all.sort(key=lambda b: b.revs, reverse=True)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   758
            olds.extend(all[1:])
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   759
    return olds
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   760
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   761
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   762
def collect_garbage(repo):
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   763
    """finds outdated bundles and get them deleted"""
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   764
    with repo.clonebundles_lock():
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   765
        bundles = read_auto_gen(repo)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   766
        olds = find_outdated_bundles(repo, bundles)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   767
        for o in olds:
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   768
            delete_bundle(repo, o)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   769
        update_bundle_list(repo, del_bundles=olds)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   770
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   771
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   772
def upload_bundle(repo, bundle):
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   773
    """upload the result of a GeneratingBundle and return a GeneratedBundle
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   774
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   775
    The upload is done using the `clone-bundles.upload-command`
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   776
    """
50711
40638610c6ee clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50710
diff changeset
   777
    inline = repo.ui.config(b'clone-bundles', b'auto-generate.serve-inline')
50686
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   778
    basename = repo.vfs.basename(bundle.filepath)
50711
40638610c6ee clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50710
diff changeset
   779
    if inline:
40638610c6ee clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50710
diff changeset
   780
        dest_dir = repo.vfs.join(bundlecaches.BUNDLE_CACHE_DIR)
40638610c6ee clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50710
diff changeset
   781
        repo.vfs.makedirs(dest_dir)
40638610c6ee clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50710
diff changeset
   782
        dest = repo.vfs.join(dest_dir, basename)
40638610c6ee clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50710
diff changeset
   783
        util.copyfiles(bundle.filepath, dest, hardlink=True)
40638610c6ee clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50710
diff changeset
   784
        url = bundlecaches.CLONEBUNDLESCHEME + basename
40638610c6ee clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50710
diff changeset
   785
        return bundle.uploaded(url, basename)
40638610c6ee clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50710
diff changeset
   786
    else:
40638610c6ee clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50710
diff changeset
   787
        cmd = repo.ui.config(b'clone-bundles', b'upload-command')
40638610c6ee clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50710
diff changeset
   788
        url = repo.ui.config(b'clone-bundles', b'url-template')
40638610c6ee clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50710
diff changeset
   789
        filepath = procutil.shellquote(bundle.filepath)
40638610c6ee clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50710
diff changeset
   790
        variables = {
40638610c6ee clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50710
diff changeset
   791
            b'HGCB_BUNDLE_PATH': filepath,
40638610c6ee clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50710
diff changeset
   792
            b'HGCB_BUNDLE_BASENAME': basename,
40638610c6ee clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50710
diff changeset
   793
        }
40638610c6ee clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50710
diff changeset
   794
        env = procutil.shellenviron(environ=variables)
40638610c6ee clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50710
diff changeset
   795
        ret = repo.ui.system(cmd, environ=env)
40638610c6ee clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50710
diff changeset
   796
        if ret:
40638610c6ee clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50710
diff changeset
   797
            raise error.Abort(b"command returned status %d: %s" % (ret, cmd))
40638610c6ee clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50710
diff changeset
   798
        url = (
40638610c6ee clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50710
diff changeset
   799
            url.decode('utf8')
40638610c6ee clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50710
diff changeset
   800
            .format(basename=basename.decode('utf8'))
40638610c6ee clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50710
diff changeset
   801
            .encode('utf8')
40638610c6ee clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50710
diff changeset
   802
        )
40638610c6ee clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50710
diff changeset
   803
        return bundle.uploaded(url, basename)
50686
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   804
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   805
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   806
def delete_bundle(repo, bundle):
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   807
    """delete a bundle from storage"""
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   808
    assert bundle.ready
50711
40638610c6ee clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50710
diff changeset
   809
40638610c6ee clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50710
diff changeset
   810
    inline = bundle.file_url.startswith(bundlecaches.CLONEBUNDLESCHEME)
40638610c6ee clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50710
diff changeset
   811
40638610c6ee clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50710
diff changeset
   812
    if inline:
40638610c6ee clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50710
diff changeset
   813
        msg = b'clone-bundles: deleting inline bundle %s\n'
40638610c6ee clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50710
diff changeset
   814
    else:
40638610c6ee clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50710
diff changeset
   815
        msg = b'clone-bundles: deleting bundle %s\n'
50686
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   816
    msg %= bundle.basename
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   817
    if repo.ui.configbool(b'devel', b'debug.clonebundles'):
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   818
        repo.ui.write(msg)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   819
    else:
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   820
        repo.ui.debug(msg)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   821
50711
40638610c6ee clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50710
diff changeset
   822
    if inline:
40638610c6ee clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50710
diff changeset
   823
        inline_path = repo.vfs.join(
40638610c6ee clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50710
diff changeset
   824
            bundlecaches.BUNDLE_CACHE_DIR,
40638610c6ee clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50710
diff changeset
   825
            bundle.basename,
40638610c6ee clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50710
diff changeset
   826
        )
40638610c6ee clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50710
diff changeset
   827
        util.tryunlink(inline_path)
40638610c6ee clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50710
diff changeset
   828
    else:
40638610c6ee clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50710
diff changeset
   829
        cmd = repo.ui.config(b'clone-bundles', b'delete-command')
40638610c6ee clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50710
diff changeset
   830
        variables = {
40638610c6ee clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50710
diff changeset
   831
            b'HGCB_BUNDLE_URL': bundle.file_url,
40638610c6ee clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50710
diff changeset
   832
            b'HGCB_BASENAME': bundle.basename,
40638610c6ee clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50710
diff changeset
   833
        }
40638610c6ee clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50710
diff changeset
   834
        env = procutil.shellenviron(environ=variables)
40638610c6ee clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50710
diff changeset
   835
        ret = repo.ui.system(cmd, environ=env)
40638610c6ee clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50710
diff changeset
   836
        if ret:
40638610c6ee clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50710
diff changeset
   837
            raise error.Abort(b"command returned status %d: %s" % (ret, cmd))
50686
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   838
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   839
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   840
def auto_bundle_needed_actions(repo, bundles, op_id):
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   841
    """find the list of bundles that need action
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   842
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   843
    returns a list of RequestedBundle objects that need to be generated and
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   844
    uploaded."""
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   845
    create_bundles = []
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   846
    delete_bundles = []
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   847
    repo = repo.filtered(b"immutable")
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   848
    targets = repo.ui.configlist(b'clone-bundles', b'auto-generate.formats')
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   849
    ratio = float(
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   850
        repo.ui.config(b'clone-bundles', b'trigger.below-bundled-ratio')
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   851
    )
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   852
    abs_revs = repo.ui.configint(b'clone-bundles', b'trigger.revs')
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   853
    revs = len(repo.changelog)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   854
    generic_data = {
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   855
        'revs': revs,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   856
        'head_revs': repo.changelog.headrevs(),
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   857
        'tip_rev': repo.changelog.tiprev(),
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   858
        'tip_node': node.hex(repo.changelog.tip()),
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   859
        'op_id': op_id,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   860
    }
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   861
    for t in targets:
50713
ddc55fb220ba clone-bundles: check we generate a valid specification
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50711
diff changeset
   862
        t = bundlecaches.parsebundlespec(repo, t, strict=False).as_spec()
50686
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   863
        if new_bundle_needed(repo, bundles, ratio, abs_revs, t, revs):
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   864
            data = generic_data.copy()
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   865
            data['bundle_type'] = t
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   866
            b = RequestedBundle(**data)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   867
            create_bundles.append(b)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   868
    delete_bundles.extend(find_outdated_bundles(repo, bundles))
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   869
    return create_bundles, delete_bundles
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   870
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   871
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   872
def new_bundle_needed(repo, bundles, ratio, abs_revs, bundle_type, revs):
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   873
    """consider the current cached content and trigger new bundles if needed"""
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   874
    threshold = max((revs * ratio), (revs - abs_revs))
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   875
    for b in bundles:
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   876
        if not b.valid_for(repo) or b.bundle_type != bundle_type:
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   877
            continue
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   878
        if b.revs > threshold:
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   879
            return False
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   880
    return True
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   881
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   882
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   883
def start_one_bundle(repo, bundle):
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   884
    """start the generation of a single bundle file
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   885
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   886
    the `bundle` argument should be a RequestedBundle object.
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   887
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   888
    This data is passed to the `debugmakeclonebundles` "as is".
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   889
    """
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   890
    data = util.pickle.dumps(bundle)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   891
    cmd = [procutil.hgexecutable(), b'--cwd', repo.path, INTERNAL_CMD]
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   892
    env = procutil.shellenviron()
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   893
    msg = b'clone-bundles: starting bundle generation: %s\n'
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   894
    stdout = None
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   895
    stderr = None
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   896
    waits = []
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   897
    record_wait = None
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   898
    if repo.ui.configbool(b'devel', b'debug.clonebundles'):
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   899
        stdout = procutil.stdout
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   900
        stderr = procutil.stderr
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   901
        repo.ui.write(msg % bundle.bundle_type)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   902
        record_wait = waits.append
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   903
    else:
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   904
        repo.ui.debug(msg % bundle.bundle_type)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   905
    bg = procutil.runbgcommand
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   906
    bg(
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   907
        cmd,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   908
        env,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   909
        stdin_bytes=data,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   910
        stdout=stdout,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   911
        stderr=stderr,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   912
        record_wait=record_wait,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   913
    )
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   914
    for f in waits:
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   915
        f()
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   916
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   917
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   918
INTERNAL_CMD = b'debug::internal-make-clone-bundles'
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   919
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   920
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   921
@command(INTERNAL_CMD, [], b'')
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   922
def debugmakeclonebundles(ui, repo):
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   923
    """Internal command to auto-generate debug bundles"""
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   924
    requested_bundle = util.pickle.load(procutil.stdin)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   925
    procutil.stdin.close()
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   926
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   927
    collect_garbage(repo)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   928
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   929
    fname = requested_bundle.suggested_filename
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   930
    fpath = repo.vfs.makedirs(b'tmp-bundles')
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   931
    fpath = repo.vfs.join(b'tmp-bundles', fname)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   932
    bundle = requested_bundle.generating(fpath)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   933
    update_bundle_list(repo, new_bundles=[bundle])
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   934
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   935
    requested_bundle.generate_bundle(repo, fpath)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   936
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   937
    repo.invalidate()
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   938
    finalize_one_bundle(repo, bundle)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   939
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   940
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   941
def make_auto_bundler(source_repo):
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   942
    reporef = weakref.ref(source_repo)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   943
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   944
    def autobundle(tr):
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   945
        repo = reporef()
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   946
        assert repo is not None
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   947
        bundles = read_auto_gen(repo)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   948
        new, __ = auto_bundle_needed_actions(repo, bundles, b"%d_txn" % id(tr))
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   949
        for data in new:
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   950
            start_one_bundle(repo, data)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   951
        return None
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   952
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   953
    return autobundle
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   954
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   955
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   956
def reposetup(ui, repo):
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   957
    """install the two pieces needed for automatic clonebundle generation
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   958
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   959
    - add a "post-close" hook that fires bundling when needed
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   960
    - introduce a clone-bundle lock to let multiple processes meddle with the
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   961
      state files.
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   962
    """
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   963
    if not repo.local():
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   964
        return
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   965
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   966
    class autobundlesrepo(repo.__class__):
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   967
        def transaction(self, *args, **kwargs):
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   968
            tr = super(autobundlesrepo, self).transaction(*args, **kwargs)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   969
            enabled = repo.ui.configbool(
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   970
                b'clone-bundles',
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   971
                b'auto-generate.on-change',
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   972
            )
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   973
            targets = repo.ui.configlist(
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   974
                b'clone-bundles', b'auto-generate.formats'
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   975
            )
50736
b852c34dc2b8 clonebundles: add warning if auto-generate is enabled without formats
Mathias De Mare <mathias.de_mare@nokia.com>
parents: 50735
diff changeset
   976
            if enabled:
b852c34dc2b8 clonebundles: add warning if auto-generate is enabled without formats
Mathias De Mare <mathias.de_mare@nokia.com>
parents: 50735
diff changeset
   977
                if not targets:
b852c34dc2b8 clonebundles: add warning if auto-generate is enabled without formats
Mathias De Mare <mathias.de_mare@nokia.com>
parents: 50735
diff changeset
   978
                    repo.ui.warn(
b852c34dc2b8 clonebundles: add warning if auto-generate is enabled without formats
Mathias De Mare <mathias.de_mare@nokia.com>
parents: 50735
diff changeset
   979
                        _(
b852c34dc2b8 clonebundles: add warning if auto-generate is enabled without formats
Mathias De Mare <mathias.de_mare@nokia.com>
parents: 50735
diff changeset
   980
                            b'clone-bundle auto-generate enabled, '
b852c34dc2b8 clonebundles: add warning if auto-generate is enabled without formats
Mathias De Mare <mathias.de_mare@nokia.com>
parents: 50735
diff changeset
   981
                            b'but no formats specified: disabling generation\n'
b852c34dc2b8 clonebundles: add warning if auto-generate is enabled without formats
Mathias De Mare <mathias.de_mare@nokia.com>
parents: 50735
diff changeset
   982
                        )
b852c34dc2b8 clonebundles: add warning if auto-generate is enabled without formats
Mathias De Mare <mathias.de_mare@nokia.com>
parents: 50735
diff changeset
   983
                    )
b852c34dc2b8 clonebundles: add warning if auto-generate is enabled without formats
Mathias De Mare <mathias.de_mare@nokia.com>
parents: 50735
diff changeset
   984
                else:
b852c34dc2b8 clonebundles: add warning if auto-generate is enabled without formats
Mathias De Mare <mathias.de_mare@nokia.com>
parents: 50735
diff changeset
   985
                    tr.addpostclose(CAT_POSTCLOSE, make_auto_bundler(self))
50686
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   986
            return tr
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   987
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   988
        @localrepo.unfilteredmethod
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   989
        def clonebundles_lock(self, wait=True):
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   990
            '''Lock the repository file related to clone bundles'''
50928
d718eddf01d9 safehasattr: drop usage in favor of hasattr
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50776
diff changeset
   991
            if not hasattr(self, '_cb_lock_ref'):
50686
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   992
                self._cb_lock_ref = None
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   993
            l = self._currentlock(self._cb_lock_ref)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   994
            if l is not None:
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   995
                l.lock()
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   996
                return l
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   997
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   998
            l = self._lock(
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
   999
                vfs=self.vfs,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1000
                lockname=b"clonebundleslock",
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1001
                wait=wait,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1002
                releasefn=None,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1003
                acquirefn=None,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1004
                desc=_(b'repository %s') % self.origroot,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1005
            )
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1006
            self._cb_lock_ref = weakref.ref(l)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1007
            return l
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1008
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1009
    repo._wlockfreeprefix.add(AUTO_GEN_FILE)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1010
    repo._wlockfreeprefix.add(bundlecaches.CB_MANIFEST_FILE)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1011
    repo.__class__ = autobundlesrepo
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1012
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1013
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1014
@command(
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1015
    b'admin::clone-bundles-refresh',
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1016
    [
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1017
        (
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1018
            b'',
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1019
            b'background',
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1020
            False,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1021
            _(b'start bundle generation in the background'),
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1022
        ),
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1023
    ],
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1024
    b'',
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1025
)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1026
def cmd_admin_clone_bundles_refresh(
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1027
    ui,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1028
    repo: localrepo.localrepository,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1029
    background=False,
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1030
):
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1031
    """generate clone bundles according to the configuration
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1032
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1033
    This runs the logic for automatic generation, removing outdated bundles and
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1034
    generating new ones if necessary. See :hg:`help -e clone-bundles` for
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1035
    details about how to configure this feature.
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1036
    """
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1037
    debug = repo.ui.configbool(b'devel', b'debug.clonebundles')
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1038
    bundles = read_auto_gen(repo)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1039
    op_id = b"%d_acbr" % os.getpid()
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1040
    create, delete = auto_bundle_needed_actions(repo, bundles, op_id)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1041
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1042
    # if some bundles are scheduled for creation in the background, they will
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1043
    # deal with garbage collection too, so no need to synchroniously do it.
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1044
    #
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1045
    # However if no bundles are scheduled for creation, we need to explicitly do
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1046
    # it here.
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1047
    if not (background and create):
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1048
        # we clean up outdated bundles before generating new ones to keep the
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1049
        # last two versions of the bundle around for a while and avoid having to
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1050
        # deal with clients that just got served a manifest.
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1051
        for o in delete:
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1052
            delete_bundle(repo, o)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1053
        update_bundle_list(repo, del_bundles=delete)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1054
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1055
    if create:
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1056
        fpath = repo.vfs.makedirs(b'tmp-bundles')
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1057
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1058
    if background:
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1059
        for requested_bundle in create:
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1060
            start_one_bundle(repo, requested_bundle)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1061
    else:
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1062
        for requested_bundle in create:
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1063
            if debug:
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1064
                msg = b'clone-bundles: starting bundle generation: %s\n'
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1065
                repo.ui.write(msg % requested_bundle.bundle_type)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1066
            fname = requested_bundle.suggested_filename
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1067
            fpath = repo.vfs.join(b'tmp-bundles', fname)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1068
            generating_bundle = requested_bundle.generating(fpath)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1069
            update_bundle_list(repo, new_bundles=[generating_bundle])
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1070
            requested_bundle.generate_bundle(repo, fpath)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1071
            result = upload_bundle(repo, generating_bundle)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1072
            update_bundle_list(repo, new_bundles=[result])
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1073
            update_ondisk_manifest(repo)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1074
            cleanup_tmp_bundle(repo, generating_bundle)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1075
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1076
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1077
@command(b'admin::clone-bundles-clear', [], b'')
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1078
def cmd_admin_clone_bundles_clear(ui, repo: localrepo.localrepository):
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1079
    """remove existing clone bundle caches
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1080
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1081
    See `hg help admin::clone-bundles-refresh` for details on how to regenerate
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1082
    them.
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1083
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1084
    This command will only affect bundles currently available, it will not
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1085
    affect bundles being asynchronously generated.
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1086
    """
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1087
    bundles = read_auto_gen(repo)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1088
    delete = [b for b in bundles if b.ready]
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1089
    for o in delete:
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1090
        delete_bundle(repo, o)
a41eeb877d07 branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50645
diff changeset
  1091
    update_bundle_list(repo, del_bundles=delete)